Warning: fastdup create() without work_dir argument, output is stored in a folder named work_dir in your current working path.
FastDup Software, (C) copyright 2022 Dr. Amir Alush and Dr. Danny Bickson.
ad88 88
d8" ,d 88
88 88 88
MM88MMM ,adPPYYba, ,adPPYba, MM88MMM ,adPPYb,88 88 88 8b,dPPYba,
88 "" `Y8 I8[ "" 88 a8" `Y88 88 88 88P' "8a
88 ,adPPPPP88 `"Y8ba, 88 8b 88 88 88 88 d8
88 88, ,88 aa ]8I 88, "8a, ,d88 "8a, ,a88 88b, ,a8"
88 `"8bbdP"Y8 `"YbbdP"' "Y888 `"8bbdP"Y8 `"YbbdP'Y8 88`YbbdP"'
88
88
2023-09-19 21:40:45 [INFO] Going to loop over dir /tmp/tmpjb5y9uvf.csv
2023-09-19 21:40:45 [INFO] Found total 100000 images to run on, 100000 train, 0 test, name list 100000, counter 100000
2023-09-19 21:44:25 [INFO] Found total 100000 images to run onmated: 0 Minutes
Finished histogram 38.760
Finished bucket sort 38.971
2023-09-19 21:44:45 [INFO] 20456) Finished write_index() NN model
2023-09-19 21:44:45 [INFO] Stored nn model index file work_dir/nnf.index
2023-09-19 21:44:56 [INFO] Total time took 251595 ms
2023-09-19 21:44:56 [INFO] Found a total of 40 fully identical images (d>0.990), which are 0.02 % of total graph edges
2023-09-19 21:44:56 [INFO] Found a total of 0 nearly identical images(d>0.980), which are 0.00 % of total graph edges
2023-09-19 21:44:56 [INFO] Found a total of 11083 above threshold images (d>0.900), which are 5.54 % of total graph edges
2023-09-19 21:44:56 [INFO] Found a total of 10001 outlier images (d<0.050), which are 5.00 % of total graph edges
2023-09-19 21:44:56 [INFO] Min distance found 0.601 max distance 1.000
2023-09-19 21:44:56 [INFO] Running connected components for ccthreshold 0.960000
.0
########################################################################################
Dataset Analysis Summary:
Dataset contains 100000 images
Valid images are 100.00% (100,000) of the data, invalid are 0.00% (0) of the data
Similarity: 0.05% (46) belong to 1 similarity clusters (components).
99.95% (99,954) images do not belong to any similarity cluster.
Largest cluster has 4 (0.00%) images.
For a detailed analysis, use `.connected_components()`
(similarity threshold used is 0.9, connected component threshold used is 0.96).
Outliers: 6.34% (6,344) of images are possible outliers, and fall in the bottom 5.00% of similarity values.
For a detailed list of outliers, use `.outliers()`.
########################################################################################
Would you like to see awesome visualizations for some of the most popular academic datasets?
Click here to see and learn more: https://app.visual-layer.com/vl-datasets?utm_source=fastdup
########################################################################################
