Warning: fastdup create() without work_dir argument, output is stored in a folder named work_dir in your current working path.
FastDup Software, (C) copyright 2022 Dr. Amir Alush and Dr. Danny Bickson.
ad88 88
d8" ,d 88
88 88 88
MM88MMM ,adPPYYba, ,adPPYba, MM88MMM ,adPPYb,88 88 88 8b,dPPYba,
88 "" `Y8 I8[ "" 88 a8" `Y88 88 88 88P' "8a
88 ,adPPPPP88 `"Y8ba, 88 8b 88 88 88 88 d8
88 88, ,88 aa ]8I 88, "8a, ,d88 "8a, ,a88 88b, ,a8"
88 `"8bbdP"Y8 `"YbbdP"' "Y888 `"8bbdP"Y8 `"YbbdP'Y8 88`YbbdP"'
88
88
2023-09-26 14:16:12 [INFO] Going to loop over dir /tmp/tmpe_pnff4j.csv
2023-09-26 14:16:12 [INFO] Found total 30607 images to run on, 30607 train, 0 test, name list 30607, counter 30607
2023-09-26 14:17:14 [INFO] Found total 30607 images to run onimated: 0 Minutes
Finished histogram 8.575
Finished bucket sort 8.638
2023-09-26 14:17:16 [INFO] 2184) Finished write_index() NN model
2023-09-26 14:17:16 [INFO] Stored nn model index file work_dir/nnf.index
2023-09-26 14:17:17 [INFO] Total time took 65661 ms
2023-09-26 14:17:17 [INFO] Found a total of 180 fully identical images (d>0.990), which are 0.29 % of total graph edges
2023-09-26 14:17:17 [INFO] Found a total of 86 nearly identical images(d>0.980), which are 0.14 % of total graph edges
2023-09-26 14:17:17 [INFO] Found a total of 4480 above threshold images (d>0.900), which are 7.32 % of total graph edges
2023-09-26 14:17:17 [INFO] Found a total of 3061 outlier images (d<0.050), which are 5.00 % of total graph edges
2023-09-26 14:17:17 [INFO] Min distance found 0.457 max distance 1.000
2023-09-26 14:17:17 [INFO] Running connected components for ccthreshold 0.960000
.0
########################################################################################
Dataset Analysis Summary:
Dataset contains 30607 images
Valid images are 100.00% (30,607) of the data, invalid are 0.00% (0) of the data
Similarity: 1.77% (543) belong to 9 similarity clusters (components).
98.23% (30,064) images do not belong to any similarity cluster.
Largest cluster has 60 (0.20%) images.
For a detailed analysis, use `.connected_components()`
(similarity threshold used is 0.9, connected component threshold used is 0.96).
Outliers: 6.48% (1,983) of images are possible outliers, and fall in the bottom 5.00% of similarity values.
For a detailed list of outliers, use `.outliers()`.
########################################################################################
Would you like to see awesome visualizations for some of the most popular academic datasets?
Click here to see and learn more: https://app.visual-layer.com/vl-datasets?utm_source=fastdup
########################################################################################