Dear researcher (I've been reading a lot of #cheminformatics fingerprint papers recently).
700,000 fingerprints is not large. N² methods like Butina clustering take less than 10 minutes. I'll identify 103K clusters from COCONUT natural products using Morgan2 with a threshold of 0.6:
$ curl -LO 'https://coconut.s3.uni-jena.de/prod/downloads/2026-11/coconut_sdf_2d-11-2026.zip'
$ unzip coconut_sdf_2d-11-2026.zip
$ rdkit2fps coconut_sdf_2d-11-2026.sdf -o coconut_sdf_2d-11-2026.fpb --morgan2 --id-tag identifier
.. error messages omitted ...
coconut_sdf_2d-11-2026.sdf: 100%|██████| 4.15G/4.15G [04:35<00:00, 15.1Mbytes/s]
$ chemfp butina --threshold 0.6 coconut_sdf_2d-11-2026.fpb -o clusters.txt
NxN: 100%|████████| 738789/738789 [08:54<00:00, 1382.32/s]
$ tail -1 clusters.txt
102987 CNP0127478.1 1 CNP0127478.1 1.0000000
(This machine has 32 GB of memory, which is not enough for a 0.5 threshold.)
Try chemfp? https://chemfp.com/
@dalke and I have CDK update that makes some of the fingerprints faster: https://github.com/cdk/cdk/pull/1376 (need to find time to finish that PR)
[WIP] Method to see if prefiltering can speed up SMARTS matching by egonw · Pull Request #1376 · cdk/cdk