Don't select Pictures with the recursive option. The import will run in a single thread. Instead select dir1,dir2,dir3...dirn all at once. Then the import will run in as many threads as are available. The number of available threads Darktable can use is set by the config menu.
You can select more directories than you have threads available. I saw several orders of magnitude speedup in the import. All those threads kept the CPU running. All that running CPU kept the IO flowing.
+ Advice: Work in two stages. Get all the images locally first. Download from Dropbox and Facebook onto that big disk. Then import them. Once they are local, you can manage them, back them up, etc.
Comments
I have 37,000+ images in my Darktable database. I rebuilt it from scratch last month. I hooked up each disk and let it chug away.
+ Tip: If you have:
Don't select Pictures with the recursive option. The import will run in a single thread. Instead select dir1,dir2,dir3...dirn all at once. Then the import will run in as many threads as are available. The number of available threads Darktable can use is set by the config menu.You can select more directories than you have threads available. I saw several orders of magnitude speedup in the import. All those threads kept the CPU running. All that running CPU kept the IO flowing.
+ Advice: Work in two stages. Get all the images locally first. Download from Dropbox and Facebook onto that big disk. Then import them. Once they are local, you can manage them, back them up, etc.