Comment on Ask HN: I have many PDFs – what is the best local way to leverage AI for search?parentComments−nl2yDoes the llamaindex PDF indexer correctly deal with multi-column PDFs? Most I've seen don't, and you get very odd results because of this.−rspoerri2yi've made quite good conversions from pdf to markdown with https://github.com/VikParuchuri/marker . it's slow but worth a shot. Markdown should be easily parseable by a rag.i'm trying to get a similar system setup on my computer.−nl2yThis looks worth exploring, so thanks. The author has done a bunch of work beyond what PyMuPDF does on multicolumn layouts.−pierre2yLocally you can choose pypdf or mupdf wich are good but not perfect. If you can send your data online llamaparse is quite good.−j452yPulling the text out of the PDFs correctly and independently is correct.
Comments
Does the llamaindex PDF indexer correctly deal with multi-column PDFs? Most I've seen don't, and you get very odd results because of this.
i've made quite good conversions from pdf to markdown with https://github.com/VikParuchuri/marker . it's slow but worth a shot. Markdown should be easily parseable by a rag.
i'm trying to get a similar system setup on my computer.
This looks worth exploring, so thanks. The author has done a bunch of work beyond what PyMuPDF does on multicolumn layouts.
Locally you can choose pypdf or mupdf wich are good but not perfect. If you can send your data online llamaparse is quite good.
Pulling the text out of the PDFs correctly and independently is correct.