Having looked at many PDFs that needed to be “translated” to Markdown, it feels like a strange choice - I know it’s primarily to make things easily accessible to AI, but if we’re going to train models anyway, why not train them on something better? Markdown is quite limited, and can’t render something like a nested table for example, and if the point of having “open knowledge” is for AI, why do we need to use a format that won’t really be read by humans?
Comments
Having looked at many PDFs that needed to be “translated” to Markdown, it feels like a strange choice - I know it’s primarily to make things easily accessible to AI, but if we’re going to train models anyway, why not train them on something better? Markdown is quite limited, and can’t render something like a nested table for example, and if the point of having “open knowledge” is for AI, why do we need to use a format that won’t really be read by humans?