I currently parse the TOC for chapter markers, then walk the spine between markers to ensure everything is captured regardless of what the TOC says. I think it is in the spirit of what epub readers do and in line with the spec.
The eval agents perform the extraction and run a comparison of chapter/word count against the epub original. They also parse the extracted text to look for inconsistencies. The quality criteria have been built up iteratively and issues backed out into unit tests. The eval runs aren't part of my standard unit tests but a one-off, heavy lift process that I run periodically against the epub collection I already had and a bunch I built up for this project. I've also just done a ton of manual validation. There may still be edge cases.
Comments
I currently parse the TOC for chapter markers, then walk the spine between markers to ensure everything is captured regardless of what the TOC says. I think it is in the spirit of what epub readers do and in line with the spec.
The eval agents perform the extraction and run a comparison of chapter/word count against the epub original. They also parse the extracted text to look for inconsistencies. The quality criteria have been built up iteratively and issues backed out into unit tests. The eval runs aren't part of my standard unit tests but a one-off, heavy lift process that I run periodically against the epub collection I already had and a bunch I built up for this project. I've also just done a ton of manual validation. There may still be edge cases.