I'd like to make a pitch for Openlibrary.org the free online library from Internet Archive that includes a fulltext search of millions of books.
I've been volunteering with them on and off for several years and it's always a lovely experience. Their backend is python and frontend mostly from python templates and some Vue for librarian stuff.
Every Tuesday they have a call on Zoom that everyone is welcome to join to share what they're working on, ask for help, and generally chat a bit. It's a great time.
Depending on what you're interested in there's a lot to do from helping build import pipelines for more book entries, writing bots to cleanup data, Performance improvements, better documenting public APIs, etc
I'm currently slowly working on a wikidata integration for their authors page. We also could use some help upgrading to Vue 3, mentors for Google summer of code would be helpful, find of ML projects needing help, moving away from old jQuery libraries, etc.
Cool project. Where is the data and how was it acquired / what's the provenance? I found this:
Data: We have a bunch of catalog data and fulltext acquired from various sources, either sitting in the Archive or to be uploaded to there. I think the acquisition processes (including web crawling scripts for some of the data) is outside the scope of an Open Library software install. There are a bunch of additional scripts to make the stuff usable in openlibrary and these need to be documented. These include TDB Conversion Scripts written by dbg, and (for OCA fulltext) Archive Spidering and Solr Importing scripts written by phr.
edit: I should add the question: does openlibrary support fulltext search of actual books or fulltext search of the metadata?
If you would like to see the data that's currently in Open library, there are dumps published every month or so. Where the data comes from is either partners or one off projects to scrape from public sources or sometimes just people adding what they like.
Fulltext search of actual books! And also metadata.
If you're serious about getting involved I'd say the best thing to do it just come to the meeting if you can. The GitHub has a lot of things that without context can be hard to pickup. Most of the easy stuff is on the JS side.
Comments
I'd like to make a pitch for Openlibrary.org the free online library from Internet Archive that includes a fulltext search of millions of books.
I've been volunteering with them on and off for several years and it's always a lovely experience. Their backend is python and frontend mostly from python templates and some Vue for librarian stuff.
Every Tuesday they have a call on Zoom that everyone is welcome to join to share what they're working on, ask for help, and generally chat a bit. It's a great time.
Depending on what you're interested in there's a lot to do from helping build import pipelines for more book entries, writing bots to cleanup data, Performance improvements, better documenting public APIs, etc
I'm currently slowly working on a wikidata integration for their authors page. We also could use some help upgrading to Vue 3, mentors for Google summer of code would be helpful, find of ML projects needing help, moving away from old jQuery libraries, etc.
They can be quite responsive to PRs too like I blogged about here: https://blog.rayberger.org/idea-to-merged-in-less-than-30-mi...
For example, here's a small issue that could use some help on the python side: https://github.com/internetarchive/openlibrary/issues/8928
Cool project. Where is the data and how was it acquired / what's the provenance? I found this:
edit: I should add the question: does openlibrary support fulltext search of actual books or fulltext search of the metadata?
If you would like to see the data that's currently in Open library, there are dumps published every month or so. Where the data comes from is either partners or one off projects to scrape from public sources or sometimes just people adding what they like.
Fulltext search of actual books! And also metadata.
Check it out:
https://openlibrary.org/search/inside?q=hackernews
If you're serious about getting involved I'd say the best thing to do it just come to the meeting if you can. The GitHub has a lot of things that without context can be hard to pickup. Most of the easy stuff is on the JS side.