Skip to content

Comment on Lucene: The Good Partsparent

Comments

(Author here.) As others have pointed out, I am here referring to SQL as a broad term for "SQL RDBMSes". When I say that it is not a good blob or document storage system, what I mean is this: You can plop unstructured data into a SQL RDBMS using something like JSON serialization, but it's not generally a good idea. Document storage requires flexible schemas, so SQL schemas are also not generally a great idea -- you end up with tables with lots of nullable fields that are usually empty.

As an example, imagine a single database where you want to store actual desktop documents, such as the formats supported by Apache Tika: https://tika.apache.org/1.8/formats.html -- If you try to model this using a SQL schema, you'll likely be in for a world of pain. From a UX standpoint, a user just wants to "search across all documents", but you have hundreds of heterogeneous types with varying degrees of field-level compatibility.

What do you mean by "not generally a good idea"? In the past I've had little cause to complain about using XML fields to store and search semi-structured data in MS SQL Server.

I have read that it doesn't scale out as nicely as document stores do, and knowing SQL Server I don't have trouble believing that. Personally I'm a very long way away from needing to worry about scale out[1] in the applications I use a DBMS for, though, so that's never really kept me up at night.

Word I've heard on the street is that the story's similar for PostgreSQL.

[1]: http://yourdatafitsinram.com

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.