SenseiDB looks really impressive. We were actually thinking about the same problem when we built ThriftDB (http://www.thriftdb.com) for Octopart so it's especially interesting to see LinkedIn's implementation.
Looking through the documentation a couple of things come to mind:
* Does SenseiDB support nested data structures?
* What happens when you modify the schema? Can you add/delete/modify attributes?
re: nested data structure is in the roadmap, but no target date yet. There are 2 ways of implementing this:
1) pre-pend field name reflecting the nesting, much like elasticsearch
2) encoding payloads in the term list indicating nested nature.
1) is straightforward, but can be restrictive if you want to in-nest faceting.
We are still debating on the route to take.
As for schema changes, we should document that better: certain schema changes simply requires you bouncing the node, but if there are data integrity changes, you would need to reindex. We will work on more documentation on the specifics.
Comments
SenseiDB looks really impressive. We were actually thinking about the same problem when we built ThriftDB (http://www.thriftdb.com) for Octopart so it's especially interesting to see LinkedIn's implementation.
Looking through the documentation a couple of things come to mind:
* Does SenseiDB support nested data structures?
* What happens when you modify the schema? Can you add/delete/modify attributes?
re: nested data structure is in the roadmap, but no target date yet. There are 2 ways of implementing this: 1) pre-pend field name reflecting the nesting, much like elasticsearch 2) encoding payloads in the term list indicating nested nature.
1) is straightforward, but can be restrictive if you want to in-nest faceting.
We are still debating on the route to take.
As for schema changes, we should document that better: certain schema changes simply requires you bouncing the node, but if there are data integrity changes, you would need to reindex. We will work on more documentation on the specifics.
If I change the faceting does bouncing the nodes cause a re-index automatically or is it more involved?
bouncing node will start reindexing from the last committed version on the disk. If the index is up to date, no re-indexing will be triggered.