Skip to content

Comment on Show HN: X to Elasticsearch Syncparent

Comments

sidiOP

1. and 2. It syncs deltas if you are either using Postgres (9.4 or above) or MongoDB. For other sources like MySQL or MSSQL, it needs to import the whole table each time. The tailing capability is DB dependent, for instance, in MongoDB's case - we use the oplog to do this.

The way it works is after the initial import if you are using the `--tail` option, it listens to further changes via oplog. More on setting oplog access over here - https://github.com/appbaseio/abc/blob/dev/docs/importer/adap...

For Postgres, similarly a replication role and a replication slot needs to be created for decoding the write ahead log. More on that here - https://github.com/appbaseio/abc/blob/dev/docs/importer/adap....

This seems like a good primer on replication slots - https://postgresqlspace.wordpress.com/2015/06/18/replication....

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.