The main difference is that Meltano is based on Singer while Airbyte is based on the Airbyte Protocol.
We don’t believe Singer is a good building block, because it requires a significant time investment from its users to compensate for the absence of centralized enforcement of the Singer protocol. Since it is not enforced, there is often no guarantee that any pair of Singer connectors are compatible. It defeats the point of a specification. All taps live in their own repo, and all contributions are made to address the contributor’s case, not the general use case. The lack of standard makes it very difficult to maintain all those connectors, and you end up with a majority of Singer taps being out of date.
Airbyte doesn’t have the same data protocol as Singer (but we are compatible). Our goal is to make building and maintaining new connectors a lot easier than it is with Singer, and therefore Meltano. That’s why we were able to ramp up our connectors (46 now) within just 5 months, while Meltano is focused on fixing the issues with Singer. We think it’s much harder to patch over Singer and reverse course on an abandonware project than it is to start from the ground up with these issues in mind. We wouldn’t be surprised if Meltano starts supporting Airbyte connectors in the future.
Does this imply airbyte only supports connectors that airbyte validates and integrates into the platform? Can I use an airbyte connector that lives in a repo on my private github?
This solves the problem of getting high quality connectors built, but how do you plan to maintain them? What if the original contributor falls off the face of the earth?
You can also use an external connector if you want to. This is a very valid use-case, especially if you connect internal APIs or private sources that wouldn't make sense for the community.
If the original contributor falls off the face of the earth, it is OK! That's the beauty of Open-Source. Another person who is using it can jump in. We can also jump in.
Comments
How do you differentiate from Meltano?
The main difference is that Meltano is based on Singer while Airbyte is based on the Airbyte Protocol.
We don’t believe Singer is a good building block, because it requires a significant time investment from its users to compensate for the absence of centralized enforcement of the Singer protocol. Since it is not enforced, there is often no guarantee that any pair of Singer connectors are compatible. It defeats the point of a specification. All taps live in their own repo, and all contributions are made to address the contributor’s case, not the general use case. The lack of standard makes it very difficult to maintain all those connectors, and you end up with a majority of Singer taps being out of date.
Airbyte doesn’t have the same data protocol as Singer (but we are compatible). Our goal is to make building and maintaining new connectors a lot easier than it is with Singer, and therefore Meltano. That’s why we were able to ramp up our connectors (46 now) within just 5 months, while Meltano is focused on fixing the issues with Singer. We think it’s much harder to patch over Singer and reverse course on an abandonware project than it is to start from the ground up with these issues in mind. We wouldn’t be surprised if Meltano starts supporting Airbyte connectors in the future.
We detail these differences here: https://docs.airbyte.io/faq/differences-with.../meltano-vs-a...
Does this imply airbyte only supports connectors that airbyte validates and integrates into the platform? Can I use an airbyte connector that lives in a repo on my private github?
This solves the problem of getting high quality connectors built, but how do you plan to maintain them? What if the original contributor falls off the face of the earth?
You can also use an external connector if you want to. This is a very valid use-case, especially if you connect internal APIs or private sources that wouldn't make sense for the community.
If the original contributor falls off the face of the earth, it is OK! That's the beauty of Open-Source. Another person who is using it can jump in. We can also jump in.