I am personally inclined to lump this in with the other enterprisey buzzwords that are generally safe to avoid. From my perspective, the 'data lake' is the actual collection of business systems and your ability to extract useful information from them.
Trying to come up with some unified standard or common API for the extraction, transformation and implementation of useful data from a heterogenous collection of systems sounds like a problematic task at best.
The other, more likely, interpretation of 'data lake' is that it is the staging ground between your ability to do the above stated activity and other downstream systems interested in the data. If the idea is that you are creating the actual normalization layer, I feel like this is still more in the realm of SQL/ETL, as there really isn't any other direction to move that would reduce your entropy in a valuable way (relative to your time invested).
SQLite or Postgres is usually the right choice. This simple rule can help you avoid a lot of pain. Once you have convinced everyone that Postgres is to be used, the only other real barriers are your ability to get a data transport to each business system and the authoring of some SQL scripts. The workload of building a SQL representation of any particular business system is fairly predictable once you break it down to entity-relationship abstractions. Also, using a language with powerful class/object, serialization and database support such as C# or Java can cut your workload by orders of magnitude if you choose a SQL architecture. In C# for instance, you can just write POCOs and use Entity Framework to build out all of the SQL for you. This is not the most performant option by a long shot, but it can get you going incredibly quickly on a first iteration.
Comments
I am personally inclined to lump this in with the other enterprisey buzzwords that are generally safe to avoid. From my perspective, the 'data lake' is the actual collection of business systems and your ability to extract useful information from them.
Trying to come up with some unified standard or common API for the extraction, transformation and implementation of useful data from a heterogenous collection of systems sounds like a problematic task at best.
The other, more likely, interpretation of 'data lake' is that it is the staging ground between your ability to do the above stated activity and other downstream systems interested in the data. If the idea is that you are creating the actual normalization layer, I feel like this is still more in the realm of SQL/ETL, as there really isn't any other direction to move that would reduce your entropy in a valuable way (relative to your time invested).
SQLite or Postgres is usually the right choice. This simple rule can help you avoid a lot of pain. Once you have convinced everyone that Postgres is to be used, the only other real barriers are your ability to get a data transport to each business system and the authoring of some SQL scripts. The workload of building a SQL representation of any particular business system is fairly predictable once you break it down to entity-relationship abstractions. Also, using a language with powerful class/object, serialization and database support such as C# or Java can cut your workload by orders of magnitude if you choose a SQL architecture. In C# for instance, you can just write POCOs and use Entity Framework to build out all of the SQL for you. This is not the most performant option by a long shot, but it can get you going incredibly quickly on a first iteration.
... the data has already disappeared.
Systems get shut down and replaced. Operational systems may discard history.
By the time you get a fully operational data warehouse set up, it may be too late to preserve the data.