its not about its shortcomings, its a database. your saying it replaces chef and opsworks. you're missing a few pieces to make that picture.
ie. how do i update the load balancer as i add capacity to my web app, how do i setup a new db instance and open up the firewall rules to an app on a different host. etc.
docker, and etcd are both nice pieces, but they don't make a complete picture. nor does fleet imo.
Don't forget Ansible; out of the box, it can perform pretty much any of the glue operations necessary to manage a set of containers and servers.
It's still pretty early, so much of the tooling will need more improvement before container-based infrastructure gets more mainstream (e.g. enterprise, smaller teams) uptake.
While I agree that etcd and config management systems do not solve the same problems, and esp the same way, the use of etcd by all of your services does enable your examples.
For instance, there is a tutorial on using vulcand as your http load balancer, which is driven by configuration in etcd. A tutorial on the CoreOS site shows you how to use this to cleanly deploy a new version of an app as new versions of a container, and to rotate them into the LB and to rotate the old ones out.
You can also connect a script that controls firewall rules based on etcd, its' changes would be reflected almost immediately, rather than the splayed 30-60min period of seemingly-randomly-applying-changes you typically see with config runs of tools like chef and puppet.
I've been trying to figure out where tools like chef, fabric, and docker/coreos/etcd/fleet will play together, what the boundaries will be, etc.. In situations where I'm using Docker and CoreOS, I don't expect to use chef, but I'm not sure it will be used for things like database servers, which we typically dedicate and tune hosts for.
It would be nice to see something like chef running on CoreOS for, say, user management so that people who want to talk to, say, fleet aren't required to all ssh as 'core'.
etcd is better compared to zookeeper (SOA orchestration, service discovery, etc) and etcd (after researching for the last two weeks for a production environment) is nowhere near as stable (ie non-beta/alpha) as zookeeper or Netflix's Eureka.
Configuration management (puppet, chef, salt, ansible) is a completely different beats than service discovery, health management, etc.
agreed, etcd is like zookeeper, their both databases that you write applications against. No! neither does service discovery and orchestration out of the box. The applications you write on top of them do.
Right, and you can achieve service discovery, orchestration, and convergence driven by something like zookeeper or etcd, rather than by bulky config runs.
While I enjoy working with Chef, have had some pretty reasonable times with Puppet, these full-run tools do have issues sometimes where you introduce a narrow bug in your user management or some other code, and all of a sudden you can't update some random conf file that happens after it. I've also seen tools like capistrano and fabric bastardized to allow this sort of precise updating, but lose the cohesion of typical config-managed systems.
With something like etcd, certain problems are solved by not relying on static configuration. That is the paradigm shift, and it is similar to Hadoop. When I've built Hadoop systems with puppet, we simply wrote out the same configs and files on every machine, then chose which services to start, and via zookeeper things like primary / secondary failover take care of themselves.
Comments
its not about its shortcomings, its a database. your saying it replaces chef and opsworks. you're missing a few pieces to make that picture.
ie. how do i update the load balancer as i add capacity to my web app, how do i setup a new db instance and open up the firewall rules to an app on a different host. etc.
docker, and etcd are both nice pieces, but they don't make a complete picture. nor does fleet imo.
Don't forget Ansible; out of the box, it can perform pretty much any of the glue operations necessary to manage a set of containers and servers.
It's still pretty early, so much of the tooling will need more improvement before container-based infrastructure gets more mainstream (e.g. enterprise, smaller teams) uptake.
While I agree that etcd and config management systems do not solve the same problems, and esp the same way, the use of etcd by all of your services does enable your examples.
For instance, there is a tutorial on using vulcand as your http load balancer, which is driven by configuration in etcd. A tutorial on the CoreOS site shows you how to use this to cleanly deploy a new version of an app as new versions of a container, and to rotate them into the LB and to rotate the old ones out.
You can also connect a script that controls firewall rules based on etcd, its' changes would be reflected almost immediately, rather than the splayed 30-60min period of seemingly-randomly-applying-changes you typically see with config runs of tools like chef and puppet.
I've been trying to figure out where tools like chef, fabric, and docker/coreos/etcd/fleet will play together, what the boundaries will be, etc.. In situations where I'm using Docker and CoreOS, I don't expect to use chef, but I'm not sure it will be used for things like database servers, which we typically dedicate and tune hosts for.
It would be nice to see something like chef running on CoreOS for, say, user management so that people who want to talk to, say, fleet aren't required to all ssh as 'core'.
etcd is better compared to zookeeper (SOA orchestration, service discovery, etc) and etcd (after researching for the last two weeks for a production environment) is nowhere near as stable (ie non-beta/alpha) as zookeeper or Netflix's Eureka.
Configuration management (puppet, chef, salt, ansible) is a completely different beats than service discovery, health management, etc.
agreed, etcd is like zookeeper, their both databases that you write applications against. No! neither does service discovery and orchestration out of the box. The applications you write on top of them do.
Right, and you can achieve service discovery, orchestration, and convergence driven by something like zookeeper or etcd, rather than by bulky config runs.
While I enjoy working with Chef, have had some pretty reasonable times with Puppet, these full-run tools do have issues sometimes where you introduce a narrow bug in your user management or some other code, and all of a sudden you can't update some random conf file that happens after it. I've also seen tools like capistrano and fabric bastardized to allow this sort of precise updating, but lose the cohesion of typical config-managed systems.
With something like etcd, certain problems are solved by not relying on static configuration. That is the paradigm shift, and it is similar to Hadoop. When I've built Hadoop systems with puppet, we simply wrote out the same configs and files on every machine, then chose which services to start, and via zookeeper things like primary / secondary failover take care of themselves.
+1
If you want to do orchestration of both the infra and the app, you need things like opsworks.
One simple solution would be to add etcd configuration items as a CloudFormation/Heat custom resource type.