OpenStack 2014 somehow reminds me a lot of OMG/CORBA 1993. It's interesting to watch the industry repeat itself.
Checklist:
- Anybody but "x" club, where X is the market leader. Openstack has vascillated between VMware and AWS here, CORBA had Microsoft.
- Large investments advertised as a penis swinging contest between HP, IBM, and others
- Chasing aging technology architecture as a panacea instead of building the next generation.
CORBA was chasing distributed objects, but the Web and REST was what really was going to matter. Today the big guys are chasing the IaaS model when we have a new generation of platform and application clouds (Mesos, YARN, CoreOS, Docker, CF, Asgard) growing out there.
The vendors will of course try to recoup their investment through reference architectures and "best practices" that tell customers they should have a multi-layer SaaS or PaaS running on an IaaS, even though lightweight alternatives like CoreOS will chug along doing nothing of the sort.
This is similar to how CORBA's IIOP was promoted to be the way to do "real work" with C++ or Enterprise JavaBeans in your middle tier, but this "Web" HTTP thing can be relegated to the front tier. I know fixed income systems around 1999 that literally wouldn't use HTTP for their blotter updates, with the lead devs insisting such events needed to be pushed over IIOP even though the front end was an IE control in a VBX. (this was shortly before TIBCO had effectively seized the market data industry)
- persistent flame wars about governance and architecture of said standard on the social media of the day
This is not to say OpenStack is bad. There is value, potentially a lot of value, as there was with CORBA. That's all case-by-case. Just saying that vendor love fests rarely indicate a true revolution and most often are just an orchestrated marketing game by the current losers.
I'm a cloud architect (a person who assembles cloud computing systems from components). I haven't used OpenStack; I've looked at it and while the concept is fine, the implementation is pretty underwhelming. Corps just want it because if widely adopted, it will reduce their cost and increase their profit margins.
Don't forget the SOAP era; it also resembles CORBA. There was a whole region of time (~2002-2005) where Microsoft and IBM jumped on SOAP. They promoted the hell out of their stacks, and then, after everybody realized SOAP was a turd, put the projects in deprecated mode.
The next step in evolution was JSON payloads over HTTP, with methods defined using REST. This model has more or less survived for 10 years now, and is a community-created (rather than top-down-imposed by a corporation) system that has lots of experience under its belt. I expect only marginal changes at this point: move towards a more compact encoding (like BSON) and a bit more standardization on where the RPC method is defined. I expect REST itself, being so hard to understand, will only be used by people who really insist on getting their API surfaces really RESTy.
SOAP itself was Microsoft's attempt to wrest momentum away from Java, and get people to pay attention to .NET, which modestly succeeded.
It also was a the final turd that broke bullshit mountain's forces: ultimately, developers had to figure out what mattered more to the Web's success: was it tagged data (HTML/XML), and we could preserve this distributed component/object conceptual model? Or was it the combination of URI+HTTP with its uniform interface and hypermedia? The REST debate took a long time (through 2007) to answer that, even though in practice with JSON we just pretend to adopt the full architecture.
I think the next revolution in REST would be if a new media type and associated toolkits/libraries come out that helps people build simpler and evolvable systems than JSON does. There are a variety of experiments out there, but as you imply, it's hard work.
I too am a "self-declared" cloud architect...mainly Azure and Windows-based platforms for Fortune 500 companies that want MS-based cloud architectures for Exchange, SharePoint, MSSQL, and Dynamics. Sorry, but other than setting it up in a lab for my own interest and tinkering, I have not had a Client request OpenStack or any OSS if they are implementing a Microsoft technology-based cloud or hybrid-cloud architecture. Not to say F-500 companies don't use Open Source, but OSS is typically not utilized in every cloud or on-prem environment.
You're reasoning by analogy, but the analogy between CORBA and OpenStack is flawed.
CORBA is mainly a standard. A standard only succeeds if most people adopt it.
OpenStack is mainly a set of open source software. Users can pick and choose the pieces to use in their system. If "OpenStack Governance" decides that Software X should be used for job Y, but something better (and open source) comes along, people can just switch to the better software. In other words, competition can happen at the component level and it can come from the grassroots (without permission from the big wigs). It's much different from everyone having to agree on one standard.
I don't need to tell you that there are many thriving open source software ecosystems. Make an analogy with one of them and OpenStack makes much more sense (e.g. Linux and its packages, Python and its packages, WordPress and its themes/plugins).
The analogy is not 1:1, but consider a few observations.
Firstly, OpenStack is not a thriving open source community in the usual sense. It has succeeded in creating a vendor ecosystem, it has not succeeded in creating a large volume of customer successes. An OpenStack deployment is notoriously hard to get right in practice and hundreds of millions of dollars have been squandered to date on failed projects. In this way it has a lot of similarity to CORBA, which was a vendor love fest far before customer adoption, which took years of failures before the initial successes.
Secondly, you may want to look into how the OMG operated (and still operates), which not as a top-down standards body, it is more of a community facilitation organization. OMG set out RFI/RFPs for set of component/facility/service-level specifications with competition in the form of RFP responses that are voted on by the membership and then consolidated. The OMG would have been much better served by having an open source reference implementation for the various specs (avoiding many of the design flaws and ratholes the OMG ran into).
That said, once the OMG picked a recommendation for an area, customers or vendors tended to use that or implement that, and ignore alternatives. This is not much different from the OpenStack Foundation, which can and will control the direction of the OpenStack software for the (debatable) benefits of its leadership and members. The startup community is risk tolerant enough to just "try something else" that's open source without a support contract, sure, but in my experience most enterprises won't mix and match things that aren't "approved" or "recommended" by the mothership and supported by the likes of EMC, VMware, HP or IBM.
Third, consider the popularity of RedHat Linux. It is a billion dollar company because they are the de facto go-to vendor for a Linux distribution, patch network, and support contract. They curate what packages and kernel patches are included in the distribution, but generally rely on Linus' governance for the kernel and do not deviate strongly from it. If Linus doesn't want it in the main kernel, it's an uphill climb to expect to see it eventually in RHEL.
I expect a similar pattern with OpenStack, with the caveat the governance itself is not user-centered, it is vendor-centered, which is not a good sign.
Some of what you said applies, but I think the analogy doesn't go that far. Just to point out / correct some things: Openstack is not very anti-VMware. There are even backend drivers that allow you to manage VMware environment using openstack as a frontend. Never tried it in practice, but some company bothered to write / contribute it, so it must be working.
Application deployment via docker - openstack supports both docker and libvirt/lxc. Unless you meant technologies running in the management layer - in that case I think coreos and similar projects aim for a completely different use case, not just architecture. Openstack works well with not trusted, heterogeneous environments and separation of tenants. If you're running your own cluster, with your own in-house produced stack and control over processes, coreos may be a better idea.
Or maybe you even want to deploy coreos cluster on real hardware automatically? That's also something openstack could do using baremetal backend.
Basically what I'm trying to say is: openstack is a framework more than a single solution. It may not work for every use case, or it may require some specific config to be useful. But it shouldn't limit the architecture beyond: it's got an api.
Technically speaking, I agree that it's a not a single solution and can be flexible. That said, it fundamentally presumes VMs, attachable disks, and networks as "the objects that matters" for your cloud. It's a good assumption for now, but maybe not for 5 years from now.
My post was mostly about the funding and political context that is driving OpenStack momentum, and its similarity to past attempts for vendors to gang up and try to take on the market leader (AWS). The community is all about the vendors, currently. Users are growing, but still the minority. And that's a warning sign, because it may mean the users really just don't care as much as the investors in OpenStack think they will.
Comments
OpenStack 2014 somehow reminds me a lot of OMG/CORBA 1993. It's interesting to watch the industry repeat itself.
Checklist:
- Anybody but "x" club, where X is the market leader. Openstack has vascillated between VMware and AWS here, CORBA had Microsoft.
- Large investments advertised as a penis swinging contest between HP, IBM, and others
- Chasing aging technology architecture as a panacea instead of building the next generation.
CORBA was chasing distributed objects, but the Web and REST was what really was going to matter. Today the big guys are chasing the IaaS model when we have a new generation of platform and application clouds (Mesos, YARN, CoreOS, Docker, CF, Asgard) growing out there.
The vendors will of course try to recoup their investment through reference architectures and "best practices" that tell customers they should have a multi-layer SaaS or PaaS running on an IaaS, even though lightweight alternatives like CoreOS will chug along doing nothing of the sort.
This is similar to how CORBA's IIOP was promoted to be the way to do "real work" with C++ or Enterprise JavaBeans in your middle tier, but this "Web" HTTP thing can be relegated to the front tier. I know fixed income systems around 1999 that literally wouldn't use HTTP for their blotter updates, with the lead devs insisting such events needed to be pushed over IIOP even though the front end was an IE control in a VBX. (this was shortly before TIBCO had effectively seized the market data industry)
- persistent flame wars about governance and architecture of said standard on the social media of the day
- Vendor adoption drastically outpacing customer adoption , developing a "buying customers" effect
This is not to say OpenStack is bad. There is value, potentially a lot of value, as there was with CORBA. That's all case-by-case. Just saying that vendor love fests rarely indicate a true revolution and most often are just an orchestrated marketing game by the current losers.
I'm a cloud architect (a person who assembles cloud computing systems from components). I haven't used OpenStack; I've looked at it and while the concept is fine, the implementation is pretty underwhelming. Corps just want it because if widely adopted, it will reduce their cost and increase their profit margins.
Don't forget the SOAP era; it also resembles CORBA. There was a whole region of time (~2002-2005) where Microsoft and IBM jumped on SOAP. They promoted the hell out of their stacks, and then, after everybody realized SOAP was a turd, put the projects in deprecated mode.
http://www.ianfoster.org/wordpress/wp-content/uploads/2014/0... (page 23).
The next step in evolution was JSON payloads over HTTP, with methods defined using REST. This model has more or less survived for 10 years now, and is a community-created (rather than top-down-imposed by a corporation) system that has lots of experience under its belt. I expect only marginal changes at this point: move towards a more compact encoding (like BSON) and a bit more standardization on where the RPC method is defined. I expect REST itself, being so hard to understand, will only be used by people who really insist on getting their API surfaces really RESTy.
I concur.
SOAP itself was Microsoft's attempt to wrest momentum away from Java, and get people to pay attention to .NET, which modestly succeeded.
It also was a the final turd that broke bullshit mountain's forces: ultimately, developers had to figure out what mattered more to the Web's success: was it tagged data (HTML/XML), and we could preserve this distributed component/object conceptual model? Or was it the combination of URI+HTTP with its uniform interface and hypermedia? The REST debate took a long time (through 2007) to answer that, even though in practice with JSON we just pretend to adopt the full architecture.
I think the next revolution in REST would be if a new media type and associated toolkits/libraries come out that helps people build simpler and evolvable systems than JSON does. There are a variety of experiments out there, but as you imply, it's hard work.
"concept is fine, the implementation is pretty underwhelming", care to expand?
I find it odd that a self-declared cloud architect hasn't used what is easily the largest growing IAAS project, free or commercial.
I have access to other clouds which provide the openstack functionality, without me having to adopt openstack.
I don't think we have accurate visibiliy into EC2 to compare its rate of growth compared to OpenStack.
OpenStack isn't IaaS. It's a PaaS middleware that abstracts the underlying IaaS.
I too am a "self-declared" cloud architect...mainly Azure and Windows-based platforms for Fortune 500 companies that want MS-based cloud architectures for Exchange, SharePoint, MSSQL, and Dynamics. Sorry, but other than setting it up in a lab for my own interest and tinkering, I have not had a Client request OpenStack or any OSS if they are implementing a Microsoft technology-based cloud or hybrid-cloud architecture. Not to say F-500 companies don't use Open Source, but OSS is typically not utilized in every cloud or on-prem environment.
Edit: a bit more clarity
You're reasoning by analogy, but the analogy between CORBA and OpenStack is flawed.
CORBA is mainly a standard. A standard only succeeds if most people adopt it.
OpenStack is mainly a set of open source software. Users can pick and choose the pieces to use in their system. If "OpenStack Governance" decides that Software X should be used for job Y, but something better (and open source) comes along, people can just switch to the better software. In other words, competition can happen at the component level and it can come from the grassroots (without permission from the big wigs). It's much different from everyone having to agree on one standard.
I don't need to tell you that there are many thriving open source software ecosystems. Make an analogy with one of them and OpenStack makes much more sense (e.g. Linux and its packages, Python and its packages, WordPress and its themes/plugins).
The analogy is not 1:1, but consider a few observations.
Firstly, OpenStack is not a thriving open source community in the usual sense. It has succeeded in creating a vendor ecosystem, it has not succeeded in creating a large volume of customer successes. An OpenStack deployment is notoriously hard to get right in practice and hundreds of millions of dollars have been squandered to date on failed projects. In this way it has a lot of similarity to CORBA, which was a vendor love fest far before customer adoption, which took years of failures before the initial successes.
Secondly, you may want to look into how the OMG operated (and still operates), which not as a top-down standards body, it is more of a community facilitation organization. OMG set out RFI/RFPs for set of component/facility/service-level specifications with competition in the form of RFP responses that are voted on by the membership and then consolidated. The OMG would have been much better served by having an open source reference implementation for the various specs (avoiding many of the design flaws and ratholes the OMG ran into).
That said, once the OMG picked a recommendation for an area, customers or vendors tended to use that or implement that, and ignore alternatives. This is not much different from the OpenStack Foundation, which can and will control the direction of the OpenStack software for the (debatable) benefits of its leadership and members. The startup community is risk tolerant enough to just "try something else" that's open source without a support contract, sure, but in my experience most enterprises won't mix and match things that aren't "approved" or "recommended" by the mothership and supported by the likes of EMC, VMware, HP or IBM.
Third, consider the popularity of RedHat Linux. It is a billion dollar company because they are the de facto go-to vendor for a Linux distribution, patch network, and support contract. They curate what packages and kernel patches are included in the distribution, but generally rely on Linus' governance for the kernel and do not deviate strongly from it. If Linus doesn't want it in the main kernel, it's an uphill climb to expect to see it eventually in RHEL.
I expect a similar pattern with OpenStack, with the caveat the governance itself is not user-centered, it is vendor-centered, which is not a good sign.
Some of what you said applies, but I think the analogy doesn't go that far. Just to point out / correct some things: Openstack is not very anti-VMware. There are even backend drivers that allow you to manage VMware environment using openstack as a frontend. Never tried it in practice, but some company bothered to write / contribute it, so it must be working.
Application deployment via docker - openstack supports both docker and libvirt/lxc. Unless you meant technologies running in the management layer - in that case I think coreos and similar projects aim for a completely different use case, not just architecture. Openstack works well with not trusted, heterogeneous environments and separation of tenants. If you're running your own cluster, with your own in-house produced stack and control over processes, coreos may be a better idea.
Or maybe you even want to deploy coreos cluster on real hardware automatically? That's also something openstack could do using baremetal backend.
Basically what I'm trying to say is: openstack is a framework more than a single solution. It may not work for every use case, or it may require some specific config to be useful. But it shouldn't limit the architecture beyond: it's got an api.
Technically speaking, I agree that it's a not a single solution and can be flexible. That said, it fundamentally presumes VMs, attachable disks, and networks as "the objects that matters" for your cloud. It's a good assumption for now, but maybe not for 5 years from now.
My post was mostly about the funding and political context that is driving OpenStack momentum, and its similarity to past attempts for vendors to gang up and try to take on the market leader (AWS). The community is all about the vendors, currently. Users are growing, but still the minority. And that's a warning sign, because it may mean the users really just don't care as much as the investors in OpenStack think they will.