Can't wait for someone to build a "Deploy to Heroku" button that I can stick on a GitHub README. It will spin up the app, do all the setup and then pass it off to the end-user; the instructions for most Heroku deployable apps always follow the same repeatable actions so it would be awesome to automate this.
That's actually been one of my backburner projects for some time. But I haven't had much time recently…
My approach though was having a Chrome extension to add that button on any GitHub repo. Clicking it would create the app, clone it and push it out to your own instance.
Some things slowed me down though. For example the Heroku API doesn't play well with requests made with XMLHttpRequest inside the browser, so I wrote a simple proxy for that purpose. I started working on a Heroku app that would do the git cloning but haven't had much time on it.
The problem I had with XMLHttpRequest was that the API would use the current Heroku cookie rather than the API token passed in with the request.
With curl, the doc says to use a blank username and the token as password in Basic Auth. When I do that with XMLHttpRequest, the token is completely ignored and the main site's cookie is used. There is no way not to pass the cookie with XMLHttpRequest unfortunately, so I would have to rely on if the user is logged in or not. That would bring other problems: I could only use the API for one account at a time for example, and I would have to add checks to see if the user is logged in, etc.
So my work-around was to write a proxy that would just pass around the requests to Heroku and the response back to the XMLHttpRequest client. (https://github.com/Timothee/Passeplat) With it, I can make the same requests I would make to https://api.heroku.com but instead make them to mypasseplatinstance.heroku.com.
Making this a Heroku-ready app is of course not a coincidence: my plan is to use my Heroku instances (the proxy and the yet-to-be-written git clone) to bootstrap a couple of Heroku instances with the user's own Heroku account. Then the extension would use these to start up other apps. I like the bootstrap idea: it's like compiling a compiler. :)
I'm now realizing that it might even be done with a buildpack… I'd have to look into that. (that would remove the need for that second "git clone" app) Hum, I'll really have to look into that! That could make things much easier and cleaner. (and also cooler)
There would still be the issue of specific configurations and add-ons to deal with, but that shouldn't be too hard if the rest is done.
edit: I haven't checked if this is still a problem with today's announcement. Also, I contacted Heroku support about it, but they only suggested I use a proxy.
Interesting. I fear I'm mostly a back-end type of guy (ie not much experience with XMLHttpRequest). Sounds like the browser may be at issue to some extent. I certainly find the behavior you describe to be surprising. Is there something we can change to better facilitate this?
It seems to me that the problem is that the backend is treating the XHR like if you were hitting the main site. So sometimes (but not for all endpoints I think), if I were logged out, the XHR would be redirected to the login page.
To me, the API endpoints should behave the same way in a browser as with curl without ever redirecting to the login page.
But, once again, it might have been fixed in the meantime… I haven't tried recently.
What if you got around it by instead just registering a domain and pointing it to the heroku api's IP? I'm not sure if heroku's api would be ok with the mismatched Host header, but it would be less moving parts.
Those repetitive setup tasks are definitely one of the things we wanted to provide an easier solution for. I definitely look forward to simple scripts instead of multi-step manual processes.
heroku config:set SECRET_TOKEN=`openssl rand -hex 20`
heroku run rake db:migrate
heroku addons:add scheduler
Open the scheduler webpage and add an hourly task
heroku restart
Exactly. For simple cases just shelling out to the API is good enough, but many of these deployable apps are a bit more complicated. Especially those that need databases or other add-ons to work.
High five for the Heroku API designer who decided to use the Accept header to indicate the acceptable format and version of the response. This is the way to design HTTP APIs.
Instead, most companies put the version number in the URL and use special suffixes on the URL like ".json" to indicate these things. I can understand this practice from the standpoint of convenience. A human using cURL can't be bothered to type out an Accept header. But when building an API support should first be considered for explicit, programmatic API clients, not humans. Then you can slap on your human-friendly URL extensions, etc.
Thanks, glad you like it. A ton of thought went into a lot of the little choices like that. We plan to dig more into how and why those choices were made as we move forward.
This will be very interesting for those of us looking to learn more about building APIs. I for one am definitely looking forward to future posts on the subject!
Great, do let me know if there are particular questions you have and we can be sure to cover them. Otherwise I hope to start covering the key things that I think are interesting or important in the not too distant future.
Thanks for sharing the link. There definitely can be a lot of gotchas and things that are easy to overlook until you see a good talk or are right in the middle of it.
I might be misinformed, but wasn't the purpose of stuffing version/format into the URL so that it is easier to develop with? I.e. setting only one URL to make a request is easier than having to set an URL + Accept Version Header + Accept Format Header.
Both the format and version are in the same header if that helps. So there are a couple values in there you might have to change, similar to the couple values you would have to change in the url to modify version/format. So in that regard I think the difficulty or complexity of one or the other is pretty similar. Hope that helps explain a bit, but let me know if I missed the point of your question.
From a glance this looks like a formalization of what already existed at: https://api-docs.heroku.com/ and which obviously the command-line client uses (although the docs and command line/production API have drifted apart)
At Pogoapp we basically reverse-engineered our API from Heroku's command line client, but it's excellent to have Heroku make the API public and formal, because it should make that process much easier and set up Heroku's API as a good target for other PaaS hosts to standardize on (in the same way Ceph & RiakCS have S3-compatible REST APIs)
We definitely used what you mentioned as a starting point. There are some changes, hopefully for the better, but it should in time provide all the same functionality in a more public and formal way. We definitely love that people were excited and motivated enough to reverse-engineer things as you did, so we hope this makes it easier on you and more accessible to others.
We did provide a legacy API (v2), the platform API is the next iteration on the design (v3). The legacy API was mostly used just internally and we knew we could do better in providing a great public API for external folks to tie into and use for awesome stuff.
Currently heroku.rb still uses the legacy API, as we did not want to introduce breaking changes to heroku.rb and because the legacy API will still be around for a while. We will build new tools and update existing ones, but plan to do so later in the beta so that we can integrate feedback and see what will best help people.
I find that amazing. I think every app I've ever worked on or built with Heroku has tied into the API in one way or another. It's one of your best features.
A number of people definitely found their way through the toolbelt code to use the API previously, which is amazing. We hope by giving a stronger focus and some proper documentation that we can make it much more accessible to more and more people.
I had not really explicitly considered that possibility in working on this. I suppose this is a step closer to that, but there are still some pieces that we would need to add. What kind of things did you have in mind?
What I'm imagining is click a button and have it create and deploy fully configured apps for me, similar to what I can do with all the add on services. They might be paid or free or tiered etc, it might be a once-off fee or a monthly/annual subscription or an on-going charge like some of Amazon's AMIs.
There's lots of commercially licensed, self-hosted software (blog, forum, ecommerce etc) and hosted-as-a-service (wordpress.com, discourse.org, duelapp.com etc) etc that would be applicable.
Enabling that by API or in general would be awesome.
I was pretty sure that was the kind of thing you meant, but just wanted to clarify. The API should certainly facilitate doing the kind of setup and management that this would require. But it doesn't currently provide a direct way to do the billing portion of things. So I think it could be a ready solution to open source self-hosted stuff, for instance, but you would have to provide your own billing stuff for commercially licensed things.
Billing is definitely on our todo list as we progress with the beta. Provisioning new accounts is a little trickier as it could potentially open the door to abuse. Which isn't to say we'll never do it or that it isn't possible, just that it is more complicated.
Comments
Can't wait for someone to build a "Deploy to Heroku" button that I can stick on a GitHub README. It will spin up the app, do all the setup and then pass it off to the end-user; the instructions for most Heroku deployable apps always follow the same repeatable actions so it would be awesome to automate this.
I tried out this project with similar aims in the past: https://github.com/rainforestapp/heroku.json but it wasn't quite enough to get an app up and running.
That's actually been one of my backburner projects for some time. But I haven't had much time recently…
My approach though was having a Chrome extension to add that button on any GitHub repo. Clicking it would create the app, clone it and push it out to your own instance.
Some things slowed me down though. For example the Heroku API doesn't play well with requests made with XMLHttpRequest inside the browser, so I wrote a simple proxy for that purpose. I started working on a Heroku app that would do the git cloning but haven't had much time on it.
Sounds like a great approach. What can we do to better support XMLHttpRequest?
The problem I had with XMLHttpRequest was that the API would use the current Heroku cookie rather than the API token passed in with the request.
With curl, the doc says to use a blank username and the token as password in Basic Auth. When I do that with XMLHttpRequest, the token is completely ignored and the main site's cookie is used. There is no way not to pass the cookie with XMLHttpRequest unfortunately, so I would have to rely on if the user is logged in or not. That would bring other problems: I could only use the API for one account at a time for example, and I would have to add checks to see if the user is logged in, etc.
So my work-around was to write a proxy that would just pass around the requests to Heroku and the response back to the XMLHttpRequest client. (https://github.com/Timothee/Passeplat) With it, I can make the same requests I would make to https://api.heroku.com but instead make them to mypasseplatinstance.heroku.com.
Making this a Heroku-ready app is of course not a coincidence: my plan is to use my Heroku instances (the proxy and the yet-to-be-written git clone) to bootstrap a couple of Heroku instances with the user's own Heroku account. Then the extension would use these to start up other apps. I like the bootstrap idea: it's like compiling a compiler. :)
I'm now realizing that it might even be done with a buildpack… I'd have to look into that. (that would remove the need for that second "git clone" app) Hum, I'll really have to look into that! That could make things much easier and cleaner. (and also cooler)
There would still be the issue of specific configurations and add-ons to deal with, but that shouldn't be too hard if the rest is done.
edit: I haven't checked if this is still a problem with today's announcement. Also, I contacted Heroku support about it, but they only suggested I use a proxy.
Interesting. I fear I'm mostly a back-end type of guy (ie not much experience with XMLHttpRequest). Sounds like the browser may be at issue to some extent. I certainly find the behavior you describe to be surprising. Is there something we can change to better facilitate this?
It seems to me that the problem is that the backend is treating the XHR like if you were hitting the main site. So sometimes (but not for all endpoints I think), if I were logged out, the XHR would be redirected to the login page.
To me, the API endpoints should behave the same way in a browser as with curl without ever redirecting to the login page.
But, once again, it might have been fixed in the meantime… I haven't tried recently.
Sure. I haven't tried either. Its on my list of stuff to dig into and sort out, so hopefully we have a better answer soon.
What if you got around it by instead just registering a domain and pointing it to the heroku api's IP? I'm not sure if heroku's api would be ok with the mismatched Host header, but it would be less moving parts.
Interesting idea. I'll have to give it a try. Thanks!
Those repetitive setup tasks are definitely one of the things we wanted to provide an easier solution for. I definitely look forward to simple scripts instead of multi-step manual processes.
That sounds like a fun weekend project! Instead of "try a live demo" you get "create a live demo of your own".
Totally. I definitely hope that this will lead to many, many fun weekend projects. Definitely really excited to see what people dream up.
How's this?
Well most apps needs some others like:
Exactly. For simple cases just shelling out to the API is good enough, but many of these deployable apps are a bit more complicated. Especially those that need databases or other add-ons to work.
High five for the Heroku API designer who decided to use the Accept header to indicate the acceptable format and version of the response. This is the way to design HTTP APIs.
Instead, most companies put the version number in the URL and use special suffixes on the URL like ".json" to indicate these things. I can understand this practice from the standpoint of convenience. A human using cURL can't be bothered to type out an Accept header. But when building an API support should first be considered for explicit, programmatic API clients, not humans. Then you can slap on your human-friendly URL extensions, etc.
It's always easier to start out being explicit.
Thanks, glad you like it. A ton of thought went into a lot of the little choices like that. We plan to dig more into how and why those choices were made as we move forward.
This will be very interesting for those of us looking to learn more about building APIs. I for one am definitely looking forward to future posts on the subject!
Great, do let me know if there are particular questions you have and we can be sure to cover them. Otherwise I hope to start covering the key things that I think are interesting or important in the not too distant future.
+1. I did not appreciate these concepts until I watched Designing a Beautiful REST+JSON API http://youtu.be/5WXYw4J4QOU
Thanks for sharing the link. There definitely can be a lot of gotchas and things that are easy to overlook until you see a good talk or are right in the middle of it.
I might be misinformed, but wasn't the purpose of stuffing version/format into the URL so that it is easier to develop with? I.e. setting only one URL to make a request is easier than having to set an URL + Accept Version Header + Accept Format Header.
Both the format and version are in the same header if that helps. So there are a couple values in there you might have to change, similar to the couple values you would have to change in the url to modify version/format. So in that regard I think the difficulty or complexity of one or the other is pretty similar. Hope that helps explain a bit, but let me know if I missed the point of your question.
From a glance this looks like a formalization of what already existed at: https://api-docs.heroku.com/ and which obviously the command-line client uses (although the docs and command line/production API have drifted apart)
At Pogoapp we basically reverse-engineered our API from Heroku's command line client, but it's excellent to have Heroku make the API public and formal, because it should make that process much easier and set up Heroku's API as a good target for other PaaS hosts to standardize on (in the same way Ceph & RiakCS have S3-compatible REST APIs)
We definitely used what you mentioned as a starting point. There are some changes, hopefully for the better, but it should in time provide all the same functionality in a more public and formal way. We definitely love that people were excited and motivated enough to reverse-engineer things as you did, so we hope this makes it easier on you and more accessible to others.
Didn't they already have this API available?
Can someone explain how this differs from https://github.com/heroku/heroku.rb ?
I would be happy to explain.
We did provide a legacy API (v2), the platform API is the next iteration on the design (v3). The legacy API was mostly used just internally and we knew we could do better in providing a great public API for external folks to tie into and use for awesome stuff.
Currently heroku.rb still uses the legacy API, as we did not want to introduce breaking changes to heroku.rb and because the legacy API will still be around for a while. We will build new tools and update existing ones, but plan to do so later in the beta so that we can integrate feedback and see what will best help people.
> The legacy API was mostly used just internally
I find that amazing. I think every app I've ever worked on or built with Heroku has tied into the API in one way or another. It's one of your best features.
A number of people definitely found their way through the toolbelt code to use the API previously, which is amazing. We hope by giving a stronger focus and some proper documentation that we can make it much more accessible to more and more people.
Does this mean you could build/sell addons that are full applications rather than external services?
I had not really explicitly considered that possibility in working on this. I suppose this is a step closer to that, but there are still some pieces that we would need to add. What kind of things did you have in mind?
What I'm imagining is click a button and have it create and deploy fully configured apps for me, similar to what I can do with all the add on services. They might be paid or free or tiered etc, it might be a once-off fee or a monthly/annual subscription or an on-going charge like some of Amazon's AMIs.
There's lots of commercially licensed, self-hosted software (blog, forum, ecommerce etc) and hosted-as-a-service (wordpress.com, discourse.org, duelapp.com etc) etc that would be applicable.
Enabling that by API or in general would be awesome.
I was pretty sure that was the kind of thing you meant, but just wanted to clarify. The API should certainly facilitate doing the kind of setup and management that this would require. But it doesn't currently provide a direct way to do the billing portion of things. So I think it could be a ready solution to open source self-hosted stuff, for instance, but you would have to provide your own billing stuff for commercially licensed things.
Billing and provisioning. Please oh please let us provision new Heroku accounts. :-)
Billing is definitely on our todo list as we progress with the beta. Provisioning new accounts is a little trickier as it could potentially open the door to abuse. Which isn't to say we'll never do it or that it isn't possible, just that it is more complicated.
I can see this being a good candidate for a Vagrant provider!
Yeah, that would definitely be an interesting way to use it.