OMG, I've been waiting for someone to build something like this forever.
I've always hated the fact that all major travel sites work based on the assumption that you already know exactly when and where you're going, rather than at least giving you the option to explore your options.
The problem you have is that flight data is incredibly fast-changing and computationally expensive to get. So in order to get enough data for this you need to do a lot of long queries very often. Companies that provide pricing information (like ITA, who Kayak uses) charge a lot for these resources, so it's not like the usual internet where your incremental cost per user or search is minimal: there's a real cost here. The entire architecture was build on one-off "price from X to Y on this day", building a layer of generality to that is fairly difficult.
Full disclosure: I used to work at TripAdvisor on flights.
Yep, that's exactly right. Every metasearch site out there uses essentially the same interface: given an origin, destination, departure date, and arrival date, return a list of itineraries in order of increasing price.
Needless to say, the number of combinations of these variables precludes doing any kind of exhaustive search (especially since you're being charged per query). And availability is constantly changing, so your results are potentially out-of-date the instant you receive them.
Given these restrictions, the only way to support such broad-ranging queries as "where can I go for less than $1000?" is to cache results from previous searches and attempt to guess at the answer. Which is why, when you start narrowing down date ranges and choosing more obscure airports, the results quickly become so sparse as to be useless.
Broadly agree with this and the parent - but there's an interesting alternative way to look at travel metasearch.
If you think of obtaining prices as being akin to 'indexing', then what metasearch engines are doing is building up a search index for flights. And not all price sources necessarily charge for data (do Google pay to index Hacker News?)
Flight prices become stale as do indexed web pages - and as long as the user doesn't pay a transaction price for any search activity, then you can re-check pages based on user activity - in the case of travel, re-check the price once the user nears the booking phase of the funnel.
As noted though, sparseness of travel queries is certainly tricky! The 80/20 rule applies to some extent, and popular routes will be the ones which have the best price cache.
Another disclosure: I work for Skyscanner (our core is a flexible travel search engine)
I imagine that the kind of traveller looking at this map wouldn't care so much about the direct A-to-B-ness of flights, given that they don't even care that much about where they're going in the first place. They just want to go somewhere interesting, for cheap.
Therefore, rather than only considering the single lowest-cost edge on the graph (that is, an A-to-B flight to an interesting place), you could consider lowest-cost paths, using a pathfinding algorithm (secondarily weighted by the popularity of intermediate cities/vertices as temporary destinations.) Thus, instead of going to city X, it would recommend going to Y, staying a month and taking in the local culture, and then finishing the trip to X with a short, local flight in the off-season (or any other method of travel—it could integrate with Google Maps to see if it would be cheaper to drive between each pair of intermediate nodes than fly.)
I fought this challenge a few years ago and made some some (albeit, little) progress. Besides not being fully versed in programming, I ran into many other problems from processing time, threading, concurrency, but as a number of you mentioned, the biggest challenge is cost. The throughput with or without intent to purchase wouldn't come close to covering the cost of queries. There are ways to offset this- using on/off site marketing, limiting query combinations, and partnerships, but the problem is big- and grows bigger with each subtle change in the query. Cacheing results can help, but the real value of the tool diminishes with each cached result.
I went back and forth with Bill O (at Kayak) while leveraging their API, and know the cost/query was fairly high a few years ago. I suspect it has decreased, but probably not to the point where the processing power needed for massive permutations and cost/query makes this any more feasible.
I set up a proof of concept in 2006/2007 that still works (most of the time). I ended up using Flash/Flex front-end to overcome the number of browser HTTP connections (and keep some of the processing on the client). I'm not sure this actually helped the server or not b/c I never did any testing. I also limited to "where can you go" (multiple city pairs) and "when should I go" (single city pair with multiple dates), but not a combination of both.
Feel free to take a look. It will shut down after the max number of queries/day is reached.
We actually started building such a product for India where the problem complexity is much smaller (only about 20 prominent airports, majority travel by train where prices are static, only availability varies with time).
Availability of data APIs turned out to be a big hurdle for a bootstrapped startup. The market is still up for grabs. Here is another attempt by an Indian startup:
Yea, I was actually seriously looking into the idea of starting a travel site like that a couple of years ago. But it all basically fell apart once I realized how much it would cost to get the necessary data I'd need.
so, i built something almost exactly like this for myself to use, except it doesn't display itself on a google map. instead, it cross-references itself with a database of people.
i use it when i'm hiring sports officials, to find the ones that would be the cheapest to fly in. never really considered how useful it would be for others.
i feel dumb as hell, looking at this and seeing what i could've done :*-(
don't feel dumb, just put it up, add user accounts/Facebook, and make it a group travel planning tool. I could've used something like that while helping plan a bachelor's party.
i probably will, but it just kind of takes the wind out of the sails to know you're half copying something. that spark of "i'm doing something neat" isn't quite there.
I think he means it would be useful when used for a geographically distributed group. You could find the cheapest place for a reunion based on total cost of airfares, and split the cost evenly.
noodle, this is a great idea, don't give this up just because there's another competitor in town.
Please ping me if you are interested in pursuing this further. I can link you up to startup that could definitely make use of this to compliment their trip planning service.
i suppose the issue i was trying to convey is that i wasn't giving up, i was never aware that i had something to give up on. never thought of the thing as a useful tool for me and a few others, since everyone i showed it to thought it was neat, but i was the only one who ever actually used it.
Sure, just so long as it doesn't tell us all to vacation in Frankfurt airport;)
To put it another way: The results for this kind of query are often going to be the big travel hubs. These may not always be the kind of destination that your users want.
Once you have the basics in place, adding a filter to say you only want to go a place with nice beaches or close to ski slopes or whatever should be fairly trivial. And anyway I'm sure my friends and I could have an awesome vacation in Frankfurt.
You just need to take it to a personal level. Bring it down to Facebook, see what your friends like/shared and generate options saying "Friend A is thinking of Trip B in Timeframe C. Would you like to join in?" or something along those lines.
Yea but dont rely on the data that is displayed, i just checked out 5 flights and the flights listed there were not even close to what i found on the website itself.
my problem is, I know my travel dates (mid-August) and I know my budget, but still not sure where to go. Unfortunately Kayak searches (including this very cool hack) are useless at solving that.
P.S. ITA Software Matrix is good at this if you have a list of places, but not for random destinations.
What am I missing, why is this useless? You can specify your budget as the max price, and the "Thinking about" to August 2010. Granted, you can't specify "mid-August", but within a single month, usually airfares only change based on the days of the week of both segments.
Comments
OMG, I've been waiting for someone to build something like this forever.
I've always hated the fact that all major travel sites work based on the assumption that you already know exactly when and where you're going, rather than at least giving you the option to explore your options.
The problem you have is that flight data is incredibly fast-changing and computationally expensive to get. So in order to get enough data for this you need to do a lot of long queries very often. Companies that provide pricing information (like ITA, who Kayak uses) charge a lot for these resources, so it's not like the usual internet where your incremental cost per user or search is minimal: there's a real cost here. The entire architecture was build on one-off "price from X to Y on this day", building a layer of generality to that is fairly difficult.
Full disclosure: I used to work at TripAdvisor on flights.
Yep, that's exactly right. Every metasearch site out there uses essentially the same interface: given an origin, destination, departure date, and arrival date, return a list of itineraries in order of increasing price.
Needless to say, the number of combinations of these variables precludes doing any kind of exhaustive search (especially since you're being charged per query). And availability is constantly changing, so your results are potentially out-of-date the instant you receive them.
Given these restrictions, the only way to support such broad-ranging queries as "where can I go for less than $1000?" is to cache results from previous searches and attempt to guess at the answer. Which is why, when you start narrowing down date ranges and choosing more obscure airports, the results quickly become so sparse as to be useless.
Broadly agree with this and the parent - but there's an interesting alternative way to look at travel metasearch.
If you think of obtaining prices as being akin to 'indexing', then what metasearch engines are doing is building up a search index for flights. And not all price sources necessarily charge for data (do Google pay to index Hacker News?)
Flight prices become stale as do indexed web pages - and as long as the user doesn't pay a transaction price for any search activity, then you can re-check pages based on user activity - in the case of travel, re-check the price once the user nears the booking phase of the funnel.
As noted though, sparseness of travel queries is certainly tricky! The 80/20 rule applies to some extent, and popular routes will be the ones which have the best price cache.
Another disclosure: I work for Skyscanner (our core is a flexible travel search engine)
I imagine that the kind of traveller looking at this map wouldn't care so much about the direct A-to-B-ness of flights, given that they don't even care that much about where they're going in the first place. They just want to go somewhere interesting, for cheap.
Therefore, rather than only considering the single lowest-cost edge on the graph (that is, an A-to-B flight to an interesting place), you could consider lowest-cost paths, using a pathfinding algorithm (secondarily weighted by the popularity of intermediate cities/vertices as temporary destinations.) Thus, instead of going to city X, it would recommend going to Y, staying a month and taking in the local culture, and then finishing the trip to X with a short, local flight in the off-season (or any other method of travel—it could integrate with Google Maps to see if it would be cheaper to drive between each pair of intermediate nodes than fly.)
I fought this challenge a few years ago and made some some (albeit, little) progress. Besides not being fully versed in programming, I ran into many other problems from processing time, threading, concurrency, but as a number of you mentioned, the biggest challenge is cost. The throughput with or without intent to purchase wouldn't come close to covering the cost of queries. There are ways to offset this- using on/off site marketing, limiting query combinations, and partnerships, but the problem is big- and grows bigger with each subtle change in the query. Cacheing results can help, but the real value of the tool diminishes with each cached result.
I went back and forth with Bill O (at Kayak) while leveraging their API, and know the cost/query was fairly high a few years ago. I suspect it has decreased, but probably not to the point where the processing power needed for massive permutations and cost/query makes this any more feasible.
I set up a proof of concept in 2006/2007 that still works (most of the time). I ended up using Flash/Flex front-end to overcome the number of browser HTTP connections (and keep some of the processing on the client). I'm not sure this actually helped the server or not b/c I never did any testing. I also limited to "where can you go" (multiple city pairs) and "when should I go" (single city pair with multiple dates), but not a combination of both.
Feel free to take a look. It will shut down after the max number of queries/day is reached.
http://takoph.com/poc/takoph.html
We actually started building such a product for India where the problem complexity is much smaller (only about 20 prominent airports, majority travel by train where prices are static, only availability varies with time).
Availability of data APIs turned out to be a big hurdle for a bootstrapped startup. The market is still up for grabs. Here is another attempt by an Indian startup:
http://www.90di.com/travel/#form_based_search_home_page
Yea, I was actually seriously looking into the idea of starting a travel site like that a couple of years ago. But it all basically fell apart once I realized how much it would cost to get the necessary data I'd need.
How much?
so, i built something almost exactly like this for myself to use, except it doesn't display itself on a google map. instead, it cross-references itself with a database of people.
i use it when i'm hiring sports officials, to find the ones that would be the cheapest to fly in. never really considered how useful it would be for others.
i feel dumb as hell, looking at this and seeing what i could've done :*-(
don't feel dumb, just put it up, add user accounts/Facebook, and make it a group travel planning tool. I could've used something like that while helping plan a bachelor's party.
i probably will, but it just kind of takes the wind out of the sails to know you're half copying something. that spark of "i'm doing something neat" isn't quite there.
i dont know.. this is interesting for me, but it would be profoundly more useful with my (admittedly small) group of friends.
why more useful for your friends?
I think he means it would be useful when used for a geographically distributed group. You could find the cheapest place for a reunion based on total cost of airfares, and split the cost evenly.
noodle, this is a great idea, don't give this up just because there's another competitor in town.
Please ping me if you are interested in pursuing this further. I can link you up to startup that could definitely make use of this to compliment their trip planning service.
bowei [ a t ] cardmunch. c o m
i suppose the issue i was trying to convey is that i wasn't giving up, i was never aware that i had something to give up on. never thought of the thing as a useful tool for me and a few others, since everyone i showed it to thought it was neat, but i was the only one who ever actually used it.
Sure, just so long as it doesn't tell us all to vacation in Frankfurt airport;)
To put it another way: The results for this kind of query are often going to be the big travel hubs. These may not always be the kind of destination that your users want.
neat idea, anyway.
Once you have the basics in place, adding a filter to say you only want to go a place with nice beaches or close to ski slopes or whatever should be fairly trivial. And anyway I'm sure my friends and I could have an awesome vacation in Frankfurt.
You just need to take it to a personal level. Bring it down to Facebook, see what your friends like/shared and generate options saying "Friend A is thinking of Trip B in Timeframe C. Would you like to join in?" or something along those lines.
Another good site that does something similar is skyscanner.com
For example, here are all of the flights from SFO: http://www.skyscanner.com/flights-from/sfo/cheapest-flights-...
Thank you. I didn't know about that site, and it's exactly the sort of thing I've been looking for. Perfect for planning my next holiday.
Yea but dont rely on the data that is displayed, i just checked out 5 flights and the flights listed there were not even close to what i found on the website itself.
But otherwise definitely damn cool
that's because they show the cheapest round-trip someone else has found previously
hmm i didnt consider that, i was hoping that these were bookable tickets
Me too! www.voyij.com is the closest I've seen. Really glad one of the bigger sites started doing this too.
my problem is, I know my travel dates (mid-August) and I know my budget, but still not sure where to go. Unfortunately Kayak searches (including this very cool hack) are useless at solving that.
P.S. ITA Software Matrix is good at this if you have a list of places, but not for random destinations.
What am I missing, why is this useless? You can specify your budget as the max price, and the "Thinking about" to August 2010. Granted, you can't specify "mid-August", but within a single month, usually airfares only change based on the days of the week of both segments.
didn't notice "thinking about" menu. sorry, my bad!
Well, here's the countries you can fly to from the US in august:
http://www.skyscanner.net/flights-from/us/august-2010/august...