Since I’ve been asked by several people, I’m going to go into a bit of detail on how the GoogleMap for the Starbucks Challenge came to be.
For those of you not wanting to follow all the links, let me give you some quick background: The Starbucks Challenge challenges the marketing message Starbucks is sending regarding fair trade coffee. According to them, you can always get a french pressed cup of fair trade coffee in any of their stores, if you just ask. Challenge participants then post their results, and tag it on del.icio.us. The map then gets the data from there.
While the using the the Google Map API itself certainly had hurdles – which are going to be covered in another article – the main difficulty to figure out at the outset was: “How do I get longitude and latitude for any store?”. Sending people a GPS unit was out of the question, but thankfully Starbucks itself is helping. Their link to the store driving directions helpfully contains exactly the data we need. Getting the page containing the link is easy, and each store is uniquely identified by a store key – perfect for tagging.
Getting the link out of the page is not so fun. To get this done in any reasonable amount of time, I had to go with a scripting language. I picked my current personal favorite, Python. It’s easy to write and has plenty of support for web tasks. Reading the page, for example, is a one-liner.
How to extract the link, though? The first idea that came to mind was writing an XPATH query. However, the state of XML parsing for Python is a sorry one. There are quite a few competing packages, all with their ups and downs. Thankfully, the point was quickly rendered moot – the web page is not well-formed XHTML, and all the candidates choke on that.
So what we need is a package to parse the mess that qualifies as HTML. Thankfully, there’s a package to do just that: BeautifulSoup. It takes the tag soup of any HTML page and renders it navigable. Lots of heuristics to cover all the mistakes most HTML parsers allow. Let me give you the two salient pieces of code:
- Extracting the address of the store. This is stored in two <span> elements with the IDs of bAd2 and bAd3. Extracting that is a simple two-liner:
Yes, that’s it. No more work.soup = BeautifulSoup( webpage )
addr = soup( 'span', { 'id' : re.compile('bAd[23]') } ) - The actual location is similarly easy. It is a link element thoughtfully classified as a “DriveLink”. (Thank you, Starbucks!). All you need to do is get the soup of all DriveLinks, and look at the href attribute of the first one:
Judicious application of a bit of regular expressions then yields longitude and latitude.driveLink = soup( 'a', { 'id' : 'DriveLink' } )
href = driveLink[0]['href']
And that’s all there is to this, at the core. Since I like playing with code, there’s a bit more to it, of course. If the link doesn’t provide geolocation any more, it falls back to Yahoo’s Geocode API – just in case Starbucks decides not to be helpful any more. And as to not unduly hammer their server, the results of the Store Key lookup are of course cached. And it’s all nicely wrapped up as a module. If I ever get to clean up the code, I’ll release it here. If you need it before, just ask.
Have you checked out Frappr? I thought it was a clever idea. It will only be a short while before Google does something similar tied into GoogleBase. http://www.frappr.com/
I had to incorporate Google maps on a client’s PHP based site a couple months ago. I wound up using a PHP include to hit geocoder.us and embed the result in the page and then parse it with Javascript on the client side. Javascript XML parsing is passable. Except it works slightly different in IE (of course). We’ve had a couple of misses with geocoder missing some addresses recently so I’m going to take a peek at the Yahoo geocoding API pretty soon.
BTW, your link to “Fair Trade Coffee” is broken at the moment.
Thanks for pointing out the link – fixed.
And yes, I have checked out Frappr. That’s what made me do this. While Frappr is cool, it’s a pain to add additional places, and it only allows you to specify ZIP codes – which is not precise enough. There’s about 300 Starbucks per ZIP in LA
I stumbled upon the mess that is JavaScript, Geocoder and IE while doing research – I’m glad I could avoid it. This site has all the data pre-coded in an xml file. (Which, at least technically, makes it all Ajax-y
, so I just load that and go. More details in the next post.
So you’re in the web design business now, I take it? Hope things are working out?
Nice idea, that map. I just wanted to write that another advantage of your implementation is the fact that other places than the US can be incorporated as well, but then I checked out Frappr once again, and noticed that they do in fact support other places (and somehow I must have missed that during earlier visits). Well, I still like your map. Good work.
Thanks for the encouragement!
I saw that Frappr did non-US places, but you could only place by city, which is a bit coarse for me. I just noticed that the newest Frappr then allows you to move the marker by hand – but still, it’s a lot of manual work, and I wanted the challenge results to be as easy as possible.
Would you share this code?
thanks