Category Archives: Fun

Turk meets GIS

Theres an absolutely fascinating use of Amazons Mechanical Turk (?) right now. There’s a HIT (a small task you get paid almost nothing to complete) that involves GIS:

Geospatial Vision are paying people to do image recognition on sequential video stills from a car that they are apparently then recombining in to videos. These are on their (flash only, sigh) site.

You are paid 5 cents to tag 50 images with yellow lines, manholes, drains, bollards and pedestrian crossings. They are also, from looking at the videos, using these locations to then magically classify the sign type (one way, no entry, speed limit etc). Most images have only one feature if at all, there were about 2,000 HITs last night and at 25 frames a second that puts it at about an hour of footage for $100. That is insane.

If you wanted to get data out of it, the video stills themselves could be captured from your screen like the above screen shot and put back in to a movie. People and number plates can be seen in the images… and street signs so you could figure out where they are. You could add bad data – bollards in the sky or whatever. Amazon have various methods to combat these attacks. But it’s all academic as they’re putting at least some of the work on their site anyway.

It strikes me that this is just scratching the surface of the potential of this class of problem, Mechanical Turk is still only known to a small subset of tech people really. People with big data sets would want entire teams of lawyers to look at this and have Snow Crash-esque schemes to keep people from ‘stealing’ their precious data. Could you imagine the OS ever touching this with a barge pole?

The barrier to entry is a little high in that you have to create a flash app or similar if you want to do more interesting HITs but simpler ones are done automagically with forms by amazon. The other barrier is that like many other companies they think the entire world ends at the edge of CONUS so you can’t really make use of it unless you have a US bank account.

There’s another HIT which just asks for an idea from you – what small program would you like to see that doesn’t exist? I imagine the person who did that one just sitting back and browsing ideas for things to work on. How meta can you get?

So we now live in a world where you can effectively treat data storage (Amazon S3), processing (Amazon ECC) and mass non-linear human intelligence (Mechanical Turk) as infinitely cheap and available. You can get programming, design, legal advice and more from rentacoder, elance and more.

Given this, I can’t think of much that you don’t have covered in Phase 1 of your average business plan. Or to look at it another way, the google kids have been living in this world for maybe 4-5 years.

So readers, if you had a big dataset what would it be and how would you get it processed using the above? Best idea gets $10 worth of HITs.

My Birthday Map

So my birthday is usually a total waste of time because it’s very close to Christmas Day. It means two-in-one presents that are a bit rubbish and everyone’s gone to somehwere better for the holiday, but this yeah I got an awsome present:

It’s a map of central London, approximately similar to this openstreetmap. I used to stare at it whenever I went past the antiques shop in central London (it was on my way home) it hung in. I even tried to buy it once but then thought I was a bit mad and didn’t as I could buy a low-end laptop for the price. It’s about a meter and a bit across by about half a meter down. It’s framed in gold leaf and is original, though I’m unsure exactly what that means in this context.

There are a few things I like about this. Firstly, it’s where I used to live on the western rim of Regents Park. The building is in the map (dated 1834), along with favourite haunts and most of my paper round(s). It also has where I (nominally) went to university and several other places I lived, pubs I’ve died in. So I know the area fairly well and it’s nice comparing it to what it looks like today.

I like the wave of development. If you look at Regents park and then east and west of it along a parallel, you can see a wave of development heading northward turning farmland in to London. A lot of rich people were made as these fields were built upon and their great-great-great (* some big number) grandsons still own vast chunks of it.

It’s slightly later than the periods of history that I like to read about but still very related map-wise. It reminds me of openstreetmap. OSM is incomplete and expanding to conquer everything that is un-mapped – you can see a freeze-frame of our map by just browsing it today. Similarly, the map shows a freeze-frame of those same streets being laid down in the first place as they swallowed up villages to build London.

I have no idea if or how it could be scanned in but I’m open to ideas. A lot of the map is vastly different (Paddington, Marylebone, Euston and Kings Cross stations arn’t there for example!) so I havn’t used it (much) for deriving street names. Interesting anyway.

So a big thank you to all who bought it for me 🙂

Have a great 2007

It’s my birthday tomorrow so I’m looking forward to the annual ‘joint Christmas-Birthday’ presents and getting very drunk (if you’re in London let me know). Then very soon I’m off for family-type things and I haven’t done any shopping yet.

Thanks so much for a great 2006. I’m sure I’ll be here before 2007 but not significantly, so at least have a great Christmas or other winter solstice festival. It’s been an amazing time and we’ve a long way to go – but just think, one day open maps will be as passée as Wikipedia. It’s our job to get there.

OSM Maplexed

Jerome Parkin at Lovell Johns has been playing with mapinfo, maplex and OSM data to produce pretty maps and get them in to Illustrator. This open mapping stuff could really take off.

The process involved the following

  • Downloading data from OSM
  • Load in file into postgis
  • Export to shape file format
  • Re-project the data from geographic to British National Grid
  • Re-coded (in arcmap) to match LovellJohns MapVU range
  • Maplexed (text placement) this re-coded data
  • Exported to Illustrator

As Jerome says ‘[It] looks promising as an alternative to expensive OS data.’:

OSM in WorldWind

worldwind ‘lets you zoom from satellite altitude into any place on Earth. Leveraging Landsat satellite imagery and Shuttle Radar Topography Mission data, World Wind lets you experience Earth terrain in visually rich 3D, just as if you were really there.’

Think Google Earth, but before Google Earth came out (I think, correct me if I’m wrong). Anyway in their continuing efforts to make geographic information in the world more readily accessible, MetaCarta labs principally through Chris Schmidt have made available a WMS-C server of OSM maps, and now WW support! Check out the screenshot:

Winter Meetup

Following the 2nd anniversary party in London, move up north a bit to make it easier for other people to get to.

  • Birmingham, Prince of Wales pub, 84, Cambridge St, Birmingham, B1 2NP. Lat/Lon: 52.47975, -1.91305
  • Saturday 16th December from 1pm

Yahoo! aerial imagery in OSM

Yes, really. Yahoo! have agreed to let OSM use their aerial imagery. Many thanks are due to Jeremy Kreitler and Scott Regan at Yahoo! for putting this together. On the OSM end, Mikel Maron has put a lot in and I’ve (steve) rewritten the applet tiling code to use Yahoo! Imagery. So what does it look like? Well this is what we could use before, Landsat:

It’s about 15 meter per pixel and so not super useful for things like streets. This is the kind of imagery (actually the same area a bit zoomed in from the above) that Yahoo provides:

Bit better, isn’t it? Yahoo has large sections of the US, 200 world cities (for example the London imagery goes out beyond the M25) and more data being added all the time. Makes that mid 2008 goal a little more real. So what’s the applet look like?

No it’s not a mock up, check out this video:

There’s code in there which updates the relevant image copyrights which we must respect, along of course with Yahoos. This updates depending on what you’re looking at. We can of course derive vectors from it for use in OSM. The code needs some cleanup and tweaks. It needs to be integrated in to the site too, expect it in a week or so.

Of course, we still need to get all those street names and features. GPS traces are by no means dead – think of the new housing estates and areas without imagery. But instead of cycling down every road you should be able to just pass lots of them at either end to get the names. Or just from memory.

Where does this bring us to? Well the ‘big map companies’ use expensive cars and expensive aircraft with expensive cameras and expensive GPS units to create maps. Maybe our GPS units are cheaper and less accurate, but does it matter? I think not. We now have all the pieces of the puzzle and we’re putting out great maps for Free using Free tools.

Thanks again Yahoo!

15-second delay

I spoke to a ticket guy on a southern train as we pulled in to Victoria today after he announced ‘there will be a 15 second delay before the doors open after we arrive at the station’. I asked why this was.

Get this, the train knows how many doors to open (because it might be a 12 car train at an 8 car-long station platform) based on GPS. It knows it’s at such and such a station and opens the doors accordingly, no need to trust the driver.

So why the delay at Victoria? Becuase IT CANT SEE THE GPS SATELLITES! At Victoria you can’t see the sky. So they have to trust the driver at Victoria after all, and he takes 15 seconds to shut the train off and override the GPS system the guard said.

Amazing.

Multimap sponsors OSM work

have started to sponsor me to work on openstreetmap.

This was very much inspired by the month of OSM (and is not a replacement) which has seen a soft launch over the last three days. I hadn’t intended to spend all this week on OSM, I wanted to launch with a camera so you could see me working and stuff but hey. I’ll count it as one day, and if you’ve been following the slippy map progress then you know it’s making an impact. More on the month of OSM in later posts, back to multimap.

I and others have been looking for sponsors which didn’t compromise the soul of openstreetmap for a while. The great Nestoria sponsorship of the related mapstraction was a great start but wasn’t going to go on forever. I met the founders of multimap and others at their offices to talk about openstreetmap and they immediately ‘got it’, to the extent of sponsorship. This makes them pretty far ahead of the pack and, I think, visionary.

What’s in it for them? Well, that’s not entirely clear right now. Obviously there is publicity. It’s early days and there are things to be explored but I think 80n has something on his user page when he says ‘My dream is of the day that Google Maps starts to use OSM data (under our license terms).’. Because multimap, google and a lot of others arn’t really in the geodata creation business, they have to license it for lots of money whether its maps, postcodes or whatever. If that is cheaper then great. If the technology behind OSM can be used on existing proprietary data then that’s good too, there’s many uses that can come as OSM matures. This might all be disturbing to some people, so I need to state some things plainly:

  • Will the OSM license change? NO!
  • Will OSM close off the data? NO!
  • Will OSM be re-branded as ‘multimap OSM’? NO! But, given that they sponsor us along with UCL and bytemark who host us I think it’s entirely fair that we acknowledge that like we do for the others already on the front page
  • Are there any strings attached to the money? NO!

Those are real ‘NOs’. What’s telling is multimap haven’t even asked me for these things. They ‘get it’. If they did, then sponsorship simply wouldn’t happen as OSM will and must remain Open. OSM is really you and the data and code you’re putting in – there won’t be an CDDB-like debacle.

So what do you get out of it? Well, thanks to you the month of OSM is getting started and now further, deeper work can occur thanks to multimap. This means more bugs fixed, more hours spent rendering places, more coding and ultimately a better OSM.

So, a big thank you to multimap and to all of you for continuing to build OSMs data and code.

Feel free to discuss all this on the mailing list and I’ll try to answer any questions that arise. Hopefully we should have some more good announcements soon.