Augmenting photos – with OSM!

You climbed up a mountain and took a photo:

And that’s very pretty. But it’s 2009! Why doesn’t it have all kind of magic over the top of it. Enter Marmota. You tell it where you took the photo (maybe your photo has a GPS attached anyway) and it generates a simulated panorama 360 degree wraparound of what the landscape looks like from height field data. It then matches your photo’s pitch, yaw and roll and lens angle against this virtual panorama to figure out exactly where you were pointing it. It uses computer vision techniques to figure out the outline of mountains in your photos to do the matching. Here it’s matched it to pointing at these mountains:

And now you can fade between the computer generated hills and the image itself:

Finally because it knows the height and location of each pixel in the image, you can now in 3D overlay OSM data (such as rivers etc below) on to that picture. You’re augmenting it with things much as wikitude does, but at a higher resolution.

Now of course the output can be played with and overlaid in proprietary and closed projects like Google Earth. Here we’re looking back from a distance toward the location from which the photo was taken. The black areas are hidden shadow areas where hills block the view from where the picture was taken:

These ortho rectified images can be played with in any GIS. Neat huh?

Here’s a short podcast with the projects creator.