Author Archives: RichardF

Providing data to OpenStreetMap – a new guide for data owners

As more and more maps are built on OpenStreetMap data, OSM is becoming the most compelling way for public organisations and other data owners to get their information out to the greatest number of people.

Two local councils in the UK, Oxfordshire County Council and Buckinghamshire Council, were recently funded by the Open Data Institute to investigate using and contributing to crowdsourced open map data – like OSM.

As part of this, I worked with the two councils to draw up a straightforward how-to guide for other organisations that want to contribute their data to OSM. The guide covers all the prerequisites – working with the community, compatible licensing, and ongoing maintenance – as well as explaining the different approaches for integrating data, and discussing which approach will be most suitable in each context. It covers the issues most frequently raised by data owners over the lifetime of OSM so far, and shows where to find more help if you need it.

I hope it will serve as a useful reference for the many organisations who express an interest in working with OpenStreetMap, and encourage more successful schemes in the future. Many thanks to the ODI and both councils for their support!

Richard Fairhurst

OpenStreetMap opens up to more contributors with easy ‘add a note’ feature

OpenStreetMap, the collaborative map of the world often called “the Wikipedia of mapping”, has unveiled a new feature designed to make it even easier to contribute.

For the first time, anyone viewing the map can suggest a correction – such as a renamed business, a missing footpath, or a changed road layout – without learning to use map editor software.

notes screenshot

The corrections will be picked up by OpenStreetMap’s million-strong army of volunteer mappers. A local mapper can visit the location to check the suggested information, and then update the map.

OpenStreetMap’s data is used by some of the biggest mapping names on the web, including Foursquare, Craigslist, Wikipedia, Mapquest, and Apple’s iOS Maps app. It is also extensively used in humanitarian operations, most famously after the Haiti earthquake of 2010.

The new feature is part of a drive to open up OpenStreetMap contributions to more people – so that everyone can add their local knowledge to the map. Later this spring, an all-new HTML5 map editor will be coming to openstreetmap.org; its intuitive controls and clear walk-throughs will encourage more map users to take the plunge and become map editors.

“It’s the detailed local knowledge of our contributors that makes OpenStreetMap so much in demand from web and app developers,” explained Simon Poole, Chairman of the non-profit OpenStreetMap Foundation and a Switzerland-based mapper himself.

“By making it even easier to add to the map, we’re increasing the amount of on-the-ground knowledge we can capture – further distancing OSM from the traditional map data companies and their lack of local expertise.”

To add a note or correction to OpenStreetMap, find your area on openstreetmap.org and click ‘Add a note’ in the bottom right corner.

Background information

Notes are freeform natural text, read by other people, making this is a very simple way to communicate any problems we notice about the map without needing to get to grips with OpenStreetMap and its tagging system. Although this functionality has been available for a while on the third party site OpenStreetBugs, adding it to the main OpenStreetMap site brings it to a much greater audience.

Although it is not necessary to have an OSM account to create a note or comment on it, registered users can see all the notes they have created or commented on, and will receive automatic email notifications if the status of a note changes. OpenStreetMap’s enthusiastic community is its greatest strength, and these notifications facilitate an exchange between the original reporter and experienced mappers.

A full API is available for the notes so that third party sites or apps can query existing notes, create and comment on them – opening up possibilities for mobile bug reporting and fixing. The system is expressly designed for high-quality reports from individual users, not automated error checks or ‘bots’.

OpenStreetMap takes copyright very seriously. Registered users agree not to copy from other maps when signing up, and are requested to verify each suggested correction from non-registered users using ‘on the ground’ information. OpenStreetMap is a map for the people, by the people, and made with personal local knowledge.

Further details are available on the OpenStreetMap community wiki.

Introducing OpenStreetMap’s JavaScript editor – alpha version

Screen_shot_2012-12-22_at_10

Back in July I wrote about building a new, friendly map editor for OpenStreetMap in JavaScript. Since then, and in particular in the last two months, the project has come on in leaps and bounds – and today marks the first alpha release.

Codenamed iD, it aims to provide an easy-to-use, comfortable editing environment for the OpenStreetMap newcomer that’s nonetheless fully featured – you’ll be able to edit any OSM data with it. Clear modes such as “Move map” and “Add point” make it easy to get started without having to read swathes of instructions.

So – try it out! You can play with a working instance at geowiki.com/iD. It’s connected to a test database, not the main OpenStreetMap database, so don’t worry – you won’t break anything.

If you find a bug, use github to check it’s not already been reported, and file one if not.

The fast progress over the past few months is entirely down to the work of Tom MacWright and Saman Bemel Benrud, John Firebaugh, and others. Some of them have been funded to work on this by a Knight Foundation grant from the Knight News Challenge programme.

As ever with OpenStreetMap, the code is fully open source, and we’re looking forward to the community getting involved with helping to build the project. For the first time since OSM was founded in 2004, this will give us a full suite of editing tools – iD and mobile tools for new users, Potlatch 2 and Merkaartor for intermediate editing, and JOSM for power users – so that anyone can bring their local knowledge to the map, whatever their expertise.

Read more about the alpha release in Tom’s blog post.

Building a friendly editor for OpenStreetMap in JavaScript

It’s been an amazing year for OpenStreetMap and it continues to get better. New users from Foursquare to Apple to Wikipedia to Esri, TV and press coverage around the world, innovative releases from MapBox, MySociety, Skobbler, Stamen, and a thousand others. More people are using OSM data than ever before.

More people are adding to it, too. Our data has grown by 43% in just one year. Most of this is added with two tools: JOSM, the offline editor (Java), and Potlatch, the online one (Flash). Launched in 2006 and 2007 respectively, they’re mature, stable, and enjoyed by thousands of users – the tools that have built the best map of the world.

But how do we harness the knowledge of millions of casual users who are now seeing a ‘Data by OpenStreetMap’ credit for the first time? How do we get people editing on the move, using phones and tablets that can’t run Flash or Java? Can we build something that doesn’t have the power of JOSM or Potlatch, yet is easier for the first-time user to grasp?

Hell yes!

Id_prototype

iD is a new project to build a simple, friendly editor for OpenStreetMap – an editor designed entirely for the first-timer to quickly add their street or their local cafe. It’s not a new Potlatch or a new JOSM: it’s ‘an editor for the rest of us’, a stepping stone into OSM.

It’s written in JavaScript: things that required a plugin just a few years ago, modern browsers can do natively, and fast. And with powerful frameworks that even out the differences between browsers, JavaScript can be a joy to program in.

Development is at its really early stages. It doesn’t do much yet: no tagging, no save. It’s not very pretty. That’s where you come in. OSM needs the best, most intuitive map editor there is, and you can help. Coder? Designer? UX specialist? Get involved.

You can read the project introduction, fork the code on github, and read the live docs. Check out the code, play to your heart’s content – tear it up and make it drastically different if you want – and let’s build something to take the OSM editing experience up another notch.

License change update: getting it right

With the new server successfully installed by our sysadmin team, we’re now onto the second part of our migration – the data ‘redaction’ work required to move to the Open Database License. We promised our first progress report next week, but lots of people have been asking, so here’s an update four days early.

The code changes to the OpenStreetMap API have been completed and successfully reviewed. openstreetmap.org is therefore ready to distribute the new data. (Thanks to Matt Amos for the code and Tom Hughes for the review work.)

The next part is the ‘redaction bot’. This is the piece of code that, for an area of OpenStreetMap data, goes through and redacts (removes/hides) any data that isn’t compatible with the new licence. This is the most crucial part of the whole process: we aim not to retain data whose creators haven’t given permission for it to be distributed under the Open Database License, and conversely, not to inadvertently delete anything from the vast majority which is compatible.

Since Wednesday we’ve been running tests against real-world data (thanks to Frederik Ramm for help with this). We’re not yet 100% happy with the results, so we are continuing to work on the code. As you would expect, we will not set the bot running until we are absolutely confident that it is producing accurate results. With the four-day Easter weekend just beginning, we currently expect that this will be next week. This puts us a few days behind schedule, but we owe it to our mappers to get this right.

If you’re a developer, you can help fix the currently failing tests: check out the code at https://github.com/zerebubuth/openstreetmap-license-change. If you’re a mapper, this gives you a few more days to get your area shipshape! And if you’re a data consumer, you can, of course, continue to use the data under our existing license, CC-BY-SA 2.0.

We’ll have a further update next week and, in any case, before the bot starts running.

A weekend of Potlatch

P2_hack_weekend_small

We usually talk about data on this blog, but OpenStreetMap wouldn’t happen without code, too. Running the world’s biggest user-editable map (and, increasingly, one of the world’s biggest maps, full stop) requires thousands upon thousands of lines of code… from the low-level stuff that keeps the servers flying along, via the API and editing software that enables you to contribute, to the programs and stylesheets that turn all this raw data into pretty maps.

Much of this work happens in isolation, co-ordinated by IRC conversations or mailing lists. But we also have ‘hack weekends’, where developers – experienced and newcomers alike – come together to share knowledge and bounce ideas off each other.

Last weekend saw a major hack weekend in Toronto, attracting developers from the US, Britain, and the Netherlands as well as Canada. This weekend saw a rather smaller gathering in Charlbury, a tiny town in the Cotswolds, England, which coincidentally is home to the lead developers of both Potlatch (our online editing software) and Mapnik (the ‘renderer’ which turns OSM data into map images).

The focus for this weekend was Potlatch, with a vast list of improvements undertaken over the two days. The theme was “little things that mean a lot”, so when the new version goes live soon, you’ll notice quicker loading, neater appearance, more reliable operation, and so on. (Those of a technical bent can see the long list of code changes.)

OpenStreetMap’s users are hungry for new features and our existing developers are run off their feet keeping up. So if you have technical skills, come and join in. Whether your knowledge is Ruby, JavaScript or design (the openstreetmap.org website), ActionScript (Potlatch), C++ (various site components) – or whatever it might be – we’d love to have you aboard. Check out the mailing lists (osm.org site, Potlatch, full list) and the #osm-dev IRC channel, and find out how you can get involved.

Introducing a new OSM editor… Potlatch 2

OpenStreetMap users will know all about Potlatch, the online editor that appears when you click the ‘Edit’ tab on the site. Well, there’s a whole new version coming soon!

Potlatch 2 is a complete rewrite still with the same principle in mind: an editor which hits the right balance between speed, ease-of-use, and flexibility. It’s under very active development at the moment and I’ll include a link at the end of this post where you can have a look.

But there are four big new features – and one behind-the-scenes change – to tell you about first.

New feature – friendly tagging system

Potlatch 2 has a friendly, intuitive tagging system. The mapper can use graphical menus, dedicated fields, and icons to get the tagging just right – without the need to remember tag names and values.

For example, you can choose highway types from a set of icons, then add a speed limit by selecting the appropriate restriction sign.

Potlatch 2 tag editor

All this is fully customisable using a straightforward presets file. Using this, you can create your own favourite tag combinations.

New feature – WYSIWYG rendering

Potlatch 2 has an all-new rendering engine far in advance of the current one.

With road names, patterned fills, rotated icons, and much more, the editing experience can be like working live on the familiar Mapnik rendering, the cyclemap, Osmarender, or anything you like -making it much more approachable for the beginner.

The Halcyon renderer used in Potlatch 2

Just like the tagging, the rendering is easy to customise. It uses a special form of CSS, called MapCSS, which lets you create wonderful-looking maps with just a few lines of text. The tagging and rendering together make Potlatch 2 ideal for ‘vertical’ mapping applications, such as a cycle-specific editor or a building/addressing editor. Stylesheets aren’t just about making the map look pretty: you can create stylesheets to help your mapping, such as one that highlights roads without names.

The rendering engine (Halcyon) is available as a compact (<100k) standalone component which you can embed in webpages, so your custom maps can be used outside Potlatch 2.

New feature – Beginners’ Guide

You couldn’t write instructions for Potlatch without writing instructions for OSM. The new Potlatch user needs to know about tagging, surveying, and copyright – but they’re certainly not Potlatch-specific.

So Potlatch 2 will have an accompanying ‘OSM Guide’, explaining the basics with friendly, illustrated text. It will be concise, focused and clear.

New feature – vector background layer

Mappers are working more and more with imports. But the approach until now has been to import data directly into the map – and many people have pointed out the problems this can lead to.

Potlatch 2 will support vector background layers. You can load OSM-formatted data from servers or files, and work on bringing it into the map the way you want, at your own pace.

Because this integrates fully with MapCSS stylesheets, you can choose to temporarily hide background data, or show (say) only footpaths… whatever you like.

Fully rewritten in ActionScript 3

Potlatch 2 is written in ActionScript 3, a Java-like language with an open source compiler and full docs available online. The Potlatch 2 source comes with instructions on getting started and is, of course, permissively licensed under the WTFPL.

Potlatch 2 thus far has been written by Dave Stubbs and Richard Fairhurst (me). But we would love to see more people hacking on the source. There’s a potlatch-dev mailing list especially for this.

Playing with Potlatch

So where are we up to right now, and how long do you have to wait?

The tagging system, rendering engine, geometry editing, and server communication are all up and running – the core of the editor, and the real hard work.

Some other features, like Yahoo and tiled backgrounds, are finished but not currently exposed through the editor: they’ll be along shortly. Others, such as GPS track support, the Beginners’ Guide and the vector background layer are not coded yet but are intended for the initial release.

Potlatch 1 has some three years of development behind it, of course, and much of this feature set has not yet been ported to Potlatch 2. There’ll be countless little UI tweaks (no keyboard shortcuts yet, for example!); and as you’d expect for an in-development version, performance can sometimes be sluggish and there’s a lot of optimisations we’d like to do.

But with work progressing so fast, this seemed a great time to talk about it. Both the tagging system and the renderer are enormously flexible and we’d like to see people hacking on them as soon as possible.

So how about some links? You can find a read-only running version at:
http://www.geowiki.com/potlatch2/

or play with the renderer alone at:
http://www.geowiki.com/halcyon/

Should you want to try a particular area, just put the lat and lon in the URL like this:
http://www.geowiki.com/potlatch2/?lat=52.2&lon=0.1
http://www.geowiki.com/halcyon/?lat=52.2&lon=0.1

and the source is at:
http://trac.openstreetmap.org/browser/applications/editors/potlatch2

and you can read MapCSS documentation at:
http://wiki.openstreetmap.org/wiki/MapCSS

Have a play, let us know what you think, and grab the source!

Google’s Data Liberation Front and aerial imagery

Data Liberation Front

Google have a couple of really enlightened guys called the “Data Liberation Front“. Their role is to make it easy for people to get their data out of Google – rather than it being locked in.

Usually, people are locked in by the lack of an export feature, or an obscure file format. In mapping, people are locked in by licences.

In Google Maps’ case, you can create your own work by tracing over aerial imagery. But you can’t use this work elsewhere, because of the licences and terms of use. (The phrase “derived work” usually crops up around now.)

At the Society of Cartographers’ Summer School last week, there was a great workshop on making a mashup from Google imagery – but when the question arose about taking the data out of Google and using it elsewhere… well, then things got a little hazy.

Google could fix this by saying that tracing from their imagery is ok – just like Yahoo have done. Or, alternatively, they could give us a clear “no”. Right now no-one really knows where they stand.

The Data Liberation Front ask people to vote for their favourite suggestions. We’ve added this as a suggestion and, at the time of writing, it has 585 votes – over four times the next most popular.

You can vote by going to http://url.ie/2ero. OpenStreetMap has shown the way in liberating geodata – maybe it’s time that some of these other guys caught up.

London to Brighton

A bike, yesterday

Everyone has their own reasons for enjoying OpenStreetMap, but for me, cycling is the “killer app” – in that OSM gives you the best cycling maps in the world (on the web and on your GPS), and mapping is also a great excuse to get out there and cycle.

This weekend, four intrepid OSMers are taking it a stage further by entering an OpenStreetMap team in the famous London to Brighton Bike Ride. Andy “Blackadder” Robinson, Etienne Cherdlu, Simon “Welshie” Hewison and Graham Seaman together form the OSM team. Another intrepid OSMer and keen cyclist, Gregory Williams, is also doing the ride though not officially as part of the team.

It’s a great fundraising event for the British Heart Foundation, and our riders are also seeking a little extra sponsorship to fund the printing of flyers explaining OSM – which can then be given out to other cyclists on the route. And with 27,000 other cyclists taking part, there’s plenty of opportunity for publicity.

Of course, the main route is already mapped… but rumour has it that our riders might be tempted to detour “off piste” to two unmapped villages, Crawley Down and Ditchling, if enough sponsorship comes in.

Find out more about the ride, and pledge your support, on the wiki page.

OSM Super-Strength Export

One of OpenStreetMap’s greatest advantages is that we don’t just give you a beautiful draggable map – we give you the data, so you can do what you like with it. Well, this weekend, that just got a whole lot easier.

OpenStreetMap now has an ‘Export’ tab, joining ‘View’ and ‘Edit’ at the top of the screen. It gives you an instant way to get the map data in a format you want.

Want a static map for your blog, without having to spend hours fiddling with JavaScript? No problem – just export in PNG or JPEG. Want a map for a book? PDF or SVG are the perfect formats – fully vectorised, so they look smooth on high-resolution printers at any scale, and are easy to restyle or edit. Want to play with the raw data? Get it in our easy-to-parse OpenStreetMap XML format. Here’s an example of a simple PNG streetmap generated in just two clicks.

And this is just the start – our mailing lists are already buzzing with possibilities for new formats, such as Adobe Illustrator for cartographers, or shapefiles for GIS professionals.

With this new feature, the difference between OpenStreetMap and the “corporate” mapping sites becomes a whole load clearer. Other mapping sites’ agreements with their data providers (such as Navteq, TeleAtlas or national mapping agencies) simply wouldn’t allow them to give the data out like this. With OSM, we actively encourage it!

The work behind this was done by Tom Hughes, winning OSM’s coveted Lolcat of Awesomeness developer award for the fifth time.