Over the past week the license change redaction bot has made automated redactions, sweeping across our entire worldwide dataset. The whole globe was covered yesterday. There has been substantial technical effort involved in developing and running the software to make those changes, and a fair degree of uncertainty about how long it would take, so this is a significant milestone. Congratulations and thanks are in order to all those who helped achieve it, and especially to Matt Amos, Andy Allan, Gnonthgol and MonkZ who carried out most of the coding work.
The data now in the live OpenStreetMap database is largely in a state where it can be declared ODbL licensed, however the license hasn’t changed yet. We will be posting a further update when this is imminent.
More than 99% of the data has been retained, and in most places, the difference is barely noticeable. There are, however, some areas of our map where the redaction was concentrated, in particular Poland and Australia. Though we would of course have preferred to retain this data, we do respect the original contributors’ decision, and we thank them for their past involvement in the project.
Fortunately the OSM community’s response in these areas has been magnificent and we believe we will be back to having a high-quality dataset in these areas in a short space of time. If, as an OSM mapper, you would like to help – by using aerial imagery and other sources, or ideally, from personal knowledge and survey – then please do get involved. Essentially we’re left with some new blank spots on the map, and can respond with the process we know and love. Use your favourite mapping techniques, go explore and fill in the blanks. (Note that you must not copy from CC-BY-SA datasets or map views based on the pre-redaction data; please treat this like you would any other incompatibly copyrighted map data.) We can begin this process in earnest now while final preparations for license changeover are made.
That is not true. There were several letters to DWG regarding removal of data “rescued” from redaction by moving many points by a small fixed distance. No response for over a week, while incompatible data still remains in the base.
Hence the phrase “the data is largely in a state where it can be declared ODbL licensed” – “largely”, not “entirely”. DWG/LWG are currently considering how to tackle the rare cases of copy & paste editing.
Sorry, but DWG is just doing absolutely nothing for several weeks.
I guess that my hometown Cairo Egypt falls within the remaining 1% then. Unfortunately the map has been hit hard here, with big white spots all over the place. With such a small local community it’ll take a while to get to pre-redaction coverage level.
@Alexander: Be patient! That’s nothing. I wrote the DWG about scripted abuse back in February 2010, and I am yet to receive any response.
Pingback: Lizenzwechsel bei OpenStreetMap: Bot entfernt Wegpunkte | virtualfiles.net
Awesome work! Glad to see things progressing well on this front. Many thanks to the people involved in these efforts.
Can somebody please explain why work by the user beldin, who has placed his contributions under the public domain (see http://wiki.openstreetmap.org/wiki/User:Beldin), was completely removed by the bot? This made the area of Adelaide, South Australia, and more, absolutely useless.
Chris, I started writing a response here (short answer: You’re linking to a wiki user, but the openstreetmap user beldin did not agree to the license change) But since this is not a great contact channel for having a detailed discussion I’ve written a more full response on the Australia forum.
Well, if we measure it only with bare statistics and numbers, it seems that there was no significant impact. BUT, looking at the importance of data that was removed and the difficulty in creating/recreating them, there was significant impact and a vertical drop of quality and completeness of the map.
Keeping a footpath of 200 nodes compared to a primary road of only 2 nodes (and some dozens of connected residential nodes) are not of the same importance to be happy if only the latter was redacted. A polygon or some ways involving some hundreds of nodes, imported by a bot, that could easily reimported by a bot, is not the same with a small number of ways with tens of nodes but created by humen. It is more difficult and needs more time to recreate this data that was removed.
The fact is, that almost all the data that was redacted was actually created manually, and probably by early contributors that had left the project and were not aware of the license change. And as early contributors, they had started with the most important data about in their area. Now what was redacted was primary, secondary roads, residential roads with their names, even parts of motorways, and as I’ve seen in my area, complete villages…
For us, people who contribute and are aware of this change and know that there was data that was removed, it is understandable why looking in some area, there are road connecting nothing with nothing, motorways cut in unconnected parts, minor roads ending where there should be a primary road that it is missing. But for visitors, users of the maps, all that they will see is an unusable map, displaying something that they cannot recoginize as the area that they live.
Before some weeks, I could show OSM to friends and say, look at this great project. It is a map of quality even better than Google Maps, Bing, or even official government maps. Today, I cannot say that.
Please put a note on the map at least for some weeks, indicating the large removal of data. Otherwise we will be turning users away from this map – that they will just think that it is crappy.
Make a map showing “redacted” data which must be re-created to concetrate users to make it usable again.
Thanks.
Pingback: OpenStreetMap Chile » Blog Archive » Resumen Semanal OSM #48 y #49
When do you expect to resume posting daily, weekly, and monthly diffs for the new, redacted data?
We’re still in the “redaction period” and we have been posting minutely, hourly, and daily diffs throughout this period, available at http://planet.openstreetmap.org/redaction-period/. I could be wrong but I think we’ve never offered weekly or monthly diffs on the foundation servers.
We moved the diffs to this location during the redaction period because we needed data users to be aware that the redactions were happening, and to make a choice. They could consume these diffs which include a lot of delete actions creating blank spots on the map or alternatively hold off and remain static until remapping has progressed to their satisfaction. Currently diffs contain a lot of remapping edits (add actions) but are still available under the old license.
We will mostly like be moving the diffs a second time when we declare the license changed, because we need developers to flush their databases at this point, and re-initialise from a re-licensed planet file. They will also need to make changes to credit OpenStreetMap differently at this point. If you’re asking when that will happen, it’s difficult to be sure, but… soon.