Service schedule March – April 2012

The long awaited and eagerly anticipated license upgrade is coming soon, the conclusion of a multi-year process. To minimise disruption to OpenStreetMap mappers and users, we’re taking the opportunity to install our new database server (funded by your generous contributions) at the same time – reducing the total amount of downtime needed.

Please be aware of the following service schedule and the list of dates further on in this article. The license upgrade will start with the database server migration. All times and dates are subject to change: our volunteers are working flat out on this, so thanks in advance for your patience and support.

Mappers

There will be a several-day period of limited API availability. The API will be Read Only while the database is moved to the new database server, ramoth. This new database server was funded by your contributions during the December 2011 fund raising campaign. No map editing will be possible while the API is Read Only.

During the remainder of the upgrade, the API should operate normally. Please postpone bulk edits where possible, until after the license upgrade is complete. As always with system improvements, your patience while you find items to refine is appreciated. Consider monitoring the friendly OSM IRC chat channel, #osm on irc.oftc.net, if you have questions.

We would ask mappers who have not yet agreed (or otherwise) to the new terms to log into OSM before the downtime starts on 1st April (0800 UTC) and signal their intention. We are pleased that the vast majority of OSM data will be unaffected by the license change, and thank all the mappers who have thus far consented to their data being distributed under the new license.

Data consumers

The typical planet file scheduled for this week has been postponed. The final old license planet file will be created from 01 April 2012 data. It will be published once the planet file generation completes which may be delayed by a few days.

The old license replication diff service will stop when the database goes to Read Only mode for the server migration. A new license replication diff service will begin at the completion of the license upgrade from a new address. There will be another message with details for using the new diffs when you are ready to start consuming them.

Important Dates

These dates and times are subject to change without notice.

  • 1st April: Enter Read-only mode. 8am UTC
  • 4th April: End of downtime. Enter Read-write mode on new server. Our estimate is that this will be in the morning, but could be subject to change.
  • 5th-6th April: bbox-based live-data tests of rebuild logic.
  • 7th April: Start automated processing of all remaining non-clean objects.
  • 9th April: Progress report and estimation of remaining license upgrade time.
  • To be Determined: On completion of the processing, subject to satisfaction with the outcome we can re-declare the dataset to be ODbL. Immediately afterwards, a first new license planet file will be generated/published, and diff creation will resume.

Thanks

Thanks, as always, to the many people who make OpenStreetMap great. These people include: the countless mappers who have improved the data, the operations volunteers who make so many things “just work”, the programmers who make participating in OSM easier every day, the donors who provide the hardware and hosting we rely upon.

Many of you ‘overlap’ in more than one of these areas. Please be aware that the thanks are cumulative. 🙂

[As excerpted from Dermot McNally’s announcement. Context added from additional sources.]

This post is also available in: Dutch French Japanese Russian Spanish

55 thoughts on “Service schedule March – April 2012”

  1. Emir Hartato

    Why this announcement is so sudden? we have a big event here in Indonesia and also I will have another OSM training in Selayar Island on April 3 – 5. This is a bad news!

    1. Λόγκικ Θέοριστ

      I personally volunteer to help you with any questions you have, now or at any time later, about how to use JOSM and Potlatch

      1. Paul Sladen

        @Kozuch: Instead of thinking “*The OpenStreetmap Foundation*”, think instead several individual OSM fans and contributors (just like you and me) who are giving up two sunny weekends, spending them installing and migrating huge databases on fairly boring remote servers.

        @Emir Hartato: it’s wonderful to hear about the effort you’re putting into training more people in Indonesia. It should be possible to do everything that you normally do during the training, except the final upload and seeing it go live. One option might be to setup a local database instance, such as your own laptop, that other people can push to. (Check on #osm about account creation beforehand).

        Happy mapping. For the next week, I’m going to focus on collecting GPS traces so that they can be uploaded afterwards.

    1. Toby

      The OSM database is a gigantic 2.5 TB in size. It takes a long time to do anything with that much raw data even with fast servers.

      2. Kozuch

        This explanation does not justify your late announcement… you tell us 5 days in advance about a longest downtime ever??? Hello???

        1. Λόγκικ Θέοριστ

          I don’t know about OSM Foundation, but OpenStreetMap has had a HUGE problem for 2-4 years now because of changing from license CC-BY-SA 2.0 to the OdbL Open database License and the ctdl(or whatever in god’s name it’s called??). This has not only disrupted about every mapper contributing but I also left OSM for at least half a year.(if you try to follow a license discussion you’ll end up either enlightened or insane)

          What I really hope is that I agreed to odbl and cttdl and any later versions of them, because if we are going to have a hell to agree to odbl version 2, then we’ll lose….we’ll just lose. It will be bad for everyone.

        2. malenki

          Since last year it is known that at April 1st the license is to be changed. Anyone who has looked a little further into OSM will have taken into consideration that the “switch” won’t be done in a minute.
          For myself I am glad this nightmare going on nearly all the time I contributed to OSM will have an end soon.

  3. Pingback: ODBL switch, now we wait. « 4253

  4. Pingback: 2012/3月 – 4月のサービス・スケジュール | OpenStreetMap Foundation

  5. HarryWood

    The sysops and communications team are acutely aware of the need to give advanced notice to people planning events involving editing. But this is the beginning of technical work which needs to happen for the license change, and overall this work is a big deal (been discussed for years!) and yes it entails some disruption. It’s also quite complex, with many factors at play. You should be aware that there maybe be further disruption in April. The plan may need to be adjusted. We have been in intense discussions with the ‘rebuild’ team to try to reach a clear decision around when the read-only time would be needed, to try to bring this announcement as soon as we possibly could. But yes it wasn’t as advanced notice as we would have liked.

    There will be more updates on this blog as things progress. If you’d like to know more, and get the inside track on how it’s all going, hop on IRC or follow the ‘rebuild’ mailing list.

  6. MichaelGrass

    HEy guys!

    Great outstanding work! I know what You’re all talking about.

    So keep on going. Just a short break, also for tired eyes of the mappers.

    🙂

    Greetings from Switzerland

    Michael

  8. tenshu

    Woaw, a schedule and a complete description on how the migration will take place … right AFTER the licence migration deadline.

    I have no word to describe how angry I feel about all those datas that will be lost, in this ‘useless’ migration.

    1. Grant Slater

      The amount of data lost is around a months worth of mapping at the project’s current growth rate. Once we have moved it is done and we never need to look back.

  9. Dave W Farthing

    In future can you please give a month or more’s notice of major downtime? I’ve arranged a trip to Uganda (at great expense to my sponsors) with the intention of adding data to OSM. If I’d known about this I might have postponed the trip by a few days.

    2. Arturo

      i’m planning to edit offline via JOSM in the meantime and add the work later when the server turns back on, you can do this too 😉

  10. H.S.Rai

    We have also planned an event in which particpants from various country are involved during this period. This event can’t be rescheduled. May I know, if there is any way (some other server), where we can upload edits, user may see rendering their edits, which may be lost or may be added to database at the end of your migration. Any tip, help, alternate will be a big relief to us.

  12. Johnny Carlsen

    I am not part of any of the working groups, but I have known for a long time that this would happend around April 1st. I may not have known the exact schedule and length of the outage, but it was definitely expected.

    I look forward to the change which means I can now again build on other peoples work without worrying that my changes will suddenly be removed because the previous mapper did not agree to a change.

    To the team performing this work, thank you!

  13. wl59

    I want still to agree with the licence changing, for the data what I included. How to declare this ??? I didn’t know about this change (or perhaps I even agreed already, I dont remember me)

  14. Peteris Krisjanis

    I can’t understand confusion and anger at distruption, but this date was known for a year. I’m no fan for this event myself, but as I want to see more OSM data usage without legality fear, this must be done. Also I think discussions around CT and license, while heated, have been quite productive and result is best compromise we could have.

  15. Matt

    Hi Guys
    Thanks – very excited about this cut over – I think it is going to be great for the OSM project and for the greater community.
    It is a huge amount of work – just wanted to say – thanks

    Matt 🙂

  16. alexey.kurinnij

    I expect some messages about this to my email, but not found it. This is not good. I not watch forum or blogs for OSM news and email to members is good choise for important news like this.

  17. Marco

    Go ahead and thanks for doing this work. A small service disruption is a very reasonable cost to pay. Don’t be upset by a few whiners 🙂

      1. H.S.Rai

        With latest josm on latest 11.10 Ubuntu (all updated) got error:

        Could not load plugin mirrored_download because the plugin main class ‘mirrored_download.MirroredDownloadPlugin’ was not found. Delete from preferences?

        Where is error?

  21. Pingback: OSM-Lizenzwechsel reißt Löcher in die Hauptstadt | virtualfiles.net

  23. mwbg

    The database doesn’t even seem to be in read-only mode. I keep getting “internal server error” from JOSM and others. I’m only trying to download, not upload.

    I’ve known about the license change for ages. Why does a simple licence-change require a mass moving of data from one server to another ?

    1. RWeait Post author

      Transferring the database to the new server requires the downtime.

      There is never a “good” time to do that transfer to the new server. So doing that transfer now is a less-bad time.

      The final license upgrade steps will be completed once the new server is up and running, while the DB is in normal Read-Write mode.

  24. Russell

    What a waste for a backwards license change. Why should we contribute data to OSM in the public domain when the OSM license itself isn’t PD? Wholes swathes of roads in Canada have disappeared. Not exactly a useful alternative to Google Maps when good data is deleted, just so the OSMF can change the license in then future at will

    1. peter gervai

      Keep in mind that the license change was actually voted by the OSM community, and not by a bunch of lowlife bums on a funny island. It is completely okay to disagree as long as you’re aware that you speak from a minority viewpoint.

      I don’t think anyone disagree that we badly need a copyright reform, and all these license jumping through the hoop is required to fit the working solution to a broken system. Since it’s broken all hacks we free content people come up with will be broken as well, we’re just busy to pick one less broken. Happens from time to time. (Apart from the server upgrade, which is actually responsible for most of the downtime and which people happily tend to neglect when writing derogatory comments about the license change. :-))

  26. Will

    Export functionality via the website currently does not work. I imagine this is the same API issue plaguing JSOM users. On what date is this functionality expected to return? It’s not clear in the “Important Dates” – especially since it doesn’t seem from those descriptions like exporting should have ever been an issue.

  27. derBeobachter

    I’m really fed up. This changes seems to be carried out by the most unprofessional bunch of people one could think of.
    The information given here (only five days in advance!) seems to be incorrect from the beginning: THE API IS *NOT* IN READ-ONLY MODE, IS DOWN! It only produces 500 errors, which is equivalent to being down.

    Well, maybe the task is harder than imagined, and probably most people could live with a downtime, IF THERE WERE INFOMATIONS on what is currently going on. But no, the blog is mute and the OSM main page is still presenting false information.

    I have been a supporter of the licence change, but the current proceedings make me think that it has been a huge error to put trust into the OSMF.

    Ist there anyone listening? I bet not. We, the ‘simple’ mappers, we have brought OSM to life, but the foundation seems to have as much interest in us as in ants – best to be trampled.

    1. Richard Fairhurst

      No, the API is not down. /map calls are not functioning because they require creating a temporary table, which is not possible when database writes are turned off.

      If you ask politely on IRC, you’re sure to get answers. But no SHOUTING please.

      Everyone here is a volunteer. If you think a better job could be done, please volunteer to help.

      1. derBeobachter

        The point is that we would like to be informed, and I think we have a right to get at least basic information. Well we havn’t got any as yet, not of the exact deletion rules (which made remapping a question of guessing), not of how you interpret “read-only”, and a schedule only 5 days in advance is hilarious. Mind if I SHOUT?

        1. imbitable

          Honestly, please go breath some fresh air in a park. 🙂
          There is no point to act like doomdays are coming. It’s known since a year that the maintenance would happen around this time. And this maintenance is badly needed.
          Thank you for your comprehension

          1. Kozuch

            This might be know to the core community for a year as you say (or even longer) but definitely is poorly communicated to the average mappers… who got to know those 5 days in advance.

  28. Cleberson Pertile

    Sorry, I don’t even have a clue of how these things work; we’ll lose what we have mapped before?

    1. Kevo

      No, you’ll only lose your previously mapped stuff if you haven’t agreed to the new license. If you go to your profile page it’ll tell you if you’ve agreed to it or not (“Contributor terms:”).

  29. Pingback: API Read – Write returns | OpenStreetMap Foundation

Comments are closed.