Author Archives: Richard Weait

geocoder.ca sued by Canada Post

Canada Post has filed suit against geocoder.ca in Federal Court. Geocoder.ca provides a crowd sourced Canadian post code database, available as Open Data under ODC’s ODbL.

The Canadian Internet Policy and Public Interest Clinic CIPPIC will represent geocoder.ca, and you can help by donating to their legal defense fund.

Details can be found with the announcement on the geocoder.ca web site.

API Read – Write returns

The sysadmin team completed the data base migration to the new DB server on schedule during the the morning of 04 April 2012. The API is now back to normal, Read – Write operation. Now the final steps of the license upgrade will proceed as outlined in the March – April service schedule announcement

Other items of possible interest as the license upgrade process proceeds:

osm.org map tile generation will recommence within the next few hours.

Replication diffs during the license upgrade period have started after community requests. These cc-by-sa data replication diffs are found in the redaction-period directory on planet. planet.openstreetmap.org/redaction-period. These diffs will only serve the period up until the switch to the new license. Mappers have requested these diffs for the redaction period. General consumers of OSM data may choose to consume these diffs or not at their discretion.

ODbL diffs will be located in another directory to be announced in future.

During the redaction period it is recommended that editors save their work early and often to reduce the chances of, and the complexity of conflicts with the back ground redaction process.

Bulk GPS point data


OpenStreetMap contributors have used track files from their GPSr devices for years while improving OSM data. They have shared those track files and the track points have been available to other mappers via editors and the web site. Now we are providing a way for you to get all of those points at once.

Announcing planet.osm.org/gps

This is the collected GPS point data from the first seven and a half years of OpenStreetMap. It is a very large collection of points and it is very raw data.

  • the compressed file is 7GBytes in size
  • uncompressed, the file is a 55GByte text file
  • the data consists of coordinate pairs only, with no track file or meta data
  • points were contributed by thousands of users
  • points were contributed as thousands of distinct track files
  • the data includes 2,770,233,904 points

Is this a big deal?

This might be the largest collection of Open Data GPS points published. Do you know of larger collections? Tell us in the comments.

Working with this file might not be your cup of tea. Over time, I expect that tools will emerge from the community to make this data easier to manage. For now, it is raw and it is extensive.

All of this data has been previously available to OpenStreetMap contributors in other forms, via editors and the web site. This file provides a new way to get the same data and to get all of it at once.

Example data

If you do decide to work with the file, this is the format that you can expect.

-900000000,1771882380
-900000000,1293757490
-891154290,1237501070
-877697750,1653442410
-871875000,1589069750
-871875000,1237507350
-843750000,1350007780
-843750000,1153132660
-843750000,1040632590
-843750000,1012507800
-843750000,1012507340
-824414060,1082922390
-815625000,1575007660
-805627440,1579614290
-805517580,1579284700
-805517580,1578845240
-804473020,1373773550
-787500000,1237507380
-787500000,1181257510
-787500000,1096882780
-787499970,1096882780
-778591613,1666901550
-778591613,1666898345
-778591384,1666911621

What format is that?

These are comma separated, raw lat / lon coordinates in a simple text format. To get the coordinates divide each number by 10**7. The points are sorted by location, starting in the far southeast of the globe (90 S,180 E) and moving northwest.

Thanks

Thanks as always to the hundreds of thousands of OpenStreetMap contributors over the seven-plus years of the project so far. Thanks to the syadmins for moving this data to a place where we can all access it.

This version of the GPS data file is CC-By-SA and published by OpenStreetMap and Contributors. The image in this article is a visualization of some of this point data in Europe. The image is licensed similarly and was created by Dave Stubbs.

OpenStreetMap layer added to Yahoo! JAPAN Local

March 29, 2012 – The OpenStreetMap Foundation Japan (OSMFJ) and The OpenStreetMap Foundation (OSMF) announce the release of a new OpenStreetMap layer for Yahoo!JAPAN map services. The new OSM layer is available on Yahoo! JAPAN Local (see the map here)

New OpenStreetMap layer on Yahoo!Loco

2011 Yahoo! JAPAN data donation

On March 6th, 2011, Yahoo! JAPAN handed over nation-wide map data to OSMF Japan. The data was originally owned by ALPS corporation, which merged with Yahoo!JAPAN. The OSM community got permission to import and use this data.
Note that the Yahoo! JAPAN ALPS data is not the map data used by Yahoo.com or Yahoo.co.jp. The maps at Yahoo.com and Yahoo.co.jp are not permitted for use by OSM (see the ‘YahooJapanALPS Data’ wiki page).

OpenStreetMap layer showing Sendai area

Mobile maps

The new OSM layer is also available for Yahoo! JAPAN Local mobile. OSM with its frequently updated data is very useful for many map users.

OpenStreetMap Yahoo! JAPAN Local layer mobile site.

About OpenStreetMap Foundation

The OpenStreetMap Foundation is a UK-registered not-for-profit organization that supports the OpenStreetMap Project. The OpenStreetMap Project based at OpenStreetMap.org, is the worldwide mapping effort that includes more than 500,000 volunteers around the globe. The foundation consists of members from around the world who elect its board of directors. Much of the foundation’s extensive work is done in the background, yet is critical to the ongoing functioning and growth of the OpenStreetMap Project.

About OSM Foundation Japan

The OpenStreetMap Foundation Japan (OSMF Japan) is an organisation that performs fundraising and supports the Japanese community; it does not control the project or “own” the OSM data. It is dedicated to encouraging the growth, development and distribution of free geospatial data and to providing geospatial data for anybody to use and share.

http://www.osmf.jp/news/pressrelease20120329

Service schedule March – April 2012

The long awaited and eagerly anticipated license upgrade is coming soon, the conclusion of a multi-year process. To minimise disruption to OpenStreetMap mappers and users, we’re taking the opportunity to install our new database server (funded by your generous contributions) at the same time – reducing the total amount of downtime needed.

Please be aware of the following service schedule and the list of dates further on in this article. The license upgrade will start with the database server migration. All times and dates are subject to change: our volunteers are working flat out on this, so thanks in advance for your patience and support.

Mappers

There will be a several-day period of limited API availability. The API will be Read Only while the database is moved to the new database server, ramoth. This new database server was funded by your contributions during the December 2011 fund raising campaign. No map editing will be possible while the API is Read Only.

During the remainder of the upgrade, the API should operate normally. Please postpone bulk edits where possible, until after the license upgrade is complete. As always with system improvements, your patience while you find items to refine is appreciated. Consider monitoring the friendly OSM IRC chat channel, #osm on irc.oftc.net, if you have questions.

We would ask mappers who have not yet agreed (or otherwise) to the new terms to log into OSM before the downtime starts on 1st April (0800 UTC) and signal their intention. We are pleased that the vast majority of OSM data will be unaffected by the license change, and thank all the mappers who have thus far consented to their data being distributed under the new license.

Data consumers

The typical planet file scheduled for this week has been postponed. The final old license planet file will be created from 01 April 2012 data. It will be published once the planet file generation completes which may be delayed by a few days.

The old license replication diff service will stop when the database goes to Read Only mode for the server migration. A new license replication diff service will begin at the completion of the license upgrade from a new address. There will be another message with details for using the new diffs when you are ready to start consuming them.

Important Dates

These dates and times are subject to change without notice.

  • 1st April: Enter Read-only mode. 8am UTC
  • 4th April: End of downtime. Enter Read-write mode on new server. Our estimate is that this will be in the morning, but could be subject to change.
  • 5th-6th April: bbox-based live-data tests of rebuild logic.
  • 7th April: Start automated processing of all remaining non-clean objects.
  • 9th April: Progress report and estimation of remaining license upgrade time.
  • To be Determined: On completion of the processing, subject to satisfaction with the outcome we can re-declare the dataset to be ODbL. Immediately afterwards, a first new license planet file will be generated/published, and diff creation will resume.

Thanks

Thanks, as always, to the many people who make OpenStreetMap great. These people include: the countless mappers who have improved the data, the operations volunteers who make so many things “just work”, the programmers who make participating in OSM easier every day, the donors who provide the hardware and hosting we rely upon.

Many of you ‘overlap’ in more than one of these areas. Please be aware that the thanks are cumulative. 🙂

[As excerpted from Dermot McNally’s announcement. Context added from additional sources.]

OSM database downtime 20 March 2012

On Tuesday 20th of March 2012 between 13:45 and 16:15 (GMT / UTC) the
primary database server will unavailable due to emergency maintenance.

The following services WILL be affected:
* www.openstreetmap.org web site will not allow user login or edits
(Potlatch). [1]
* API and map database editing (using JOSM, Merkaartor etc.) will be
unavailable.
* planet.openstreetmap.org will be available but no new diffs will be
generated during the outage.
* Forum (no logins)
* trac (bug-tracker, no logins)
* help.openstreetmap.org (no logins)

Other services will NOT be affected – all of the following are
expected to function normally:
* tile serving (“View The Map” & “Export”)
* Wiki
* Nominatim (search)
* mailing lists
* subversion and git (source code repositories)
* donate.openstreetmap.org

Technical: Database Server Smaug: Replacing faulty motherboard.
Supplier Engineer Onsite. We have contingency hardware available.

1: Maps will still be viewable on the openstreetmap.org homepage and
on other people’s websites.

Sincerely
Grant Slater
On behalf of the OpenStreetMap sysadmin team

Database downtime – 20 March 2012

On Tuesday 20th of March 2012 between 13:45 and 16:15 (GMT / UTC) the
primary database server will unavailable due to emergency maintenance.

The following services will be affected:

  • www.openstreetmap.org web site will not allow user login or edits (Potlatch). [1]
  • API and map database editing (using JOSM, Merkaartor etc.) will be unavailable.
  • planet.openstreetmap.org will be available but no new diffs will be generated during the outage.
  • Forum (no logins)
  • trac (bug-tracker, no logins)
  • help.openstreetmap.org (no logins)

Other services will not be affected – all of the following are
expected to function normally:

  • tile serving (“View The Map” & “Export”)
  • Wiki
  • Nominatim (search)
  • mailing lists
  • subversion and git (source code repositories)
  • donate.openstreetmap.org

Technical: Database Server Smaug: Replacing faulty motherboard.
Supplier Engineer Onsite. We have contingency hardware available.

1: Maps will still be viewable on the openstreetmap.org homepage and
on other people’s websites.

Sincerely
Grant Slater
On behalf of the OpenStreetMap sysadmin team

OSMF Hardware Update

Servers, ramoth and bowser, in position.

The sysadmin team have brought some more hardware on-line for our delight. OpenStreetMap servers are named after dragons, taken from “Here be Dragons” the inscription denoting incomplete / unexplored places on historical maps. Learn more about OpenStreetMap dragons.

  • azure – Java-XAPI. Experimental. Provides read-only OSM data from a refreshed XAPI code base. Azure has recently received a long anticipated disk upgrade.
  • bowser – joins soup and fiddlestick as another Web Front End server. This server will make browsing the osm.org web site snappier for browsing the map, etc.
  • eustace – Web stats. Experimental. Tracks user behaviour across OSM servers to understand and improve user experience.
  • gorwen and orm –geoDNS tile caching. gorwen is kindly supplied and hosted by Teleservice Skåne AB GeoDNS serves tiles from the closest tile server. The sysadmin team hope to have a North American server available shortly. We seek a host for other, geographically diverse servers. If you are interested and not worried by 100Mbits/s, please speak to a sysadmin on #osm-dev at http://irc.osm.org
  • poldi – Nominatim. Provides search and geocoding of OSM data. Return of a local nominatim instance after a hiatus.
  • ramoth – is the second database server. The successful fund raising campaign of December 2011 led to the installation of this server. This server increases the reliability and performance of OSM database operations. Current status: in rack, being configured.

We also welcome the two newest members of the server team, Ian Dees (iandees) and Sarah Hoffman (lonvia). They will be maintaining the Java-XAPI and nominatim servers.

Photo credit

Photo by Firefishy.

Foursquare chooses OpenStreetMap

The social location check-ins application FourSquare has 15 million
users worldwide, and now they are using OpenStreetMap! The foursquare
website now features lovely custom rendered OpenStreetMap maps.

We ultimately ended up switching because, after all our
research and testing, OpenStreetMap and MapBox was simply the best fit
for us

http://blog.foursquare.com/2012/02/29/foursquare-is-joining-the-openstreetmap…

This will surely be a great boost to OpenStreetMap contribution too,
as people spot areas where the map needs to be improved. Foursquare
users, welcome to the OpenStreetMap community.