Category Archives: Operations

Hardware and system administration related posts. Anything related to the Operations Working Group

New Tile Render Server in the USA

We have a new Tile Render server in the United States! The hardware has been kindly provided by OpenStreetMap US and hosted by the Oregon State University Open Source Lab. Big thanks to them, and to Ian Dees who coordinated this response to the Operation Working Group’s request.

Our distributed tile serving infrastructure brings the “standard” map tiles to your browser wherever you are in the world in a reasonably fast fashion, resulting in a pleasant map viewing experience on the OpenStreetMap.org front page, and with new map edits reflected a few minutes after they are made. It should always be noted that this is far from the only way of using our maps, and we encourage developers to take our data, render it, and otherwise make it available to users in a new ways. However, we do like the front page map to work well. We have a set of “rendering” servers doing the hard work of creating and refreshing raster map tiles, and a larger set of caching servers. With the introduction of a new rendering server in the United States (the first outside of Europe) tiles will load faster. The server itself is fast, and for users in the United States we expect to remove about 100 milliseconds of latency for people viewing the map.

Network latency for requests to the new tile server from various locations in the USA

Details of this new server (which we’ve named “Pyrene”) can be found on the hardware.openstreetmap.org site.

We’re still building our tile serving infrastructure, with a lot of help from people and organisations donating resources. If you are in a position to help with this sort of thing, a caching server – or better yet a rendering server – in India would make a huge performance improvement for people there. Learn more about the kind of servers we need at our wiki page and contact the Operations Working Group.

Server moves: Goodbye Imperial. Hello Equinix Amsterdam

Servers de-racked and ready to move

Some of our servers are moving to a new home. Quite a few of our important servers have been housed at Imperial College in London for the past few years, but it’s time to move on from there as they look to reclaim some space for offices. We’d like to thank Imperial for our time together!

We continue to be thankful to University College London, and Bytemark who are still generously providing hosting for some other keys servers, not to mention our many Tile Cache hosts around the world. If you’re interested in server details you can see the full list on our hardware page.

That list is set to change very soon, as Imperial machines are powered down and moved. The move is being carried out this week by volunteers from the OWG/OSMF.

Where are we moving these servers to? We sought proposals for a new home (thanks to all those who replied), and Equinix Amsterdam has been selected as our new data centre provider. This brings a little more diversity of locations for our servers (many of the others being in the UK), but it’s still not a million miles away, in case our operations team need to visit. Equinix Amsterdam provide excellent “smart hands”, removing the need for physical visits on a regular basis. That being said, the Operations Working Group are seeking someone to help in Amsterdam who can visit the data centre if we need. To quote the OWG folks this volunteer would “need to be trusted, competent and did I say trusted”!

As ever, we owe a big thanks to OWG volunteers for all the hard work going into managing these server moves.

OSMF Request for Proposals: Data Centre 2018

Photo by cosheahan on flickr. Licence: Attribution 2.0 Generic (CC BY 2.0)

Statement of Purpose

The OpenStreetMap Foundation (OSMF) Operations Working Group (OWG) is looking for proposals to provision space in a data centre to continue to run the OpenStreetMap (OSM) project’s infrastructure.

Background Information

OSMF is a nonprofit organisation dedicated to supporting, but not controlling, the OpenStreetMap project. OSMF created the OWG in order to support OSM’s technical infrastructure, including the main website, API, data distribution, community sites, and manage them for the benefit of the project.

The map data created by OpenStreetMap and distributed through ​OSMF​ is the best free global map available. It powers services all over the world, including for companies such as Apple, Foursquare, Craigslist and Mapbox. Ple​ase see https://www.openstreetmap.org/about​ for more background information.

Scope of Work

The data centre provider must meet the requirements set out below.

Requirements

Primary

●  The data centre must be in the EU.
●  One rack (at least 40U) of space, at industry standard rack dimensions.
●  Power capacity at least 3kW w/ dual redundant supplies.
●  Cooling to keep the servers suitably cool, e.g: under 35 degrees celsius.
●  Secure cages, so that only access authorised by the data centre or OSMF is possible.
●  On-site “remote hands” to be able to receive and replace HDDs and press power buttons during weekday business hours and at least some service weekends and holidays.
●  Network connection capable of 1Gbit/s peak traffic and 500Mbit/s sustained.

Secondary

●  Control over the configuration of any upstream firewalls for the purposes of ensuring necessary ports are open.
●  Good peering connection to major European backbone network. Ideally within 20ms of our existing sites on JANET.

Additional questions

Please provide detailed information on:
●  The procedure for shipping parts to the data centre, and
●  The procedure for raising a ticket for “remote hands” work, and
●  Whether “remote hands” would be available outside of business hours, and
●  The site’s uptime and network reachability over the past year, and
●  The procedure for an OSMF representative to visit and access the data centre.

Term of Agreement

The agreement would start on or before 1st April 2018 and run for a minimum of 3 years (at
OSMF’s option), preferably renewable annually or on a longer basis after that. Any renewal or
cancellation on either side would need a minimum notice period of 3 months.

Terms and Conditions

If you have Terms and conditions or Acceptable Use Policies then you should submit them in editable form for legal review, where possible. T&C/AUP changes should be expected to ensure we meet the privacy and security commitments required for our users.

Schedule, Evaluation and Award Process

This RFP is expected to be open until 28th February 2018. Only applications received prior to this date can be considered for this RFP. All proposals will be received in confidence and will be kept private.

After the date above, all proposals will be evaluated by the OWG against the requirements set out above, after which OWG may contact candidate sites with follow-up questions or to arrange site visits. The final agreement will require legal review and approval by the OSMF board.

OSMF particularly welcomes responses from anyone willing to support the work of the foundation at minimal cost.

The successful candidate will be publicly thanked on the main OSM project website as well as OWG websites in accordance with OWG’s Hosting Provider Credit Policy.

Points of Contact

Many thanks for your interest. If you have any questions or proposals, please send them to
operations@osmfoundation.org​.

Server maintenance May 13-15

Some secondary services like the wiki may be affected by maintenance this weekend. Following this weekend, OpenStreetMap users may experience some slow uploads due to continued tuning after our recent database move.

On Monday May 9, our Operation Team smoothly moved the master OSM database server to Bytemark hosting, from Imperial College. Setting up multiple data center redundancy avoided further downtime due to planned power testing and maintenance at the Imperial data centre.

Over the weekend, we are switching some additional services to Bytemark, due to that power maintenance at Imperial. Primary OSM services will be operational, but the Wiki will be in read-only mode during this time. The slow uploads are a known issue, and are being addressed following the maintenance this weekend. More details on the talk mailing list.

Planned server outage – 9th May

The Operations Working Group are in the process of planning some hardware moves as a result of maintenance at our primary datacentre. The plans are subject to change, but these moves are expected to take place on Monday 9th May (a week from today), and this will mean a period of read only operation lasting for up to 24 hours (into Tuesday).

The read-only period will mean the OpenStreetMap website and map is still available, but no map editing will be possible during this time.

The ‘2016 May server maintenance’ wiki page has some more technical details, and will be updated with more precise timing information as the plans are finialised in the coming days.

We’re putting out this announcement to give some advanced notice, but we appreciate some people will be wanting to know more about the exact timings of the read-only period. Please have patience. The Operations Working Group are busy making plans, and will aim to firm things up in the coming days.

Target reached – Thank you!

Today we’ve reached our fund-raising target: £56,000 !

thank-you-restaurantosm.org/go/ZUfMByg0Z

The Operations Working Group set out their detailed plans for the server upgrades they’d like to do: A big new database server, new tile servers and an imagery server. Thanks to your generosity, we now have the funds in place to make that a reality.

This has been our largest fund-raising target, but given the scale, reach and impact of OpenStreetMap, £56,000 is a small amount of money. For the thousands of data consumers, and millions of map end-users viewing OpenStreetMap, or the mega-corporations our maps are compared against. Don’t we deserve more? Lots of interesting discussions around this topic, but I will just say for now… Yes we do! And of course you’re very welcome to donate more:  donate.openstreetmap.org

The OpenStreetMap Foundation remains open for donations. If you have ideas on how to get more recognition and big donations for OpenStreetMap, this is always welcome too!

For now though, we want to say a big Thank You again to Mapzen and Mapbox who made big donations to kick things off, to various other organisations who made donations to support us, and to all the individuals (more than 1200 of them!) who generously chose to put their personal money towards the OpenStreetMap cause.

OpenStreetMap tile CDN continues to grow

Since the last additions to our OpenStreetMap tile serving network in December, there has been a lot more server set-up going on.

osm-cdn-2015-03The German tile cache server tabaluga is now retired and is no longer serving tiles. This may sound like bad news, but quite the opposite! Tabaluga has been replaced with a new server, katie, which has taken over its job.
The new tile cache server katie is still located in Falkenstein, Germany, and still hosted by Hetzner.

More good news: There are two tile cache servers in Germany now!
The second tile cache server, konqi, is located in Jena, Germany, hosted by EUserv.

The Russian tile cache server gorynych just had a memory and SSD upgrade, and with this it can deliver even more content.

There is another new server in Hungary. With this Hungary becomes one of 12 countries hosting OSM CDN servers.

Tile cache server sarkany is located in Budapest, Hungary, hosted by szerverem.hu.

With all of these, the CDN (Content Delivery Network) server count comes to 16 active servers.

Tabaluga was running, thanks to Freerk Ohling, at Hetzner since May 2013, and served its last tiles in January. Freerk approached us back in April 2013 to suggest we implement EDNS client subnet support (implemented in December 2014) and to offer us a sponsored tile cache server. Now he has also kindly sponsored new tile cache servers in Germany.

Tabaluga primarily served traffic to visitors from Germany. Approximately 56 million map tiles per day. (avg 652/sec, peaking at 1245/sec). Serving close to 1TB of data per day. It was the highest traffic OSM tile cache server.

OpenStreetMap tiles are free for everyone to use, but should be used with moderation. If you are a high traffic site you should look at switch2osm.org to find out how to use the data and keep the tiles available for everyone.

The OpenStreetMap Foundation seeks additional distributed tile servers. If your organisation would like to donate a tile server and hosting, please see the Tile CDN requirements page on the wiki. You can also support OpenStreetMap by donating to the OpenStreetMap Foundation.

The OpenStreetMap Foundation is a not-for-profit organisation, formed in the UK to support the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data for anyone to use and share. The OpenStreetMap Foundation owns and maintains the infrastructure of the OpenStreetMap project.

Four New Tile Servers

Have you noticed faster tiles lately? Browsing the map on openstreetmap.org should now be even more responsive. Three new servers, started providing tiles over the last 2 weeks, joining a server which started earlier in the year.

osm-cdn-2015-01

Map tiles are delivered to users based on their GeoDNS location. The OpenStreetMap tile content delivery network (CDN) now supports EDNS-client-subnet to improve locating the closest region tile cache.

OpenStreetMap tiles are free for everyone to use, but should be used with moderation. If you are a high traffic site you should look into switch2osm.org to find out how to use the data and keep the tiles available for everyone.

Thanks to generous donations and active local community members, the OpenStreetMap distributed tile delivery infrastructure continues to grow.

The OpenStreetMap Foundation seeks additional distributed tile servers. If you would like to donate a tile server and hosting, please see the Tile CDN requirements page on the wiki. You can also support OpenStreetMap by donating to the OpenStreetMap Foundation.

The OpenStreetMap Foundation is a not-for-profit organization, formed in the UK to support the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data and to providing geospatial data for anyone to use and share. The OpenStreetMap Foundation owns and maintains the infrastructure of the OpenStreetMap project.

OpenStreetMap Enhances User Privacy

Today, OpenStreetMap has enabled encryption (SSL) to all of the openstreetmap.org website, thereby enhancing the privacy of its users.

You can now browse the site at https://openstreetmap.org (note the ‘https’). This means your browsing activity is secure from snooping.

OpenStreetMap stands with the Open Rights Group and the Electronic Frontier Foundation in asserting greater Internet freedom, including the right to individual privacy. With this action providing the highest quality Free/Open Data Geographic resource to everyone.

We are proud to roll this out on the same day as the “Day We Fight Back” campaign.

Other aspects of privacy around OpenStreetMap are discussed on this wiki page.

OpenStreetMap infrastructure donation – Bluehost

Thanks to generous donations and active local community members, the OpenStreetMap distributed tile delivery infrastructure continues to grow.

Two tile servers, nadder-01 and nadder-02, have been added to the OpenStreetMap tile cache network.  Based in Provo, Utah, USA, these servers provide tiles to the Americas.

Map tiles are delivered to users based on their GeoDNS location. The OpenStreetMap Foundation seeks additional distributed tile servers. If you would like to donate a tile server and hosting, please see the Tile CDN requirements page on the wiki.

tile serving geodns map

We would like to thank the BOSS team (Bluehost Open Source Solutions) and especially Jared Smith at Bluehost.com for this generous donation to OpenStreetMap infrastructure.

The OpenStreetMap Foundation is a not-for-profit organization, formed in the UK to support the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data and to providing geospatial data for anyone to use and share. The OpenStreetMap Foundation owns and maintains the infrastructure of the OpenStreetMap project. You can support OpenStreetMap by donating to the OpenStreetMap Foundation.