The OpenStreetMap Foundation’s Engineering Working Group has an update on the effort to create vector tiles for openstreetmap.org. Read on for why this work is important, what’s been done so far and how they are incorporating community feedback, and the technical details for those who want to know more.
The background
Currently, the openstreetmap.org website serves raster tiles, which are image tiles made up of pixels — think a downloaded image of part of a map. But the effort has begun to create vector tiles for the site, which will help improve how the map looks and how it works. You can read more about the background of the project here.
Vector tiles serve up maps as vectors: points, lines and polygons. They store geographic data (like what makes up OpenStreetMap) in a format that allows for dynamic styling and interactivity. For users, vector tiles will mean a new, modern-looking map style with seamless zoom on openstreetmap.org, the map can be updated more quickly when data changes, and it should perform better for users.
Looking further ahead, the most exciting part is what this vector tile project will make easy for volunteers and tile users: 3D maps, more efficient data mixing and matching and integration of other datasets, thematic styles, multilingual maps, different views for administrative boundaries, interactive points of interest, more accessible maps for vision-impaired users, and surely many other ideas that no one has come up with yet. You may recall that many of these have been long-term interests for people in the OSM community.
The plan
The goal of vector tiles project is to provide a vector tiles setup that can work for openstreetmap.org — that is, a worldwide, complex basemap site in heavy demand from users and services around the world, where the data underlying the map changes all the time.
Or to put it technically, to create a setup for a worldwide complex basemap under high load which requires minutely updates.
Paul Norman is leading the vector tiles project.
He is working on adding to his Tilekiln project which generates vector tiles from a PostgreSQL database (like OpenStreetMap’s), making use of the Shortbread schema, which is a data format for how to name layers & properties within a vector tile, and improving Themepark, which allows one to add OSM data to a Postgres database.
The work is split up into three steps:
1. First round of Tilekiln improvements and Shortbread Themepark improvements
2. Parallelism improvements
3. Shortbread publicly available in production
The first two steps are nearly done. Tilekiln now generates tiles in parallel, making it practical to generate tiles for the entire world. The next step is to start the deployment into OSMF hardware to prepare for production.
Technical details on step 1
For those interested in the technical details of what’s being worked on, there are five main components of the first step above.
1. Automated packaging of Tilekiln
2. Tilekiln metrics being published with a Prometheus exporter
3. Themepark Shortbread reviewed
4. A demo server running with minutely updates of Shortbread tiles, rendering tiles on-demand
5. Demo shown to community
Items 1 and 2 are complete without need of further discussion. For item 3, Paul found that the osm2pgsql Themepark Shortbread implementation needed more work than anticipated as it was missing a layer and had some issues.
Item 4 and 5 are complete. Paul’s demo server is running with minutely updates and the hardware requirements are more modest than expected.
The community has also been providing useful feedback, such as on Paul’s OSM Community Forum post.
The community offered a lot of suggestions, some of which have already been incorporated. The remaining, in-scope issues from the community are: Curved lines rendering as jagged and vector tiles being too large.
The jagged lines issue is due to how smooth curves are represented in vector tiles. It has mostly been addressed but similar issues are expected to crop up in the future. A target scale equivalent to the minimum scale of the standard tile layer has been set. Zooming in to an even lower scale is possible, but artifacts may start to appear.
Vector tile size will continue to be an issue that needs continual work, but the current tiles are particularly large. Since this part of the testing some changes have been made that cut the size in half. Tile size optimization will be an issue that needs ongoing work, as tile size is the biggest factor in user experience.
The tiles being produced are usable, but more work remains to be done. Now that the parallelism work is complete it’s possible to generate large sets of tiles in order to test, so Paul will be returning to working on the tile definitions to improve tile size and fix some remaining issues, but the current tiles are usable.
Background on the tools being used
Here is some information on the various tools used for this project.
Tilekiln is software written by Paul Norman for generating vector tiles from a PostgreSQL database. Alternatives are martin (or maybe t_rex). Tilekiln is in new development, although it uses a lot of standard PostgreSQL features to generate the vector tile data. Most OSM based maps (incl. osm-carto on osm.org) are generated from SQL database queries from a PostgreSQL database. Tilekiln generates vector tiles from similar queries. Tilekiln is new.
Themepark is part of the osm2pgsql suite of tools, to allow one to add OSM data to postgres, and share those processing steps between other projects. Many PostgreSQL based OSM map styles (like osm-carto) use osm2pgsql
osm2pgsql has been around for 15+ years in OSM, and is used in many many places. Although Paul has contributed code to it, he is not the main developer. osm2pgsql has gotten more advanced, and better, in the last few years. Part of the power is pre-processing the data, and Themepark is an attempt to make these pre-processing steps easier.
Shortbread is a “vector tile schema” created by Geofabrik. It’s a data format for how to name layers & properties within a vector tile.
This blog posts contains contributions from Adam Hoyle, Mikel Maron, Amanda McCann, Paul Norman, and Andrew Wiseman
The OpenStreetMap Foundation is a not-for-profit organisation, formed to support the OpenStreetMap Project. It is dedicated to encouraging the growth, development and distribution of free geospatial data for anyone to use and share. The OpenStreetMap Foundation owns and maintains the infrastructure of the OpenStreetMap project, is financially supported by membership fees and donations, and organises the annual, international State of the Map conference. Our volunteer Working Groups and small core staff work to support the OpenStreetMap project. Join the OpenStreetMap Foundation for just £15 a year or for free if you are an active OpenStreetMap contributor.
