Thoughts on OSM design, and looking forward and back

I really liked something I read in The New New thing ( http://www.amazon.com/New-Thing-Silicon-Valley-Story/dp/0140296468/ref=ntt_at… ) about Jim Clark ( http://en.wikipedia.org/wiki/James_H._Clark ) when the author Michael Lewis would ask about the past of his companies and stuff he’d say “that’s boring” and “That’s the past. I really don’t give a shit about the past”. ( http://books.google.com/books?id=UJw3i9_ZqQkC&lpg=PP1&dq=the%20new%20new%20th…&q=shit%20about%20the%20past&f=false )

I think we’re locked in between three groups of thought on the OSM right now.

Right up front we have the school of thought that everything is perfect the way it is. That uservoice is some kind of inherently crappy system (see the uservoice ideas page at http://osm.uservoice.com/ ). That we shouldn’t allow people to use tools which make fixing the map easier (see @chilly on twitter), that people are inherently stupid and there should be a barrier to entry to editing in OSM because it’s complicated. This school of thought is essentially still living in 1991 and I’ll call this school the Game Haters: everything is wrong, even talking about it is wrong.

In the middle we have a bunch of thought on how the site should or shouldn’t be. Legitimate questions about putting the map or help up front, or using OSB or uservoice, or some new system, or something. But nobody can agree with anyone else, and anyone who actually does anything comes under attack because they’d never encompass everyone’s idea of what the design or UI should be. Let’s call this school the Player Haters: the game is there, we can play by the rules but don’t like it when someone plays better

Lastly we have a school which is looking forward and willing to throw out ideas and try them. They don’t instantly hate everything or dismiss it because they don’t personally like it. There is room in this school to understand that there are other schools out there, that what works for them might not work for someone else.

At points like these, I think we have to decide though some debate where the project is going to go. If we want to just keep the tools hard to use and subject people to PL1 and trac, then that’s a legitimate point of view. If we can stand some innovation like group 2, then that’s cool too, or if we’re able to just move on and keep innovating.

If we look back, we’ve actually mostly not given a shit about the past. We threw out segments, threw out entire codebases (like 0.1 0.2 and so on) in the search for something better. We in OpenStreetMap tend to innovate. That’s not to say it’s not messy, it’s a horribly messy process from a ‘consensus’ and community point of view, because often their isn’t any consensus on anything, ever.

It’s that central freedom to not conform that is the most important, beautiful and gratifying thing in the project but sometimes like now with the design, it holds us back.

I don’t want the entire design debate to be about uservoice, but it’s a great example that exposes the extremes of thought. Going through the extremes:

* Some people *literally* don’t want any feedback.
* Some want feedback, but in trac or hidden in some other horrible system
* Some want feedback that’s easy, but just not on the front page
* Some want feedback that’s easy and upfront but not too exposed
* Some want feedback that’s easy and exposed to the most people (like having maplint or keepright or OSM switched on the front page by default)

Will we ever get a consensus through debate? I highly doubt it.

For the record – yet again – I’m not proposing uservoice as the final solution. I’m not proposing we use it for map bugs. But, it is a brilliant tool for many sites and it’s provocative and brings up cool ideas of what we can do in the future with something similar.

It’s worth also thinking about where the schools of thought communicate. Mostly the negative ones are on the lists, and the positive ones have been in uservoice and on opengeodatas comments on the blog posts. Why is this? That’s hard to answer. I think it might be simply that there are a lot of barriers to entry on the list, flames and baggage that a newbie doesn’t want to deal with – because *they’re a different group of people*.

The project can exist with these different schools of thought.

When I think back to most of the beginning years of OSM I’m struck remembering how much time I spent fucking around with SQL doing the big horrible jobs that nobody wants to do. Our sysadmins today mostly do all this awesome work and probably enjoy it like I did even, we need that skill set and school of thought to make the project run.

In other words – we have people who contribute in all sorts of ways.

At the same time, we growing in to the realm of a new school of thought. We’re increasingly hitting people who can contribute enormously but just not in the way we’re used to. Basically it’s a question of time and how much mapping/software/community/etc you can contribute per unit time if you’re a random member of the public new to the project:

* If you want to contribute a half-decade to OSM you can, and many have
* If you want to contribute a year to OSM you can, and get a lot out but you need that time
* If you want to contribute a month, that’s reasonable
* If you want to contribute a week, you can do it just about probably with some pointers
* If you want to contribute an hour you need lots of help, like a mapping party
* If you want to contribute a minute, you’re screwed

Everyone in OSM has basically been contributing for the kinds of extended periods of time as above, not the minutes or hours. Many see someone contributing so little as wrong or pointless. I say just the opposite. The people who spend minutes or hours disappear because we just don’t welcome them.

It should be perfectly possible to contribute an error in 1 minute in OSM, and have someone who’s prepared to spend a lot of time on it fix it. But so many people fight that idea. There’s nothing to be afraid of, we’re just increasing the size and reach of the community – and that’s a good thing.

To think of it another way, consider a scale free distribution of OSM contributions, because that’s what it looks like: we have a very few people spending 24/7 on OSM, we have a few people spending hours on the site a day and then *lots* people spending 5 minutes a day. What we should be able to do is connect those groups. If we have 60 people spending one minute to report a bug, say textually, on OSM then there are plenty of people in OSM willing to spend an hour going through and doing the actual editing. And the project would be infinitely stronger for it.

It’s not like this is something new, it’s exactly what map maker and waze do in their own way. We can do it better though. Our simple attempts like keepright, OSB and the dupe_nodes stuff points how big these kinds of feedback could be with more polish.

And we have to be honest about how bad things are. I know when I say that OSM is crap, PL1 is crap and so on many of you get all offended… but it’s just the reality of it. I don’t think we really need to get newbies to do screencasts on usability like wikipedia did? Get people in and record them trying to fix a street. We can, but if you can put yourself in a newbies shoes for a second you can see it.

We need new thinking and a fresh push on design and usability. That might not come from within the existing community, by definition if it’s going to attract new and different kinds of people, which it needs to if we’re going to scale to the next level of contribution.

Basically it comes down to trusting someone to do this stuff and not giving them too much crap for actually getting it done. We made many similar arguments with the license change process you might recall. Many think you can have a valid legal opinion without the nuisance of an actual law degree in the same way you can write kick-ass C code without having a degree in computer science. Everyone has those legal opinions the same way everyone has design opinions, but they’re rarely right.

Back to what we need to fix – design, the editor experience, the logo (this spontaneously came up on uservoice again).. they all have fundamental problems. Just because they’ve worked for us old timers, we like the existing logo and we’ve learnt to put up with PL1′s foibles doesn’t mean that’s the right thing to keep going forward.

I know matt takes it personally that the logo is anything other than perfect, and richard takes it personally that potlatch is crap for newbies… but that’s just fact of the matter as I constantly hear from designers, newbies and so many others. I don’t care about personal feelings on those particular topics nearly as much as it physically pains me to think about all the people we turn away every single day because of it.

We have IIRC about a 70% drop off rate of people who create an account and never do anything else. I’ve heard the school of thought that says “fuck them they wont contribute anyway” but I have to disagree. A simple prominent feedback tab to report map bugs or feature requests is the simplest possible thing you can add, and I promise it will lead to a huge spike in contributions. I’m willing to bet people money on that.

Most maps in the world try their best to hide their bugs, like closed software. We should be bold end expose them so they’re fixed faster.

Lastly I want to talk about implementation.

It’s clear after years of chatter that the community is the wrong place to innovate on design and probably editing too. The model of ‘wait for someone to do it’ works well on a bunch of things, but not everything. How did I get the design done? I paid $70 to a really great designer and html coder in peru who I worked with over skype to come up with the straw design. For $70 (at $7/hour) I got more done than the last 1-2 years of design in OSM. That should be celebrated not attacked because the current site is perfect (really, it isn’t) or I didn’t warn everybody (JFDI is supposed to be a virtue here).

There are talented flash coders, designers and more who will even work for free to help us too but just can’t put up with people pissing all over their work, which is what usually happens on these lists, or bizarre tool chains or having to refactor crappy code. They don’t have that thick skin and time for it. I think we need to find ways to work with them, or pay them, to work on this stuff. And that involves putting a buffer between the old timers in the community and people who want to move it forward.

Or maybe there’s a better way, you tell me?

Case in point, PL2. We have no idea when it’s coming, if it will work or what. What I personally want to see is a community of people behind building the thing like there is behind the rails codebase or even JOSM. But everyone’s so afraid of pissing off richard, or doesn’t have the time to work it all out, we’re not moving forward like we should be.

Here’s a radical solution – flash programmers are $10-$20/hour on oDesk.com and others (and it will be quicker and cheaper than you might think). Why don’t we club together to get it finished? I’ll come back to how in a second – but just think for a minute how many people are turned off because of the editor. If we can bring in just 1% more people a day with a better user experience and editor, then compounded over time we have a huge gain in map quality, community and everything else.

Back to paying someone. Is it the best solution? No. Will the result be perfect? No. Is it the best way to get open source code built? No, but I point out that most of the Linux kernel is now built by paid employees. Would Richard like to be paid to work on PL? No, I’ve tried a bunch of times. Would someone else in the community? Maybe. Should we try something again like bounties? Maybe that too. But just sitting around with the status quo on all these issues isn’t getting anywhere fast.

At this point I will get a load of flames that Richard is awesome, he’s spent loads of time on the project and all that. I agree. But, I take that as a given as I do with anyone in the project. We’re all here giving time, love and effort. We shouldn’t have to preface every criticism with three paragraphs about how we’re all so great.

Lastly – I’m saying all this to promote a debate and discussion. Paying someone is just one option. Do I want to do it tomorrow? No, but it does look interesting. And, by the way, just paying doesn’t mean we just get the software out. It means from experience you usually also really get the paid coder interested in helping long term (not always, but if you’re good at picking who does the work) and in this case we’ll also be able to pull in lots of people who are professional flash coders who will expect the code to be laid out a certain way, with such and such a toolchain and all that.

Let the flaming commence.

6 thoughts on “Thoughts on OSM design, and looking forward and back

  1. tom_chance

    Steve,A few quick thoughts…One – I’m completely behind your sentiments about needing to make it *much* easier for people to get involved with OSM, and behind just trying something new, and behind paying some people to go off with a brief and come up with something unexpectedTwo – so C monkeys shouldn’t expect to lead on legal stuff, and lawyers probably aren’t great SQL wranglers. Maybe it’s time that ‘senior’ OSM people accepted that they aren’t great community organisers? Talk to anyone who has spent their life working with communities and they’ll tell you stuff like "consensus is near-impossible to arrive at online", but that jacking it in completely will only breed resentment!Three – telling people to trust you as you re-focus a project on a new audience whilst having a go at them isn’t going to win you many friends. You need to reassure people that there will be some process to come up with some principles in a very light-touch brief, and a second process to discuss a few redesign candidates. These processes should be set-up by somebody with experience in that area, and played out at State of the Map and perhaps local chapter meetings if possible.Given that OSM can support many back-end tools and front-end presentations, it shouldn’t be too hard to re-think the default osm.org approach.Hmm, each one got longer than the last!

  2. Erik Johansson

    <i> * If you want to contribute a half-decade to OSM you can, and many have* If you want to contribute a year to OSM you can, and get a lot out but you need that time* If you want to contribute a month, that’s reasonable* If you want to contribute a week, you can do it just about probably with some pointers* If you want to contribute an hour you need lots of help, like a mapping party* If you want to contribute a minute, you’re screwed </i>I like this part, but you are not giving any ideas on how people should contribute in one minute or less.When I help people we spend minimum 5-10 minutes trying to edit in OSM. I agree that PL is not the easiest, but neither is Mapzen nor OSB. With my help they can edit and add roads+names+numbers in 5 minutes, it would be very interesting to see a mockup where they can do it themselves.So perhaps we shouldn’t be reiterating facts about how much PL sucks, nor how to build complete replacements. What you want to build is just a small tool that does the job, have you done or seen anygood mockups of that?Regards ErikYou come off less flamey the more text you post.

  3. Steve Coast

    Erik if you look back you will find exactly a mockup etc of what I think we could do. That’s what were all discussing.

  4. Erik Johansson

    Well I concur with Frederik that it’s hard to keep up with the flurry of text that you generated, but I’m pretty sure the only thing I saw was a redesign of the front page (which is good step imo). What I haven’t seen is a good 1 minute tool for casual mappers. Which I want myself, when I have no time to load up Potlatch.Don’t get too distracted trying to piss everyone off, while infinitely amusing, not much gets done.. :-)

  5. Candid Dauth

    I think your definition of “easy” is a bit too simple. We find an application easy to use when we can work with it without having to learn anything additional. That is the case if we have used a similar user interface before, but everyone has used different applications in his life and thus finds different things easy.It might be that uservoice or any other “newbie-friendly” bug tracker would be easy to use for people who don’t use computers very often and aren’t that experienced. But many of the people who report bugs in Trac have used Trac thousands of times because lots of open-source projects use it. For those people, Trac is thus the easiest possible user interface.For many open-source projects, I am someone who “contributes a minute”, as you describe it. At work, I often have to evaluate several open-source tools. When I discover a small bug in such a tool, I look into the source code, fix the bug and then submit a patch before I go looking at the next tool. Usually I don’t look at the bug tracker ever again. As most open-source projects use the same bug tracker, it really takes no time for me to submit a patch. But will it be that easy to submit a patch with uservoice? Without really having looked at it (that page is so slow here that I gave up trying to load the demo), I am almost sure that submitting a patch is a functionality that "might confuse normal users" and thus doesn’t exist there.I like your approach to simplify OpenStreetMap and to make everything easier for newcomers who just want to fix one error on the map. Actually I agree with you that at the moment it takes a lot of time to find out how things work here. But I have made the experience that as soon someone (especially a company) tried to simplify a user interface, they hid all the functionality that was important for developers and experienced users (and by that I also mean programmers who only want to “contribute a minute”) and thus made it incredibly hard to use for them. OpenStreetMap depends on those geeks, so when you make it more newbie-friendly, please don’t hide all the technical stuff and never forget that not all users are not developers.

  6. steven feldman

    I think you have touched on a key issue for the OSM community going forward. IF OSM really aspires to scale it is going to have to find ways to engage with a less geoliterate/technical/committed community. There just aren’t enough passionate hardcore people out there. Have a look at the people who worked on the marvellous effort for Port au Prince – 85% of the edits came from 15% of the 200 plus contributors but more importantly for this discussion 50% of the contributors provided just 2.25% of edits. So if we want to capture contributions from the very large number of occasional contributors we need to find interfaces and other approaches to engage them once they have signed up.Oh and can we try to insulate them from some of the flame wars and accusations that have characterised the efforts on the new license. They are a big turnoff for a newbieSadly I speak from personal experience as someone who would like to contribute (perhaps not as a regular mapper) but just can’t work out how.steven

Comments are closed.