UX Review

Dear all

One of the clear pieces of feedback from all the talk about improving the OSM UX was “show us these users who really find it difficult to use OSM”. So, we’re going to do that. We have a rough plan of action below

Nate Bolt of the splendiferous boltpeters.com has volunteered to help OSM with a user interaction review. Nate, as part of Bolt | Peters, did exactly this work with wikipedia:

http://usability.wikimedia.org/wiki/Usability_and_Experience_Study

Where they basically screen recorded people trying to edit in wikipedia with the following goals:

• identify obstacles that novice users encounter in editing a Wikipedia article—including, but not limited to—adding personal content, fixing a typo, adding a reference, and contributing to discussion pages
• identify obstacles in creating a new article
• evaluate the self-sufficiency and legibility of help materials and documents found on Wikipedia.org
• evaluate how novice users interact with templates
• discover user experience patterns and issues that have not been previously identified.

Together with Mike Migurski, famed geohead, creator of walking papers and graphical butterfly at Stamen.com, we’ve come up with a rough plan to get feedback from new users to OSM. It’s a very similar approach to that taken by Bolt | Peters with wikipedia. We want to find out from the real newbies what the issues are and lay them out clearly. Nate is independent of OSM and will be able to present a cold hard look at what’s good and what we need to work on to improve things so we get more newbies contributing.

* A small piece of javascript from ethnio.com is put on the new user page in openstreetmap
* Once in some very small sample size (perhaps between 1 in 1,000 and 1 in 10,000 signups) a popup appears
* The popup says something like “Hi! We’d really like to know why you came to OSM” and they say simply why. This is open ended on purpose so we catch as many things as we can, not just what we’re looking for, but things we won’t expect.
* They’re offered to record a short (10 minute max) screencast of them trying to achieve whatever it is (like look at a map, find OSMers, add a PoI and so on)
* That screencast is analyzed in aggregate with many others by Bolt | Peters with all their expertise in doing this stuff, and they come back with a set of findings.

We’re looking at both http://www.usertesting.com/ and http://www.openhallway.com/ to do the recording. Both Mike and I will pay for it, and might solicit donations and stuff if it looks beyond our budget.

I can’t say this enough, and I always get responses from people who think that I just set something out in stone – so I’ll be super super super clear: NONE OF THIS IS SET IN STONE. We need your feedback on everything before we go ahead. I mean _everything_. So, please tell us what you think about it all. Here are some questions:

What should our goals be? (General UX? How good/bad signup is? How good/bad editing is? How is it finding info?)
How often should we ask a signup for feedback? (the more the better but we can only look at so many)
How can we include more crowd source feedback? (I think of asking random signups for feedback as crowdsourcing it)
What else should we think about?

Yours &c.

Steve