“I wish they’d implement feature x.”
“Why won’t they put feature y into core? It’s rated really high in the Ideas forum!”
“It doesn’t matter what I think, all the decisions get made by an elite crime-fighting squad funded by an anonymous millionaire. Er, I mean the four core devs.”
These sentiments, and others like them, are the focus of today’s post. Setting aside the similarities between Ryan, Andrew, Mark and Peter to Charlie’s Angels for a moment, the question of how decisions about features are made needs to be addressed. There are a number of mechanisms in place for communication between the community and the core team, but with so many different channels, it’s hard to keep up with them all and still focus on production. Here’s where we are now…
#wordpress-dev IRC channel: The IRC channel used to be more active. These days there’s rarely more than a dozen or two people online at any given time, and hours go by with no activity. When a question pops up, it’s often a tech question from a less experienced developer or site manager looking for help, as opposed to ongoing discussions about the best way to approach core code and features. When core-focused discussions do occur, they tend to fade out as time zone variances cause people to log off before a core dev enters the room.
wp-hackers list: The hackers mailing list reaches thousands of contributing developers, plugin developers, and lurking interested parties. Discussions range from how to use hooks to whether or not something in core should be changed to troubleshooting for other list members. Conversations on this list sometimes can get heated and occasionally stray into rudeness, which makes some people hesitate to utilize this communication channel.
This dev blog: This blog is used mostly for “official” announcements, and more recently, for surveys and polls intended to give the core devs an idea of community opinion on things being considered for future versions. Posting is irregular, sometimes with new content every other day, sometimes with nothing for a couple of weeks.
wpdevel.wordpress.com: Another blog, also an “official” outlet, in which the core team posts about any big code changes they’re working on. This gives plugin authors and contributing developers a heads-up, and provides a place for community discussion around specific issues like the new widgets API.
Trac: The ticket system used for active development has gotten out of control. Hundreds of tickets are already lined up for future versions because they were punted from current releases; many aren’t even relevant anymore. Trac has wound being a place where people report bugs, suggest code changes, request features and debate methodologies; some of these conversations are years old. This broad use of the system makes it harder to power through tickets and get bugs fixed.
Ideas forum: The Ideas forum is a place where anyone can suggest a new feature, rate features suggested by others, leave comments, and generally discuss the future of the WordPress application. However, like Trac, some of the items here are years old. Because of the way the rating system works, older items remain at the top of the list. Some threads are simply he said/she said preference arguments, as opposed to contructive discussions about the value of implementing certain features or changes. There’s no direct connection between the Ideas forum and Trac.
WordPress is an open source project, successful because of the community that both develops and uses it. At the same time, some people find it difficult to become involved in the project, and are unsure of how to engage with the core team and community at large. The channels listed above can be overwhleming to someone just joining the community, and/or frustrating to longtime community members who feel like they used to have more influence. We need to fix this. The WordPress project needs to be welcoming, easy to navigate as a contributor, and provide useful feedback to help grow the expertise of its community members.
I think we should figure this out together. You, members of this community, know how you feel about the communication channels available to you. You probably have ideas about how to make it better. Some of you may even have sketched out digrams of systems that you think would work better. Link Ideas to Trac? Change the Ideas rating algorithm? Close Trac tickets that don’t get resolved within a certain period of time? Just do everything through Trac? What do you think? What would make it easier for you to keep up with development progress and get involved with the varius contribution opportunities? I *know* you have an opinion.
Over the next few weeks, we’ll be gathering your input about how we can improve communication and participation, and then we’ll embark on a project to fix/create a system for collecting ideas, opinions and feedback that will allow WordPress to grow as an inclusive community. Here’s the plan: Gather ideas from people via IRC, forums, live chats, surveys, etc. Assemble a small group of interested parties to help figure out possible approaches, put suggested approaches to a community vote. If redesigning something (like the Ideas forum) is deemed necessary, utilize community designers to create layouts. Beta test it to see if it does work as hoped. Launch and make everyone happy with the new, improved communication/ideas/feedback system!
Up First
Use this forum thread to post your suggestions about this. What do you think needs to be changed or improved? How would you structure it? How do the existing channels fit into your suggestion?
On Tuesday, May 12 at 21:00 UTC (5pm New York time), hop into the #wordpress-dev IRC channel (irc.freenode.com) and talk about your suggestions for how to improve communication. I’ll be there, taking notes and answering questions, and asking follow-up questions when someone pitches a good idea. An hour later, I’ll be joining the WordCast Podcast to talk about this issue. They’re trying to set up a call-in format; if that pans out, we’ll post the call-in info in the dev channel. Otherwise, A call-in number has been set up through TalkShoe.
1-724-444-7444
Meeting ID: 50127
Pin (if you don’t have a TalkShoe account): enter 1#
We’ll also read off suggestions being made in the dev channel and discuss them.
More opportunities to weigh in on this issue to come. Also, further investigation into the similarities between the core devs and Charlie’s Angels. 🙂
One of the reasons WordPress 2.7 was such a success is the amount of usability testing that took place during the development cycle. Starting with testing 2.5 and the Crazyhorse prototype and following with the 2.7 beta, the testing program looked at almost every feature and function in the application. That kind of thing? Takes a lot of time. 🙂
For readers who aren’t familiar with the process behind usability testing, here’s an overview. First, determine the scope of your test and create a test protocol/script. Determine the audience segments to be included in the test group(s), and begin recruiting. Recruiting may mean hiring an agency to find participants, but for testing WordPress, it makes more sense to recruit from within this community, so that means making a screening survey, reading all the responses, segmenting the respondents into categories and contacting people until you’ve filled your desired quotas (for whatever segments you’re seeking, such as newbie, experienced user, developer, CMS user, photoblogger, mobile user, etc. ). Then come the test sessions.
Depending on what is being tested, these last anywhere from half an hour to an hour and half apiece. Sessions are generally recorded using screen capture and web cam, with a video camera for backup. The moderator(s) generally take notes during sessions and/or (depending on what software is being used for the session capture) set markers in the video to indicate task completion, comments of interest, etc. In some cases, auxiliary test methods such as eye-tracking may be included. When the sessions are complete, the results are analyzed. All the notes and videos are reviewed, patterns are identified, and ultimately a report is written and the feedback informs the next round of revisions.
Some people think it shouldn’t take much time to do all this. I’ve lost count of the number of people who cite an old article by Jakob Nielsen that says you don’t need to test with more than 5 users because usability issues become clear right away. While I’ve found that to be generally true, when your user base is as diverse in experience level, usage, platform configuration, language (right to left languages have a pretty different experience) and demography as the WordPress community is, 5 users really isn’t enough to get a clear picture. We try to test with at least a dozen people each round, but then we are limited to a geographic region (test in NY, test in SF, or test wherever we can schedule enough people back to back to make it worthwhile), while WordPress users are located all over the globe.
To address this, we’re introducing a set of new contribution opportunities to expand our usability testing program. As with development and graphic design, we’re going to create an infrastructure to allow community participation in usability testing on a regular basis and in a much broader capacity than existed before, when it was limited to announcements that we needed participants in x city on y date. We’ll be looking for volunteer moderators as well as participants, hopefully from all over the world.
Moderators. Observational usability testing isn’t rocket science, but neither is it a simple task to reduce bias. Because of this, at first we’ll choose only moderators who have professional experience conducting usability tests. People who conduct testing for design agencies, software companies, usability consulting firms and the like will be our first round draft picks. In the future, when we have a group of regular volunteers and have ironed out any kinks in the process, we’ll ideally match up experienced testers with aspiring ones, using a mentorship model to increase the number of people who can contribute in this fashion.
Participants. If you use WordPress, chances are you could participant in a usability test at some point. In some cases we’re looking for particular behaviors (people who upload large video files, people who blog from their iPhone, people who manage more than 5 blogs, etc.), while other times the behaviors we’re looking for are much more common (do you have widgets in your sidebar, have you changed themes in the last 6 months, is there more than one author on your blog, etc.).
So how will these opportunities come into play, and how will it make WordPress better?
We’ll start with the moderators, and try to get volunteers with a decent geographic spread. Then, we’ll start signing up potential test participants in those areas (though we’ll also allow at-large registrations, since traveling testing will still be happening). We’re working on a registration process for potential participants. You’ll enter your basic info (location, contact info) and answer some questions about your WordPress usage to be entered in the database, and when there’s a testing opportunity coming up that’s appropriate for you, a local moderator will get in touch to see if you’re interested. Further down the road we may experiment with remote testing and other methods, but for now, this approach will broaden the geographic scope of our testing.
All moderators will follow the same test protocols and script, and their results/reports/video will be reviewed and collated, with a composite report (including the protocol/script that was used) published to the community. This will provide designers and developers with broader feedback during the dev cycle, and will allow the wider community to both understand and participate in the testing program.
If you’re interested in being a moderator during this initial phase (meaning you do it professionally), send me an email and introduce yourself. If you’re interested in signing up as a potential test participant, watch this space. We’ll post a link to the registration survey once it’s ready.
Recent Comments