Ulrich 4:57 am on May 17, 2017  

Comparison between the Theme and Plugin review process

  • As we are discussing how we can improve the Theme Review process it would be good to see how our process compares to the PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party Review process. It would help both teams to align so that we could both profit from improvements made to either system. Both processes are always changing. Parts of the process today may change tomorrow. The plugin review team is working on making the plugin review process public so that others can contribute.
Review Process Themes Plugins
Submission zip file  zip file
Review management in tracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. (Public) in the WordPress admin area (Private)
Review updates  via trac via email
Reviewers Anyone with a w.org account select group of experienced reviewers (will change in the future but only experienced reviewers will communicate with the developers)
Number of reviewers varied between 10 – 30 active reviewers 5-6 reviewers
Experience level of reviewers variable highly experienced
Reviewer training regular  not needed
Order of reviews order of submission  order of complexity of the plugin
Volunteer time minimal sponsored time sufficient time sponsored
Approval A final review is done by an experienced reviewer before going live The review approves the theme and then the developer must commit the code to SVNSVN Apache Subversion (often abbreviated SVN, after its command name svn) is a software versioning and revision control system. Software developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS). WordPress core and the wordpress.org released code are all centrally managed through SVN. https://subversion.apache.org/.
Theme Updates Via zip files via SVN
Theme update reviews only automated reviews no additional reviews
Feedback on the review process Anyone can give feedback and work to improve the review process The small group of reviewers make the decisions
Automated Testing checking for common issues simple testing for PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. errors
Requirements A single detailed document of all of the requirements. Multiple pages on the guidelines: The general guidelines, a reviewer’s checklist and a page explaining how to check for these items.

My learnings from this have been:

  • A small highly experienced team with the resources can sometimes been more effective than a larger team.
  • Having separate information for developers and reviewers may change the public perception of the number of requirements.
  • As the plugin and theme reviewers are looking for similar issues it may be better to work together to unify some of the documentation.