Ensuring that a theme follows all WordPress Theme Directory requirements can be challenging. While theme requirements are important for maintaining a high standard for themes that are distributed through WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/, checking that themes meet these requirements can be time-consuming for reviewers. Additionally, sometimes theme authors aren’t sure whether their theme meets our requirements, before submitting a theme to the directory.

The Theme Review team maintains a number of different tools and datasets to help theme authors. The Theme Check pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and Theme Sniffer do a good job of informing theme authors of problems with their code but aren’t able to cover some of the tricker aspects of theme reviews, as both tools run static code analysisStatic code analysis "...the analysis of computer software that is performed without actually executing programs, in contrast with dynamic analysis, which is analysis performed on programs while they are executing." - Wikipedia. As a result, theme authors & Theme Review team members spend countless hours manually testing themes. 

If we could combine some of the existing code analysis tools, automate away some of the manual testing and open them up to more development workflows, could we improve theme quality, alleviate pressure on manual testing and speed up the theme review process?

The Theme Review Action project is looking to answer this question.

It’s also worth noting that this project was loosely informed by the Theme Review’s proposed updates to the overall process designed to improve communication during theme reviews. This proof of concept doesn’t intend to be a replacement for that work, but hopefully an extension.

Guiding Principles

The project should:

• Go beyond simply identifying where themes don’t match our guidelines and help improve theme quality
• Leverage existing tooling
• Provide actionable feedback to theme authors
• Be able to run in different contexts
  • Continuous development platforms
  • Local machines

Proof of Concept

In the last quarter of 2020, we worked on getting something in a state for developers to try, in the hopes of sparking discussion and educating a path forward. 

The project has been added to GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/: https://github.com/WordPress/theme-review-action.

How does it work?

As mentioned in the project’s Readme.md on GitHub, you can test a theme by doing any of the following:

1. Navigate to your WordPress theme folder and run npx wordpress-theme-check-action
2. Add the Action to your WordPress Theme Repository on GitHub
3. Clone the project and run it locally.

Once triggered, the project roughly does the following: 

1. Creates a development environment using wordpress/env (& docker)
   1. Installs the latest version of WordPress
   2. Installs parent theme if applicable
   3. Imports the test data
   4. Activates theme
2. Tests to make sure theme has minimum required files
3. Runs Theme Review Plugin
4. Opens a Browser Session programmatically and runs User Interface tests.
5. Outputs results of tests

The User Interface tests use the Browser (Puppeteer), mimicking a user, to answer the following questions :

• Does your theme have appropriate: 
  • Skip links?
  • Element focus states?
  • Keyboard navigation?
• Does your theme follow other WCAGWCAG WCAG is an acronym for Web Content Accessibility Guidelines. These guidelines are helping make sure the internet is accessible to all people no matter how they would need to access the internet (screen-reader, keyboard only, etc) https://www.w3.org/TR/WCAG21/. AccessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) standards?
• Are there any errors in your:
  • JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/.?
  • HTMLHTML HTML is an acronym for Hyper Text Markup Language. It is a markup language that is used in the development of web pages and websites.?
  • PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php.?
  • GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ BlockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. Templates?
• Does your theme include prohibited content?

Of course, since this is only a proof of concept, the test coverage isn’t extensive and will require more refinement. 

Example 1: Here’s an example of it running locally using npx. 

Note, running using npx on windows isn’t working yet :).

Example 2: Here’s an example of its output GitHub:

What’s next?

Can you try it out on your a theme and answer any of the following questions:

• Did you find it easy to use?
• Does the information help you make changes that improve your theme? 
• Did you find any bugs?
• Is there anything missing that could be added?
• Although it’s currently a little slow, can you see yourself using it during your development process (or theme review) if it were faster and more complete? 
• Is there an alternative approach to consider?

Ready?

Check out https://github.com/WordPress/theme-review-action and read the Readme file for more detailed information on how to use the project. Any issues or feedback can also be logged as issues in the GitHub repository. 

Sincere thanks to @poena for the help!