A few people have complained that they feel the review process is inconsistent. I’d like to take a moment to explain exactly why that happens. The tl;dr is, of course, humans make mistakes. But if you really want to understand what’s going on, read on!
There is no automated review process
This is the big thing. Every single plugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is opened and read by a human being. We download the plugin, read it, and try to catch the myriad things that are wrong, insecure, not permitted, etc. And we’re humans. We do our best to scan/grep for things we know are easy to find (like I love checking for wp-(con|load|blog) to see if that’s being called). But a lot of times things are buried or hard to catch.
This means mistakes are made. We don’t claim to be perfect. We claim to try our best to give you the best review we possibly can for your sake, as well as your users.
Some replies are canned, the process is not
I’m sure a lot of you have gotten an email starting with this:
There are issues with your plugin code. Please read this ENTIRE email, address all listed issues, and reply to this email with your corrected code attached. It is required for you to read and reply to these emails, and failure to do so will result in your plugin being rejected.
Yes, that’s a canned auto-reply. In order to get through reviews faster, we have replies for the common issues. Right now I have 60 in A-Text. That means there are at least 60 problems with plugins I see every single day.
This makes us able to keep up with reviews. It’s impersonal, we know, but we try to cite examples from your plugin to help you understand what needs your attention.
We don’t test your plugin on all environments
Sometimes we do. But really that’s your job, not ours. We do if we notice things that are weird and we think may be problematic. Some days we test on VVV with PHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. 5.6, sometimes it’s HHVM, and sometimes its PHP 5.2. Why? It depends on what we have available just then.
This means sometimes we catch that you coded something for PHP 5.3 and up and sometimes we don’t.
Every new version is checked top to bottom
Think about that for a second, please. If you submit a plugin and we pend it for changes, and you send us the new version, we read the whole thing all over again. Every. Single. Time. We check to make sure you did your changes first, yes, but then we go back and re-read everything to make sure we didn’t miss anything, or you didn’t accidentally add in something new.
This is why, sometimes, you get an email that starts with “We missed this before…†or “This is also not permitted because…†We’re giving you the best review we can.
No, we don’t list everything wrong
It’s not what you’re thinking. Every time we do a review, we list everything we see that’s wrong. We do not list out, for example, every instance of a non-sanitized/validated POST call. We do not list out every single usage of script tags instead of enqueues. We will give you an example, especially if you miss some on your first edit, but we expect you to know how to search your own code.
This helps you learn how to better vet and review your own code. Also it saves us a little time.
There are multiple people doing reviews
Some of us are better at some thing than others. When we find a plugin we don’t feel confident in reviewing on our own, we raise a flag and ask our cohorts to spot-check our work.
This also lets us hand off troublemakers. Let’s be honest here, folks, we don’t all get along with everyone. When it’s clear we’re at an impasse with someone, we ask each other for help.
Our goal is protecting users first, then you
The people we care about the most are the users who can’t code or who don’t understand the severity of things like offloading CSS CSS is an acronym for cascading style sheets. This is what controls the design or look and feel of a site.. You may think it’s trivial and makes your plugin smaller. Someone in another country could find them sued for not disclosing it. Or your plugin may not work because Google is blocked where they are.
We care about protecting the users from XSS and SQL injections. We care about protecting their information. We care about keeping them safe. But we care about you too! We’re so techy about you documenting ‘This plugin calls service XYZ’ because, yes, the users have a right to know where their data is going, but also because you deserve not to have a slew of angry 1-star reviews that you didn’t tell them.
This is a tricky road to walk. Some people may get exceptions. Some people may teach us more about code! Some people may be told ‘no’ flat out.
Guidelines evolve over time (so do security best practices)
We’re constantly looking over the guidelines and evaluating them for clarity, consistency, supportability, and real-world applicability. Have you read our Detailed Plugin Guidelines lately? You should. Similarly, our security checks have gotten better over time. We used to allow you to call wp-config.php directly. We don’t anymore. The more a specific vulnerability is targeted, the harder we are on your code to ensure you are not the weakest link.
This is for your protection! We’re doing our best to make sure you don’t get dog-shamed for being the reason sites go down.
Remember: We are mortal
I said this to start off this post and I’ll say it again. We, your review team, are human beings.
We make mistakes. We miss things. We read code incorrectly. We don’t test everything as fully as we should. We screw up. We never miss things out of maliciousness or an intent to blacklist you from the repository. We believe you submit your plugins in good faith, and we respect you enough to treat you as adults and point out what you missed or explain how you can do things better.
This means you should give us the same benefit of doubt we give you.
#reviews
