Displaying PHP Compatibility results from Tide on wp.org

[JoshuaWold]

On Github we've been discussing how we can show compatibility results for PHP versions on WordPress.org for plugins. You can read more about the discussion and design decisions previously: ​https://github.com/wptide/wptide/issues/152

This Trac ticket is to kickstart that discussion on Trac and try to move forward.

[ottok]

I suggest you start out with something much more simple. Instead of the modal thing we could start up with a dead simple list of supported PHP versions.

There are two sources of information to consider here. First of all the current "PHP Version" line uses the "Requires PHP: N.N.N" from the plugin readme.txt. This should still be honored and used to set the minimum PHP version supported in a list fo supported PHP versions.

The second source of information is the JSON report by WP Tide:

"compatible_versions": 
[
  "5.6",
  "7.0",
  "7.1",
  "7.2",
  "7.3"
],

From this list all values above the readme.txt stated minimum version should be used.

My suggested UI is below. These examples are hopefully self-explanatory.

(Ticket owner: please delete attachment https://meta.trac.wordpress.org/attachment/ticket/4309/compatible-with-up-to-latest.png, it was the wrong mockup.)

[dd32]

How often is it likely that a plugin that was compatible with PHP 7.0 and 7.2, but incompatible with PHP 7.1? Does displaying each version of PHP that's compatible useful to the average user or developer?

Wouldn't we be better off sticking with minimum and potentially maximum supported versions of PHP?

[BrashRebel]

Could we make it consistent with the way the WordPress version compatibility is presented now?

"PHP Version: 5.6 or higher"

[JoshuaWold]

Thank you @ottok for that wonderful mockup!

I think the answer here comes down to what's actually needed for users. If there's situations where, as @dd32 mentioned, I could be compatible with 7.0 and 7.2, but not 7.1, then we should go the way of the mockup @ottok suggested.

Otherwise I agree with @BrashRebel that a simple statement works.

@JeffPaul do you or Derek know the answer to this? :)

[dd32]

In 8975:

Plugin Directory: Add a job to import data from Tide into the plugin directory.

This job isn't yet in use, and a follow up commit will enable it.

See #4309.

[dd32]

In 8976:

Plugin Directory: Import Tide data for plugins after commits.

See #4309.

[dd32]

In 8977:

Plugin Directory: Tide Sync: Handle the case of an empty tide summary report.

See #4309.

[dd32]

In 8978:

Plugin Directory: Tide Sync: Only support plugins who use versioned versions, ie. no stable tag: trunk.

See #4309.

[ottok]

Thanks @dd32 for working on this! Attaching mockups of the very simple UI version.

If the plugin is not compatible with latest PHP version there should be a link to a page that explains how to run PHPCS yourself and fix it (for plugin authors), and something for users about why PHP compatibility matters.

[JoshuaWold]

These looks great. Thanks!

[afragen]

Does this mockup mean the current Requires PHP: header is being replaced ?

[dd32]

Replying to afragen:

Does this mockup mean the current Requires PHP: header is being replaced ?

No, anything we do is in addition, or as a fallback for a plugin not specifying it.

I don't think we'll be adding an upper bound of supported php either for the time being, due to difficulties in parsing that with confidence.

[afragen]

Should it say Compatible PHP as there will also be a Requires PHP? As requires is a minimum Tide data would be a bonus.

[BrashRebel]

I just wanted to raise one other concern real quick: With this addition, "Tested up to:" could become a new point of confusion. We would have for example:

...

• WordPress Version: 4.9 or higher
• Tested up to: 5.2
• PHP Version: 5.2 to 7.1

The second item does not state what it pertains to. WordPress is implied currently but if we add another item to the list with an independent version number, we could introduce confusion. The matter is potentially exacerbated by the fact that numbers will commonly be similar since coincidentally WordPress has now reached 5.x which is close to much of the supported PHP numbering.

It might be wise to consider revising all three labels for clarity if we are going to introduce independent version numbers.

Separate, less important question: Could we introduce capitalization consistency throughout all the labels?

[JeffPaul]

@ck3lee @dd32 @ipstenu assuming we're able to get to an agreement on HOW the PHP compatibility data from Tide should be displayed on the plugin pages and a patch is created for this update, what would be the process to get that approved and committed?

[Ipstenu]

We need to keep Advanced View, as that's also where you manage contributors. Unless you're pulling the WHOLE thing into the Development tab.

TBH, I'm actually not the voice to decide what the page looks like. I just make sure I have access to what I need. I defer to designers and data folx to the rest. I would want to get a designer in on this to make some educated decisions, and if @tellyworth and/or @otto42 like it, go forward :)

[dd32]

I had a number of conversations with others at WordCamp US, and one thing that I heard multiple times was that hosts wanted a way to say "This plugin works on PHP 7.3" - I asked if Tide would suffice for that, and the main thing I heard from that was basically a no (from those who had an understanding of it).

The known false positives/negatives of the Tide data meant that those who I spoke to weren't comfortable using it for that purpose, and I tend to agree - which is why there hasn't been any movement here.

IMHO

• The proposals to show the PHP versions supports (Either as a list, or a range) is pretty "meh" to me. It's not useful to end-users. And hosts evidently can't rely upon it.
• My proposal to fill in the Requires PHP header in from the Tide data seems worthwhile, but I don't think we actually can. Since WordPress now prevents updates unless you meet the PHP requirements, a false negative here just has the possibility of stranding users from getting updates until an Author adds the header.

Polyfilling would result in (This is looking at plugins who don't specify Requires PHP but that tide has processed)

• 1044 plugins marked as requiring PHP 5.2 (That's actually good?)
• 814 plugins marked as requiring PHP 5.3~5.5 (That seems legitimate, but also not really useful to modern WordPress installs given the PHP 5.6 minimum)
• 76 marked as needing PHP 5.6 (I'm surprised it wouldn't be more, I would expect a number of plugins to have jumped on board with the PHP bump)
• 25 as PHP 7.0 (4 of which I can immediately see require PHP 5.2/5.3), 6 as 7.1, 1 as 7.2, and 6 as 7.3

Looking at the Requires WordPress header, the numbers also just don't stack up.. For those who require PHP 5.6 according to tide over half of them require WP 3.x and 80% is 4.6 or older.. The PHP 7.x's with only 5 plugins out of 38 (~13%) requiring WP 4.9+
It's entirely possible that authors have simply not updated that header, and have decided to require a much more recent PHP version.. either intentionally or accidentally, but either that's most authors, or the data is wrong.

I'm just not seeing a real benefit here to the majority of users. For the 38 plugins that need PHP >= 7.0 based on Tide, It's probably worthwhile to email the authors and get them to update their plugins with the Requires PHP header to find out what they actually need... see how many of those Tide got right.

Arguably what this is showing is that the upper-bound limit of the Tide data is more useful, ie. "What's the highest version of PHP this plugin will run on" but I also struggle to trust that data:

Maximum PHP supported by plugin	Plugin Count	SUM() Active Installs
5.3	2	60
5.6	141	1.6 million
7.0	42	480k
7.1	44	5.6 million
7.2	589	20 million
7.3	2772	125 million

(Interestingly, PHP 5.2/5.4/5.5 is never the maximum supported php in the data we have)

The Active installs is all the plugins in that group with their install counts sum'd together (So that will mean lots of duplicate sites) which when put up against the number of WordPress sites running on PHP5.6 and >PHP7 suggests that these plugins are more than likely compatible with a higher version of PHP as otherwise the forums would be full of complaints.
I looked at the PHP 5.6 plugin list, and quite a few of them work fine on PHP7, but probably contain back-compat functions or other similar things which makes them incompatible according to the tide tests.

So that's why I've personally stalled on this ticket, and why I question the benefit of exposing the Tide data.

I tend to think that we can probably get a better "What's the chances this plugin works on PHP 7.3" for a lot of plugins based on WordPress.org stats (Although we don't currently expose, nor count, PHP usage by plugins) by looking at the number of sites that a plugin is running on and going "Well, 10% of the plugins install base is now using PHP 7.3 - It's highly probable that the plugin works" and combine that with adding the "Tested up to PHP version X.Y" header to readme's to get actuals from authors who do test.
Same thing for "What's the chances this plugin works with the latest version of WordPress?" "Pretty high, 80% of the users are using X.Y"

[JeffPaul]

Thanks for the detailed response @dd32. I'm curious if you, or the folks you spoke with at WCUS, see a benefit to providing this information privately to the plugin authors so they can work to update/resolve issues on their end and thus improve the accuracy of self-reported data? Also curious if you think this approach should be totally abandoned in favor of helping the Plugin and/or Theme Review teams automate some of their review processes with Tide?

Essentially, I'm reading your comment as a major roadblock in continuing with our current approach/plan for Tide and wanting to see what you'd recommend we pivot and focus on instead.

[dd32]

Replying to JeffPaul:

I'm curious if you, or the folks you spoke with at WCUS, see a benefit to providing this information privately to the plugin authors so they can work to update/resolve issues on their end and thus improve the accuracy of self-reported data?

Speaking personally - I've always held the opinion that clear steps authors can take to run tests on their own is a benefit to the greater community, unfortunately getting most of those things up and running is way too complicated for most users (WPCS is impossible to use for some for example), which is one thing that may have ultimately lead to Tide being a thing.
This doesn't invalidate the need or intentions of Tide, only that Plugin and Theme authors shouldn't have to rely upon WordPress.org or a service to be able to get that data.

Also curious if you think this approach should be totally abandoned in favor of helping the Plugin and/or Theme Review teams automate some of their review processes with Tide?

Yes, I think both of those teams need more automation, but I personally question if Tides current model can easily apply to that though.
The Plugins and Theme teams currently most need instantaneous checks, things that can run in a few seconds and validate no major issues, potentially then followed up with a more detailed set of checks being run. Both already have tools setup for this, but both need iteration and far more checks being added but lack of resources in doing that.
The Tide Output *may* fill some of the secondary-check requirements, but I have no idea, I've never compared what those teams look for (And in the case of the Plugins team, some checks are private due to the nature of malicious actors) to what Tide outputs (I've only ever seen a tide report second-hand, and that was a long time ago)

Essentially, I'm reading your comment as a major roadblock in continuing with our current approach/plan for Tide and wanting to see what you'd recommend we pivot and focus on instead.

I don't want to act as a roadblock here, but acknowledge that I am given the above.
I'd love if the current and original plans for Tide could be completed, but I think automated scanning with coderule sets is only going to go so far, it can't be reliable for many hard yes or no situations, only be pushed into more general "code quality scores". It's a question of "Does this provide certainty? Or does it provide recommendations and potential red flags someone needs to review manually?"

If you look at the way the Theme Check rules have worked, they've outlawed certain things in a blanket rule that has lead to authors misusing other things - for example file_get_contents() in favour of WP_HTTP and WP_Filesystem has resulted in many legitimate uses of the function to be done in hoop-jumping ways where it should've only ever been done as a "RED Flag! Is this being used to request a URL? Bad! WP_HTTP! Are you trying to Write to theme files? Uh.. probably shouldn't be doing that.. but WP_Filesystem exists for interactive requests and you probably want to use that" (This is just an example).