The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
If you’d like to help test Full Site Editing, please join the FSE Outreach Program. You can find current calls for testing for this program here and you can join the fun in #fse-outreach-experiment.
The team gathers in #core-test. Please drop by any time with questions or to help out.
Welcome to this week’s edition of Week in Test! This post is a curated list of where you can get involved (i.e. where testers of all skill levels and expertise are needed), learning opportunities, and some reading to keep you informed.
Looking for ways to contribute? The following tickets and patches need contributors.
Manual testing help needed
Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.
The following tickets need testers to manual test and provide feedback (test report):
#53801BlockBlockBlock is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-based widgets screen does action wp_footer after each widgetWidgetA WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.
#16206: (test patch 16206.8.patch) Comment text not marked as required
#54243: Inlined block styles for external assets (images/fonts) and relative URLs not working as expected (patch is upstream in GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/)
PHPUnit tests help needed
Who? Any QA or PHPPHPPHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.
#47642: Order by comment count – posts list tables
#49985: REST APIREST APIThe REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
#52252: PHP Notice when monthnum query var is set without the year QV
Reproducing reported issue help needed:
Who? Any contributor.
The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:
Currently no tickets in the queue are ready for testing
Juliette and I hosted a live working session to fix and test infinite loopLoopThe Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. (#52241), incorrect data type, and PHP 8.1. Watch replay here.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. It was intended to be a casual walkthrough of the current call for testing with @sparklingrobots taking the lead sharing their screen. Big thank you for being brave enough to meander around the call for testing publicly.
As a reminder, there is still 1 week left to participate so please join in if you can.
Throughout the session, some specific issues were noted as problem areas to follow up on, including some previous reported and some new:
It’s not clear when you are adding a theme blockBlockBlock is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. outside of the Query LoopLoopThe Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. vs within. In this case, it was confusing when the Post Comment Count block wasn’t properly displaying the number of comments and it wasn’t clear the issue was that it needed to be within the loop. There’s an open issue to discuss this general phenomenon.
She had a desire to change the post excerptExcerptAn excerpt is the description of the blog post or page that will by default show on the blog archive page, in search results (SERPs), and on social media. With an SEO plugin, the excerpt may also be in that plugin’s metabox. length, which has an open enhancement issue.
There was an issue with the post title overflowing outside the bounds of a Query Loop block pattern. Need to replicate and open an issue!
There was a strange Inserter issue where you couldn’t find any theme blocks based on the current block you were selecting. Need to replicate and open an issue!
The call for testing itself needed to have updated instructions as you need to install GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ first before activating a block theme. This was updated live on the call.
Outside of specific points of feedback about the experience, there were also high level themes that became apparent as @sparklingrobots made her way:
They were confused about where various settings were expected to be, sometimes seeking out the block toolbar and other times the block sidebarSidebarA sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme.. For example, it was unclear at a glance how to change width of overall query and how to change the number of posts displayed.
The placeholders for various theme blocks were underwhelming and often not informative enough to know quite how to customize.
She was confused by how by selecting one Query Loop pattern, they instead ended up with two Query Loops! This is part of one of the default patterns included in the Query Loop block and could cause confusion long term when thinking about having more complex patterns.
There was a desire for more dimension controls for various blocks including the Columns Block and Post Comments block. This is under active iteration!
We ended the call talking about how the future of the Query Loop block powering more user friendly variations, an integrated block pattern library, an overhaul of the IA of the various design tools, & more will help ease this current experience. For now though, we’re in an in between state where loads can still be learned to improve the default tools themselves.
Next Steps
@annezazu will follow up next week after she returns (she’s out Thurs/Fri) to replicate and open issues.
Where e2e tests live: GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ vs. CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress.
Consensus reached as to where e2e tests will live within each repository:
Core: non-Gutenberg features, UIs, and UXUXUX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. including:
new stuff being built during a release cycle
existing interfaces such as CustomizerCustomizerTool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings., login, pluginPluginA plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme installation/activation, interaction with list tables, settings page, etc.
Gutenberg: its functionality being built within its repository
5.9 Core e2e focus areas
All recent major changes to Core with user facing functionalities are top priorities to receive e2e tests as part of WordPress 5.9. These areas are:
The application passwords feature
The uploading of new versions of plugins/themes feature
Tests that test Gutenberg’s functionality will remain in its repository and not be ported to Core
Core specific tests (such as login) can be moved to Core
Along with the previous areas to test, the following priority would be to port some Core tests that are currently implemented in Gutenberg (e2e-test-utils package). Concerned tests are those related to the login feature.
The next step after this would be to improve the login tests to make them more performant. This includes for instance cookie based authentication across all tests in Core and Gutenberg.
Other Roadmap Items
These items are not necessarily part of 5.9 Goals. However, these are part of the bigger picture Test Roadmap.
Documentation: The goal is to help contributors quickly contribute to testing.
Test Handbook: Clearly document multiple workflows for folks to pick their onramp into testing
Visual Regression: #49606 is an experiment to allow local vision regression testing.
From these learnings, plans can be crafted for how to build it into an automated CI process.
The challenges for the CI are storage of the artifacts and unreliability of testing these across different environments. A third party service may be possibility to explore in the future.
Is Core a good experimental sandbox for Playwright?
Proposal to migrate to Playwright for e2e tests
tl;dr:
Not a blocker to build Core’s e2e tests
Requires changes in e2e-test-utils package (maintained by Gutenberg)
Needs a migrationMigrationMoving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. plan
All agreed Playwright for e2e tests would indeed have a lot of advantages for both Core and Gutenberg. However, consideration must be given for breaking changes and impacts for extenders using the test utils. Kai noted the utils could be made compatible with both Puppeteer and Playwright, meaning no breaking changes.
Also, a migration plan will need to be discussed on developed once the test utils are ready for Playwright. Part of this plan is to figure out how to change Gutenberg and Core as well as how to upgrade existing, not yet committed patches/PRs. The how and when is yet to be determined.
As there is still work to do to prepare for Playwright, the team agreed to continue with the roadmap to build e2e tests in Core.
This post is a summary of blockBlockBlock is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. theme switching exploration for the FSE outreach program. This was the first of its kind, enabling folks to share very early feedback on something that has yet to be robustly defined. Thank you to everyone who participated, whether through sharing feedback directly or sharing the exploration with others.
Shout out to @richtabor@elmastudio@anariel-design who officially got badges for responding, despite having engaged with surveys the program has done in the past.
Overall, the current experience proves to be frustrating and inconsistent, especially when taking into account custom block styles, keeping customized templates, etc. Thinking long term about what folks would want to be able to have across themes, there was mass consensus around being able to retain templates, template parts, and menus. There was somewhat mixed feedback around whether Global Styles should persist as some saw those as differentiating a theme. When it came to ideas for how to best manage the switching process, it quickly became clear that there’s a balance to strike between not adding too much friction to the process while also offering users options to pick and choose what can come with them when they switch. Ideally, there can be a simple and visual way to intuitively guide users and help them take advantage of the power of what block themes unlock without discouraging them with too many options.
On templates and template parts
There was mass agreement around the desire to keep customized templates and template parts across themes, with many expressing surprise and frustration at the current experience. This was previously documented and discussed here as part of an earlier call for testing.
I’m very surprised that any templates I’ve created are tied to the theme that was active when I created them. I’d expect that my custom templates should remain applied to pages when the new theme is active, instead of being disregarded. I’m not sure why templates are omitted when a theme is changed.
I would like to be able to use templates and templates that I have created and saved, no matter which theme that is active. I know that I can view them under appearance templates/template parts, open them, copy the code and paste it into a new template, but I don’t think that is a good experience. It should be easier.
Similar to templates and template parts, this was another area that folks inherently expected would persist across changing block themes.
An issue I’ve ran into now a few times when trying out different Full Site Editing themes is that losing menu data is frustrating. I think as a non-technical user it would be confusing, because you are prompted to “Add an existing menu”, which I would think would be my menu from the last theme I was using.
I think it is important that navigation blocks that I have set up remains. The “Add existing menu” feature in the navigation block assumes that I have already created a menu in the navigation screen. If I only setup the navigation block as part of a headerHeaderThe header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. template part in the previous theme, then I can’t re-apply or reuse that navigation block. Perhaps navigation blocks should also work the other way around? I mean why can’t I select a name for my navigation block as I create it in the editor, save that in isolation like I can save the site blocks in isolation, and have that navigation menuNavigation MenuA theme feature introduced with Version 3.0. WordPress includes an easy to use mechanism for giving various control options to get users to click from one place to another on a site. present on the navigation screen?
Understandably global styles settings would adapt when a theme is changed (just like the customizerCustomizerTool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings.) – and I like how my custom GS settings persist when I change back to a theme (just like the customizer as well).
When you export the demo and import it to the other installation, theme.jsonJSONJSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. file styles are not imported. As a theme developer, I would love to develop one theme with different demos for example. When I export the demo file I would love that global and block type styles are exported too and imported to other installations.
Have the option to keep Global styles modifications. Perhaps a kind of dialog box that shows up when entering the Site Editor listing adjustments I made to the previous theme, asking if I wanted to keep these adjustments or to start anew.
One question that keeps me up at night is how cross-theme compatibility will work on the content level. Default block output should translate from one theme to the next with little or no issues. However, custom block styles, font sizes, colors, and the full range of presets are already a problem area.
Outside of a desire for the experience to be overall easier and more seamless, the following ideas were shared with a split in terms of folks who wanted decisions upfront vs after switching:
Offer an option to pick and choose what you want to keep before switching themes.
Make switching easy upfront but, after switching, offer an option to import various items from the previous theme.
Offer a side by side visual comparison of various parts of a theme before switching (templates, patterns, etc).
Offer a way to import a color palette or template into your current theme so you don’t have to switch fully but can take advantage of different pieces.
I have experimented with one theme but figured out along the way that it does not have the patterns or finished templates or something else I had hoped for. Instead of creating the patterns and templates myself I switch themes. When I click to switch a theme I get a warning message saying that switching themes will remove the adjustments I made to the current active theme, but I have an option to save these adjustments in a kind of twilight zone between one theme and another. I select to save changes I made, and notice that these carry over to the new theme that I activate. I check and notice that the changes do carry over. I am relieved that I am able to create adjustments in one theme and have these with me to the next time…In the Site Editor I can check out what the new theme offers and when I feel ready for it I can either say yes to bringing over the changes or no because I notice that the new theme has what I need.
I actually don’t want to be prompted with having to make several decisionsas soon as I activate a new theme. I would find that stressful. I want to take my time. I want to understand what the differences are between the themes, and what changes I need to make. Perhaps there would be a side-by-side comparison of common page templateslike page, single post, home? Like a revision?
It could also be awesome to pull a color palette and drop it into an existing theme. Sort of like having a Colour Lovers directory to pick color schemes from but keep all the other bits. This could be fun for people who can recognize a palette that they like but would never be able to handpick all those colors. I’ve often seen color schemes that I’d love to use from other themes but didn’t like other things about them.
Of the various questions folks could answer, some touched on both reasons for switching and the current experience. I’ve listed each response below since only a few folks addressed this area specifically. I’m also including images from @greenshady’s post where he took a simple blog post with some custom block styles, gradient colors, and font sizes and compared the output across three different themes highlighting current problems with theme switching.
To see prebuilt template layouts (could be done in a template mosaic view to where I can choose various prebuilt layouts instead of switching themes). To have a base that I want to start from. A design that I would like to use and modify.
I think the most common scenario is a missing functionality in one theme like WooCommerce support. Next would be outdated design and lack of updates and support from the theme author.
When I switched to the Quadrat I mostly lost everything that I set up in the Clove theme. That means, About page doesn’t look anything similar, colors, fonts are now from the Quadrat theme and button style too. From the user’s side, this is very confusing. If u ever used Elementor for example, and many are using it they are used to the similar overflow. If I create a template and change the look and styles and switch to any other theme this template will look the same and it will remain available.
I am not one for switching themes. Since I learned how to design for WordPress well over a decade ago, I have never moved from one theme to the next. At least not in the same way that the average user would. Instead, every time I have added a new coat of paint on my websites, I have simply switched over the foundation to whatever I had been working on at the given moment. WordPress themes, for me, were always just an iteration upon the last project…The first thing I do when testing any theme is to load a demo post. Lately, this has been the “Welcome to the GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ Editor” test post. The primary question: Can I read the content comfortably? If I do not get past this stage, I simply deactivate the theme.
@critterverse is exploring how to approach these flows from a design perspective and has been following along as feedback has come in. You can expect to see a more in depth design exploration shared soon enough with some of these pieces of feedback and ideas integrated in! I’ll flag this in the outreach program channel when the time comes and will see how we can explore these experiences in future calls for testing.
Welcome to this week’s edition of Week in Test! This post is a curated list of where you can get involved (i.e. where testers of all skill levels and expertise are needed), learning opportunities, and some reading to keep you informed.
Looking for ways to contribute? The following tickets and patches need contributors.
Manual testing help needed
Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.
The following tickets need testers to manual test and provide feedback (test report):
#53275: Wrap options on pluginPluginA plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party page to second line Done ✅
#53801BlockBlockBlock is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-based widgets screen does action wp_footer after each widgetWidgetA WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.
#52241: Windows OS specific – infinite loopLoopThe Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. in clean_dirsize_cache() Done ✅
PHPUnit tests help needed
Who? Any QA or PHPPHPPHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.
#49985: REST APIREST APIThe REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
#52252: PHP Notice when monthnum query var is set without the year QV
Reproducing reported issue help needed:
Who? Any contributor.
The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:
#54205 – jqxhr is undefined inside of deferred.done() when using wp.media to add a custom image upload Needs reporter feedback ✅
#54211 – Small css bug when using customize-controls in customizer.php Done ✅
Reproducing the reported issue is the first step in a new defect ticket’s lifecycle. Why? In order to fix a bug, first step is confirm the bug is reproducible and is due to WordPress CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress. itself (and not a third party like a plugin or theme).
In the spirit of improving the E2E developer experience, I’d like to make a case for migrating CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress.’s browser automation library to Playwright. I was asked to write this post after opening an experimental pull request, where I migrated a selected portion of specs to Playwright, making them available for running both locally and on CI. That happened some time ago, so there’s already been quite a bit of discussion going on there! Having said that, I’m going to try making the case again here, taking the feedback I’ve received so far into consideration. I also encourage you to check out the PR to see the implementation details and Playwright advantages in action.
Why drop Puppeteer in favor of Playwright?
It’s easier to write stable tests.
There are a few reasons why. Please read on to find out which ones I think are the most relevant for the project.
The APIAPIAn API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.
Playwright’s API is almost identical to Puppeteer’s, which means that the developer transition should be close to effortless. I think that’s a big factor here, as it significantly lowers the cost of this transition also from the migrationMigrationMoving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. side. A good thing to start with! 🤞
Playwright performs a range of actionability checks on the elements before making actions to ensure these actions behave as expected. It auto-waits for all the relevant checks to pass and only then performs the requested action. (…)
I think it’s the number one reason for the stability improvement over Puppeteer. In practice, it means the following:
No need to perform any additional presence checks
Most of the DOM changes happen asynchronously, so in order to avoid flaky behavior, a test usually explicitly wait for an element before performing an action on it. A good example would be the most frequently used click action, which usually looks like this when performed with Puppeteer:
With Playwright, thanks to the auto-waiting mechanism it becomes just:
await page.click( 'button' )
No need to disable CSSCSSCSS is an acronym for cascading style sheets. This is what controls the design or look and feel of a site. animations
This is thanks to the stability check, which makes sure the element has a bounding box and is not moving before the action is performed. I think this is a major advantage because it allows to fully test the application, including the CSS animations, which are an integral part of the user interface.
In my PR, as a proof of concept, I removed the code that disables CSS animations and the forced reduced motion, which slowed down the refactored tests (21 suites) by around 37 seconds. This number will grow with every test, but judging by the current data it shouldn’t be more than a few minutes in total. I’d say the trade-off would still be worth it, but this can be discussed and decided upon later.
What do you think about testing without animations? Should they be enabled if it’s possible, even for the cost of extra wait?
Less code!
In general, all of the above comes down to writing less and simpler code to get the same or better results than Puppeteer. If you go to my PR, you’ll notice that there are more lines removed than added in the refactored tests!
With Playwright, the tests and test utilities are easier to write and follow, and the environment requires less customization (e.g. disabled animations) which actually makes it closer to what users are experiencing.
Are there any downsides to the auto-waiting mechanism?
There were some concerns about how this mechanism could affect the performance tests. Because it could potentially become a blocking factor for this migration, I decided to migrate Gutenberg’s performance specs to Playwright as a proof of concept and see what happens. So far, thankfully it looks like there isn’t much difference between Puppeteer and Playwright — the performance metrics are very close, which would be the desired outcome.
Do you think there could be any downsides to the auto-waiting mechanism? Please let me know in the comments!
The Advanced Selectors Support
This part has changed a bit due to some feedback received in the PR. Originally, I mentioned text selectors and layout-based selectors as the number one reason for making the tests and utilities easier to write and follow, as well as making them more resilient. While prioritizing user-facing attributes is still a good practice for most applications, GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ is a bit different in this regard. Apparently, CSS classes are considered to be the API there, so they change less often than the user interface. Nevertheless, as some folks noticed advanced selectors are still a big win as they’d allow to, e.g. drop the use of cumbersome XPath selectors and more with powerful selector chaining. Currently, with Puppeteer, CSS selectors can only be used.
The built-in Inspector is also a big advantage of Playwright. It’s quite intuitive and has some neat features like stepping through the script while running headfully or dynamically recording actions to a script — a really convenient way of quickly drafting the test. See the video below for a short demo of the script recorder.
Code generation with Playwright inspector in action 💥
Trace Viewer
Playwright offers a complete tracing solution. Trace can be recorded and stored in a zip file, which then can be viewed via the Trace Viewer GUI:
Viewing a recorded trace in Trace Viewer
On the image above, you can see that the trace is displayed in a form of a film strip. Each frame can contain Before, Action, and After snapshots visualizing a complete action execution. On the left-hand side is a list of all the actions Playwright performed. Each of them can be inspected in more detail in the section on the right-hand side, where you can switch between the action log, location in the source code, and the network log.
I think it’s great to have that kind of functionality out of the box. It also shows how Playwright is intended to be a complete testing solution. With Puppeteer, there aren’t really any first-party tools for debugging, as far as I’m aware – The suggested way is to either slow down the tests in headful mode with DevTools open or use the Node.js Debugger when running headlessly.
If there’s a goal to expand testing to browsers other than Chrome, it wouldn’t be an issue for Playwright as it supports all other major players: Firefox, WebKit, and Microsoft Edge. At the time of writing this, Puppeteer supports only Chrome and Chromium, and the official support of Firefox is currently experimental.
Playwright has a first-party test-runner, which is very similar to Jest test-runner (currently used for Puppeteer) but written from scratch. It contains a lot of end-to-end testing utils, tooling, concurrency, reporting, assertions, artifacts, etc., and extensive configuration support. Another quite nice thing to have without having to install and rely on third-party libraries!
I think it deserves a mention here, as it’s easy to navigate and really well written, in my opinion. Please take a few minutes and check it out for yourself at https://playwright.dev/docs/api/class-playwright – maybe you’ll find even more reasons to switch to Playwright? 😉
Writing good, stable E2E tests is often a struggle. If there’s a chance of improving that, especially with such a low cost, then it should be done. I would be happy to work on this task if there’s a consensus to move it forward.
Thanks for reading. I’m looking forward to all the feedback!
@hellofromtonya introduced agenda of the meeting and shared a post that describes where the testers are needed during the week. At the end of introduction, Team Test Update was shared.
The team was able to decrease the numbers of tickets.
Modernizing to the Latest PHPUnit
@hellofromtonya described how this changes impact CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress. testers. Anyone that’s running the PHPPHPPHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. test suite or buil automated integration tests, should read the dev note.
For running the PHP test suite with Docker, nothing changes in your workflow:
npm install
npm run build:dev
npm run env:start
npm run env:install
and for running the tests:
npm run test:php
for running a specific group:
npm run test:php -- --group groupname
When not using Docker to test, then there are extra steps. The handbook and Dev Note provide the details.
5.9 Test Roadmap
In 5.9 release, Test Team will introduce the roadmap.
Everyone agreed that it’s necessary and help with the process of analyse and future improvement. @hellofromtonya also mention that we’ll implement a longer term vision, called North Star.
@justinahinon shared his document that is a great roadmap for automate tests focal group.
Focal Groups
@hellofromtonya shared an idea to have 5 separate focal groups in Test Team, each will need folks to shepherd the group:
Shepherd your focal group – meaning be active in it, help contributors, know the status, help move work forward, stay aligned to the roadmap, and help to develop the roadmap
Each week – share an update for your group (can go in the team’s update)
When blocked or need more help, pingPingThe act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.” Team Reps or another group for help
Last words
Team agreed that we need a bit of advertisment of our group.
As a reminder, if you’d like to suggest an idea for a call for testing, it’s very welcomed and all ideas will be weighed against current project priorities to figure out what makes the most sense to pursue. You can share ideas directly in the slackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel or via DM to me (@annezazu).
Feature Overview
Because Full Site Editing is a collection of features that allows more items to be easily edited without knowing code, new blocks needed to be created to cover more parts of your site. These blocks are generally called “Theme Blocks” as they match functionality that used to only live in themes. While a number of theme blocks were introduced in WordPress 5.8, there’s always more work to be done, including shipping even more theme blocks in future releases!
This test is focused on pushing these lovely Theme Blocks to their limits to better determine what to prioritize and what features might remain to be documented. To make the experience feel a bit more fun and practical, we’re going to approach this test from the vantage point of creating patterns for blogs using some of these blocks. If you really like what you make, remember you could even register them on your site 🙂
As a refresher, here’s a rundown of all of the theme blocks ready for testing with a note around which ones are included in WordPress 5.8 in case you’re inspired to use them on your site now:
Site Logo: allows you to display and edit the site logo [shipped in 5.8].
Site Tagline: allows you to display and edit your Site Tagline [shipped in 5.8].
Site Title: allows you to display and edit your Site Title [shipped in 5.8].
Query LoopLoopThe Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop.: allows you to display posts and pages in various formats [shipped in 5.8].
Post Title: displays the Post Title [shipped in 5.8].
Post Content: displays the contents of your post [shipped in 5.8].
Post Date: displays the post date [shipped in 5.8].
Post ExcerptExcerptAn excerpt is the description of the blog post or page that will by default show on the blog archive page, in search results (SERPs), and on social media. With an SEO plugin, the excerpt may also be in that plugin’s metabox.: displays the post excerpt [shipped in 5.8].
Post Featured ImageFeatured imageA featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts.: allows you to display and edit the featured image of your post [shipped in 5.8].
Post Categories: displays the categories of a post [shipped in 5.8].
Post Tags: displays the tags for a post [shipped in 5.8].
Login/out: displays login and out links [shipped in 5.8].
Page List: displays a list of all pages on your site [shipped in 5.8].
Template Part: allows you to display and edit various global regions of your site (headerHeaderThe header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes., footer, etc).
Post Comment: displays an individual comment.
Post Comment Author: displays author for a comment.
Post Comment Content: displays content of a comment.
Post Comment Date: displays comment date.
Post Comments: displays all comments.
Post Comments Count: displays comment count.
Post Comments Form: displays comment form.
Archive Title: Displays archive title.
Term Description: Displays the description of categories, tags and custom taxonomies when viewing an archive.
Testing Environment
While there’s more information below to ensure you get everything set up properly, here are the key aspects to have in place with your testing environment:
Use a test site. Do not use a production/live site. You can follow these instructions to set up a local installLocal InstallA local install of WordPress is a way to create a staging environment by installing a LAMP or LEMP stack on your local computer. or use a tool like this to set up a development site.
Use the latest version of WordPress (downloadable here).
Use the latest version of GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ (11.6 as of writing this).
Generally speaking, please use the latest versions of each part of the setup and keep in mind that versions might have changed since this post was shared.
Testing steps
Setup Instructions:
Have a test site using the latest version of WordPress. It’s important this is not a production/live site.
Install and activate the Gutenberg pluginPluginA plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party from Plugins > Add New. If you already have it installed, make sure you are using at least Gutenberg 11.6.
Install the TT1 Blocks theme by going to Appearances > Themes > Add New. Once installed, activate the theme.
Create at least eight posts with two different categories and featured images of your choosing. Alternatively, you can download and import the demo Gutenberg content created especially for this test (open the link and select “Download”) via the WordPress importer under Tools > Import. You can also follow this lesson for how to use demo content.
Go to the website’s admin.
You should now see a navigation item titled “Site Editor (betaBetaA pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.).” If you don’t see that in your sidebarSidebarA sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme., you aren’t correctly using the Site Editing experiment.
General Instructions:
Head to Pages > Add New and create a new page. Title it whatever you’d like!
Add the Query Loop blockBlockBlock is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. and select whatever pattern you want to build upon. You can also add in a container block, like a Columns or Group block, and add in the Query Loop as you’d like.
From there, make the pattern your own using as many Theme blocks listed above as you can and customizing the various settings. For example, you could create a comment heavy pattern utilizing the various comment blocks or have a particularly image focused one thanks to new improvements to the Featured Image block. Try to be as unique as possible and don’t be constrained by adding the blocks only within the Query Loop.
If you’re up for the challenge and want to take this test further, try to create your own pattern from scratch, make multiple patterns, or recreate some with your own twist from Theme designers at Automattic shown below:
What to notice:
Remember to share a screenshot of what you created if you’re up for it!
Did the experience crash at any point?
Did the saving experience work properly?
Did you find any features missing while creating your custom blog pattern?
What did you find particularly confusing or frustrating about the experience?
What did you especially enjoy or appreciate about the experience?
What would have made this experience easier?
Did you find that what you created in the editor matched what you saw on your site?
Did it work using Keyboard only?
Did it work using a screen reader?
Leave Feedback by October 13th, 2021
Please leave feedback in the comments of this post. If you’d prefer, you’re always welcome to create issues in this GitHub repo directly for Gutenberg. If you leave feedback in GitHubGitHubGitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/, please do still comment below with the link. If you see that someone else has already reported a problem, please still note your experience with it below, as it’ll help give those working on this experience more well-rounded insight into what to improve.
You must be logged in to post a comment.