The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
If you’d like to help test Full Site Editing, please join the FSE Outreach Program. You can find current calls for testing for this program here and you can join the fun in #fse-outreach-experiment.
Last week, the Test Team agreed to do bi-weekly test triage sessions. These meetings will happen every other Tuesday at 13:00 UTC (starting this week).
Next test triage meeting will be held on 2021-05-04 13:00 in the #core-test on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/.. Come join to triage with the team.
The meetings are now in the calendar.
In context of bug repositories such as TracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. or GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/, triage means sorting, labeling, and closing duplicate incoming issues and testing means manually testing bug reports, questions, and feature requests.
Triage & Testing Issues in Test Team’s handbook
The goals are:
There are multiple “testable” states in a ticket’s lifecycle including these (keyword shown in parentheses):
has-testing-info)has-patch)needs-testing)has-testing-info)has-patch)needs-unit-tests)has-testing-info)Props to @francina for peer review.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. Thank you to everyone who joined in! If you’re keen to join an effort like this in the future, please join the slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel.
Attendance: @mkaz @annezazu @sabrinazeidan
Video Recording:
Topics Covered:
Next Steps:
@annezazu found a small bug with the navigation block where changing global styles typography changes the typography of the placeholder text and will report it!
With the Go/No Go Next Steps outlined ahead of WordPress 5.8’s release in July 2021, let’s use this time to dig into any general questions you all might have around Full Site Editing! If possible, please focus questions specifically around WordPress 5.8 as those will be the most high impact to address. You are welcome to submit questions using the form below or to leave them as a comment on this post by May 12th:
Keep in mind that because, depending on the questions it’s likely that some answers might take the form of “people are working to figure this out and feedback is welcome here,” rather than a definitive answer. This is especially true for features/milestones that are planned for the 5.9 release.
Where will you share the answers?
I’ll share a recap post on this blog (Make Test). Questions will be grouped with corresponding answers for easy review. You can see what the outcome will look like based on the first round here. I will track down answers to every question and share my work as I go by creating a collaborative Google doc where people can help find answers or simply see how the work evolves. I very much welcome collaboration here!
While the main result will be a lovely list of answers, this collective effort will also be useful for future documentation updates and potential tutorials. Once the post is published, I will follow up via email with everyone who left their email and a question in the form. For anyone who leaves a question as a comment on this post, I will @ your username in the recap post so you don’t miss out too!
For more information about this experimental program, please review this FAQ for helpful details. To properly join the fun, please head to #fse-outreach-experiment in Make Slack for future testing announcements, helpful posts, and more will be shared there. To help with planning your involvement, you can see the upcoming/current schedule for the FSE Outreach Program here.
#fse-outreach-program #full-site-editing #gutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ #coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.-editor #fse-testing-call
In light of the 5.8 planning and next steps, I wanted to share the upcoming schedule for the FSE Outreach Program in order to ideally help people participate more in what’s to come. As changes occur and I’m able to create plans further out for 5.8, I’ll edit this post and note exactly what I changed at the very end of this post. I do expect changes so please see this all as what I anticipate vs set in stone plans.
As always, if you have any questions, feel free to ask here or to DM me in slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. (@annezazu).
Based on what’s planned for 5.8, the following is lined up for testing:
When I can, I’ll try to leave three weeks for testing rather than two but, due to the release timeline, this may not always be possible.
| Date | GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ | Questions Round 2 | CfT #5 | CfT #6 | CfT #7 |
|---|---|---|---|---|---|
| April 28 | 10.5 | Start | |||
| May 5 | 10.6 RCRelease Candidate A beta version of software with the potential to be a final product, which is ready to release unless significant bugs emerge. | End | |||
| May 12 | 10.6 | End | Start | ||
| May 19 | 10.7 RC | ||||
| May 26 | 10.7 | End | Start | ||
| June 2 | 10.8 RC | ||||
| June 9 | 10.8 | End |
Outside of these calls for testing and questions, there will continue to be live streams, hallway hangouts, important posts flagged, etc. This post will be updated to include the relevant links as time progresses:
This program has many pieces to it and there are tons of ways to get involved. At a high level, here are ways to help:
For any of the items that involve working with me directly, please comment on this post or message me directly (@annezazu) so I know you’re interested and we can talk through what helping out might mean. To those of you already doing this work, thank you so much! It all truly helps.
April 29th 2021: updated to include links to the fifth call for testing and the second round of questions.
The fifth call for testing is already underway, so join #fse-outreach-experiment in slack and/or subscribe to this Make blog and stay tuned for more.
This post is a summary of the fourth call for testing for the experimental FSE outreach program. Thank you to everyone who participated, whether through testing directly or sharing the call for testing with others. It all helps! Special thanks to the following people:
How far can one go?
It’s always fun to see how far people can take these tests in creating something cool without code. Here are a few screenshots of people’s creations that make me hungry just looking at them:
High-Level Feedback
Here’s what a few folks had to say about the overall experience that’s important to keep in mind as you read the rest of this post:
All of this could be because of my inexperience with GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party. I’m used to working with Astra and other blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. libraries rather than the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. blocks.
@suhayse in this comment.
The most problematic issue is that what I saw in the editor was not what I got on the front end. I have played around with it enough to know in my mind what it might look like on the front end to make adjustments without previewing the changes. However, that is not the user experience that WordPress is shooting for.
@greenshady in this WP Tavern post.
Most of us were confused by the current UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. and UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. of the full site editing experience. For some of our colleagues, this was the first time using the block editor for a whole day.
@francesca in this Yoast blog post.
Repeated Feedback: Improving saving, desire for a preview option, and differences in spacing
As with last time, to better consolidate repeated pieces of feedback, this section only contains new bugs or enhancement requests while still sharing quotes that highlight how these areas continue to be a pain pointPain point Pain points are “places where you know from research or analytics that users are currently getting hung up and have to ask questions, or are likely to abandon the site or app.” — Design for Real Life. In this case, keep in mind that spacing refers to everything from differences between the front end and back end to enhancement requests around setting the width of various blocks. In general, though, it further underscores how the differences in experience between the editor and front end break the promise of WYSIWYGWhat You See Is What You Get What You See Is What You Get. Most commonly used in relation to editors, where changes made in edit mode reflect exactly as they will translate to the published page. currently. Thankfully, lots of work is underway to continue iterating on this aspect of the experience!
One frustration point was the ability to preview as others have mentioned (the live site definitely looked different from the dashboard preview). When I did view the live site, there wasn’t any margin or padding on the main headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. section but there was on the added column set on the top, even though both were set to full width. I tried changing that main header column width back to wide, saving, then going back to full width but it didn’t help.
@suhayse in this comment.
Another thing I noticed was that some small changes, like adjusting the percentage width of the individual columns didn’t activate the “update design” button.
@suhayse in this comment.
I click to Update Design. As there is not yet any simple way to preview on the frontend like we do in the Post/Page screen. I copy the site address url and open a new browser tab and paste it into the new tab to see the frontend. The frontend does not show any margin along the left edge.
@paaljoachim in this comment.
I noticed along the way that the Update Design button was greyed out on occasion when I made some adjustments inside the Cover block and other inner blocks. I had to click into various blocks to get the Design button active again so that I could save. (This seemed a bit hard to track.)
@paaljoachim in this comment.
The site editor makes it looks like there is a small margin all around the full-width header. On the site itself, this isn’t seen. I had set a background color for the full-width header which is edge to edge on the site, but has a margin all around it in the editor.
@kristengunther in this comment.
Columns Block Improvements
Because this call for testing required people to make great use of the Columns Block, it was also the focus of a lot of feedback from various participants. Overall, this feedback mainly came down to two interrelated areas: difficulty navigating between nested blocks and confusion around properly setting width. What follows are the new issues created as a result of this call for testing:
Testing was smooth overall except when it came to setting the Columns Block to full-width (both in the header and body of the page in the Site Editor). I was unable to set the block to full-width within the Block Toolbar settings. I was able to do this outside of the Site Editor on a fresh page though.
@synorae in this comment.
I see no visual difference from selecting Wide width or Full width in the backend.
@paaljoachim in this comment.
It’s not clear that the symbol/icon is Full width. It would make sense to have arrows to indicate that it should be full width.
Anonymous Yoast employee in this GitHub issue.
I don’t see an option for full width? Ah it’s under alignment. “Alignment” sounds like left/center/right, not the size. What’s the difference between wide and full wide? I don’t see much difference in the preview.
Anonymous Yoast employee in this GitHub issue.
Setting Styles
As part of this test, people explored setting various styles to customize their heading to their liking and bring to life the feeling of a restaurant. Similar to the complexity in navigating between nested Column Blocks, though, setting styles proved to be pretty confusing considering how unintuitive it was to figure out how to properly select and then style the section one wanted to. Tied to this, it wasn’t always clear where one could find the setting that would do what they wanted since various settings are spread across the block toolbar and block settings. In some cases, the setting to accomplish something doesn’t exist yet too! As more work is underway to add in more styling options and normalize block level controls in a more intuitive way, this is an area ripe for continued iteration.
This is global settings vs individual page template settings. It’s pretty confusing right now. I don’t know exactly where I would set universal global header colors. I would expect to be able to do that in the Template Parts/Header but I don’t immediately see how to do that.
@suhayse in this comment.
I found I had to set the background color for my header 3 times, once for the index template (like in your video), once on the page home template, and once on the page template.
@kristengunther in this comment.
Discoverability of settings, not ideal. Some things are in the popup toolbars, others in the sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. (both in Site Editor and Appearance), other in the top toolbar. You had to find the cog and the Global Style icons to open more settings.
@francina in this comment.
When you are new to this, you are really wondering if you need to find the settings in the list overview left sidebar, or the cogwheel right sidebar or on the block itself, all the options are all over the place.
Anonymous Yoast employee in this GitHub issue.
Why does the column change size when changing the color in the settings? At least it definitely seemed like it happened that way. That’s unnecessary and unexpected.
Anonymous Yoast employee in this GitHub issue.
General enhancements & feature requests
As with every call for testing, it’s not just for finding bugs! It’s also important to hear about features that people reach for and find are missing. This section is a “catch-all” to cover all additional features that were reported that didn’t nicely correspond with a particular block or categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging.. This only includes new feedback and doesn’t include previous findings from prior tests:
Flow is the actual user experience, in many ways. If you like, you can think of flow as a really comprehensive set of user stories. When you think about user flow, you’re thinking about exactly how a user will perform the tasks allowed by your product.Flow and Context.
Like the first design I was shooting for, I wanted my Navigation items to look like individual buttons, each with a bit of whitespace in between. However, the Navigation block does not currently support adding backgrounds to each nav item. Even if it did, it also does not have a horizontal margin setting to add the spacing.
@greenshady in this WP Tavern post.
Collection of Miscellaneous Bugs
Because this was a more open call for testing, not all bugs fit nicely into a category or theme with many of them being standalone problems. To make it easier for those working on full site editing to get a sense of bugs at a glance, they have all been shared here:
On April 5th, I shared a 3 question survey to help improve the FSE Outreach Program by pausing to ask for feedback so that I can better understand current levels of participation, what you all want to see more of, and what can help folks contribute more. Thank you to the 16 people who took the time to share their feedback! If you are reading this and didn’t answer the survey, feel free to respond here with additional thoughts as I’m always looking to improve how this program works.
Full Results
If you want to read the full reports, I have included two options below. Keep in mind that I intentionally removed the final question that listed people who optionally included their WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ usernames for follow up.
What top 3 items would be most helpful to have covered in streams or zoom calls?
The top 3 items were as follows:
There were very clear winners here!
What would help you be more active in the FSE Outreach Program?
The “other option” (3 responses) mainly touched on predictability of testing calls and wanting more time/reminders/expertise. For now, I’m hesitant to set a strict timeline for the calls for testing because the testing needs are variable and partially depend on what’s ready in the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party at various release points. In time, I do expect calls for testing to become more predictable as the experience stabilizes and there are less brand new things to test.
What best describes your current participation?
The “other option” (4 responses) are as follows:
I tested many things, discovered the same issues discovered. Didn’t have enough time to go deeper with my tests
New to FSE and coming up to speed so my time can be spent testing and not setting up to test.
I don’t quite follow the script when participating. I don’t just test with TT1 blocks.
I’d like to participate as much as possible as I see FSE as a huge part of WordPress future, and yet another opportunity to give back to the community.
While I was aiming to see more people in the categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging. of finding it easy to participate, I’m hoping that some upcoming changes can shift this, particularly in extending the current call for testing to a three week window rather than a two week one. On a whole though, it seems like people are keen to help out here but, as seen across the WordPress project, we’re all dealing with limited time/capacity.
Next Steps/Changes
In light of this feedback, I plan to do the following:
If there are any changes/next steps you’d like me to explore, comment below and I’ll see how it fits into what I can manage. On the flip side, if you’d be interested in helping with various aspects of improving the program, let me know! I’d love to create a little team of people working to expand the capacity and reach of this outreach effort.
