The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
If you’d like to help test Full Site Editing, please join the FSE Outreach Program. You can find current calls for testing for this program here and you can join the fun in #fse-outreach-experiment.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. Thank you to everyone who joined in!
Attendance: @bobbingwide @poena @jcasabona @richtabor @desrosj @optimizewordpress
Video Recording:
Topics Covered:
If you’re a theme author who has worked with theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. in some capacity, please fill out this theme survey.
With WordPress 5.8 came the official introduction of theme.json to the project allowing you to configure both existing editor settings like enabling custom colors and adopt new ones as they are released. Since this new mechanism is an early step towards a comprehensive style system for the future of WordPress, it’s important to hear from everyone who is currently using theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. to learn more about how folks are using this tool and what might make sense to include in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. going forward.
If you’re a theme author who has used theme.json or is interested in doing so, please fill out this survey by August 13th to help shape its future.
If you haven’t yet used theme.json, this survey won’t be applicable to you. Thanks in advance for taking the time to share your thoughts! The results will be shared in a recap post.
If you’re curious about theme.json and haven’t had a chance to explore just yet, check out this previous call for testing offering different difficulty levels of flows and the latest WordPress News post on Configuring Theme Design with theme.json.
The meeting took place here in Slack. @hellofromtonya facilitated.
Discussion starts here in Slack.
A rep is an active team member who is reliable and trusted, advocates for and is knowledgeable of one or more areas of testing, and wants to represent, nurture, and grow the team to serve the WordPress project.
Action: @hellofromtonya to publish Call for Nominations once peer review is complete. ✅ Done.
Update: Here’s the post: Test Team Reps: Call for Nominations.
Discussion starts here in Slack.
Broadly testing entire release is hard to capture the feedback. Right? But testing something that has a ticket or Call for Test (CfT) is trackable for not only testers but the Core folks who are working on that thing under test. Sharing how you tested (the test environment, theme, plugins, etc.), steps you did, results, and maybe even a screenshare …. this type of info is valuable.
Commenting: “It works” is not as valuable as a feedback.
Action: @hellofromtonya will start a proposal for discussion:
laying out some guidelines for how testing is captured, what testing is not, and then that will guide everyone to know what a testing contribution is and isn’t.
Discussion starts here in Slack.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. It ended up just being me and @bobbingwide so I decided to not record and instead have a free flowing chat with Herb that was less structured and more in line with our interests. We chatted for just over an hour!
We mainly chatted about theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML., what’s coming to 5.8, a few individual blocks (Query LoopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop., Featured ImageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts., etc), responsive controls, and various explorations Herb has done including building a few blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. plugins. Of those, we went through the following links that I recommend spending time reviewing too:
Next Steps
I followed up on two issues that we found while chatting: The reset to default option being too close to X for global styles UI and a bug with an empty template name and trying to rename it.
The meeting took place here in Slack. @hellofromtonya facilitated.
This was the team’s first team chat in a year or more.
@lucatume asked “what’s the purpose of the group?” Great question!
A healthy discussion happened around the history of the team and what role we serve in the project. The discussion starts here in the slack thread.
Some highlights are captured here:
Action Item: Tonya to publish a vision of the team’s big picture of how the team can serve the project.
@lucatume asked why the team went dormant. The slack thread starts here.
Some of the reasons given include:
@lucatume raised a discussion about how to autogenerate e2e tests from written step-by-step processes or recording the actual testing process. The slack thread starts here.
The team agreed this is a great idea to help anyone contribute to testing.
Action item: @lucatume will work with @justinahinon and others to identify an auto-generator tool.
This post is a summary of the seventh call for testing for the experimental FSE outreach program. Thank you to everyone who participated, whether through testing directly or sharing the call for testing with others.
On a more personal note, it’s so neat to see the various ways people engaged and to really feel the power of the WordPress community in these calls for testing — WordCamp Japan used the seventh call for group testing this week, a meetup in Philadelphia used it as part of their event (shoutout to @accessamy and @itsjusteileen), the call for testing was translated into Italian and Japanese (shout out to @piermario and the folks from WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. Japan) and three folks did write ups encouraging others to test alongside their feedback (@greenshady, @bgturner, and @bobbingwide). Plus, I had some of my coworkers go through the test for good measure! I am super stoked to see a diverse set of ways folks are exploring this program and deeply appreciate you all making it happen.
Teamwork makes the dream work. Anything I can do to make participation easier and more fun, let me know!
How far can one go?
It’s hard to compete with @greenshady’s awesome explorations at this point! Check it out below:
High Level Feedback
Here’s what a few folks had to say about the overall experience that can help frame the following detail oriented feedback. Since this was a more open ended test compared to the prior one, it was interesting to hear about the ways in which people explored things on their own and the resulting joys/frustrations that caused.
Compared to the earlier tests, the overall experience is way more stable and polished. My biggest issue with the GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ and FSE is still the same: lack of visual references while designing, unless I do some hovering dance on the blocks and – this time – I didn’t have a clear picture of how exactly changing some elements (site title, navigation) on the portfolio template would affect other pages, so I got a little lost between pages.
@piermario in this comment.
Generally I love the query blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. though. Really cool that you can do that now without coding! I am looking forward to using it in projects.
@michaelsndr in this comment.
I had a lot of fun with this. And frustration. Some more fun. And…you guessed it…some more frustration….I enjoyed the process — yes, I revel in both the fun and frustration. Aside from everything that I think is broken, the overall system is pretty dang sweet. There are far more things that the development team has nailed down than there are that feel janky.
@greenshady in this post.
Repeated Feedback: Switching between editing modes (template vs page/post) & various block improvements
This section is dedicated to repeated items from previous calls for testing. Once more, despite the three ways to visually distinguish the editing modes, there remains confusion around when one is in each mode. The deeper into these calls for testing we go, the more it becomes clear how valuable it will be to do things like view a template while editing content and have some good friction in place while interacting with post blocks in template editing mode.
Across a few blocks, some repeat items came up that are worth mentioning considering they were each mentioned at least two times:
Today I got lost quite often. I didn’t always know if I was editing the Portfolio Template or the Portfolio page.
@piermario in this comment.
However, once I was in the Template Editor it wasn’t clear when I was editing the template or the content itself. When I used the block navigator – I could see the post content block (which made sense) but only because I was already looking.
Automattic employee feedback.
Some general usability feedback of the column block: I’d love a way to make the vertical margins disappear so that full-width sections that have background colors don’t show any space between them.
@bjturner in this post.
Post Title Block – no way to style text (bold, italics etc), and no way to have a totally custom colour. Do these color options come from the theme itself?
Automattic employee feedback.
As a user, template editing is a great tool when you have a good visual understanding of what your post or page content will look like in the context of the full site. The issue is, when in the post editor I don’t know that, unless I am checking “Preview” as I create/edit my content. Has any thought been given to how we could improve this experience so users are more aware, as they’re editing, of how their content will be displayed on the site (depending on the template used)?
Automattic employee feedback.
Query LoopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. Block & Related Improvements
Since the Query Loop Block featured heavily in this call for testing, it’s no surprise it was also an area of both great praise and criticism. On the whole, there was loads of excitement around this powerful block with folks keen to have access to it with WordPress 5.8. Outside of that though, the following items were raised for the Query Block itself and some of the related blocks used within it:
Tied to the above issues, there was repeated frustration around deeper customization and limits of the nested blocks within the Query Loop, especially if someone wanted to go well beyond what the current patterns offer. It’s also important to note that this test was done without this PR merged for the Query Loop block, which makes the Post Blocks uneditable within the Query Loop block itself ahead of WordPress 5.8.
The next section of template testing consisted of adding a Query pattern and customizing it. I have a love/hate relationship with queries in Gutenberg right now. The Query block itself works well. It has a solid balance between advanced usage and simplicity for the most part. I am amazed at what the development team has done over months upon months of iteration.The downfall is that the Query block is merely a wrapper. It is only as good as its weakest sub-block.
@greenshady in this post.
There’s a bit of a confusion point in the Query Block with Items per Page. Despite having multiple published posts only one appeared by default. I found the controls in the Block Toolbar to increase, but also found it a bit cumbersome to toggle between the Block Toolbar and Block SidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. to refine the underlying query.
@dryanpress in this comment.
Query Block: Block outputs nothing on the frontend when there are no posts to show. Generally, it requires to show something that tells visitors that there are no posts or some custom message.
@sagarnasit in this comment
General Usability Enhancements
As people explored template editing mode, the following items came up as areas that would make the experience more intuitive going forward. Many of these were repeat items but it felt important to call these out separately, in particular the quotes describing the current experience. While some of these areas have design explorations in place for potential inclusion in the future, this section captures the current pain points:
Flow is the actual user experience, in many ways. If you like, you can think of flow as a really comprehensive set of user stories. When you think about user flow, you’re thinking about exactly how a user will perform the tasks allowed by your product.Flow and Context.
It appears that I didn’t save the template since it’s showing a 404, even though the title says “portfolio.” I think what confused me was the “Publish” button in the upper right corner. Coming from a WP background I think I understand that “Publish” meant to publish the page template I was editing, but on initial use, I was hesitant to push the button because my context was the original page that I had created, not the page template I was editing.
@bjturner in this post.
While in Template Editing Mode, I clicked the Preview button, clicked Preview in new tab and didn’t see the addition of the navigation block or other template changes. If this could work that’d be great, but if these won’t be available to preview outside Gutenberg due to how Templates are saved and stored, that preview dropdown item probably shouldn’t be available inside Template Editing Mode.
@dryanpress in this comment.
If I create a new template, the new template is not available in the drop-down selector until I refresh.
Automattic employee feedback.
When saving the template change, if I uncheck all the items that appear, the Save button gets defunct. If we are allowed to uncheck one of those, I think we should be allowed to uncheck all items too.
Automattic employee feedback.
The Update option isn’t available once I’ve switched alignments on the block. I needed to alter the post title to trigger the Update option.
Automattic employee feedback.
When you’re creating a new template, for each existing template part that you insert, you have to remember to set the same attributes for the template part as used in other templates. Attributes that will need setting include the Width and Colours.
@bobbingwide in this comment.
Similar to my last post, I wanted to share the upcoming schedule for the FSE Outreach Program for June and July in order to ideally help people participate more in what’s to come and to know what to expect. As always, if you have any questions, feel free to ask here or to DM me in slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. (@annezazu).
Outside of the work above, there will continue to be live streams, hallway hangouts, important posts flagged, etc.
| Date | CfT #7 | Theme.json CfT | 5.8 RC CfT |
|---|---|---|---|
| June 9 | |||
| June 16 | End | ||
| June 23 | Start | ||
| June 30 | Start | ||
| July 7 | |||
| July 14 | End | ||
| July 21 | End |
This program has many pieces to it and there are tons of ways to get involved. At a high level, here are ways to help:
For any of the items that involve working with me directly, please comment on this post or message me directly (@annezazu) so I know you’re interested and we can talk through what helping out might mean. To those of you already doing this work, thank you so much! It all truly helps.
#fse-outreach-program, #full-site-editing
This post is a summary of the sixth call for testing for the experimental FSE outreach program. Thank you to everyone who participated, whether through testing directly or sharing the call for testing with others. It all helps! Special thanks to @piermario for translating the call for testing into Italian once more. Translations are such a big help as they really help bring the wider community along.
As a reminder, the seventh call for testing is currently underway and you’re welcome to join!
How far can one go?
Once more, I’m excited to share @greenshady’s creative take on this call for testing that utilizes some awesome custom colors and reuses a logo from the WC Birmingham team:
High Level Feedback
Here’s what a few folks had to say about the overall experience that’s helpful to keep in mind as it’s easy to get into the themes or details without seeing the big picture of how using template editing mode felt. For most, they felt it was smoother than expected but that the outcome still was lacking in terms of creating a truly refined template.
It was fun! I liked testing the new features and feeling like I was contributing to WP. I didn’t realize you could drag and drop blocks! What a cool idea to place the Page Title in the Cover blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.! Always learning new things.
@askdesign in this comment.
In the very broadest sense, yes. Someone in an earlier call-for-testing made a simple but eloquent comment where they described the overall output of the editor as a “website-shaped object”. I’m not sure what this means in terms of specific missing features, but, without an awful amount of work on the part of both theme developer and content editor, this still best sums up the resulting output for me.
@chthnc in this comment.
The problem is that custom templates are tied to the theme. I see the logic in this. Certain aspects could be specific to the active theme (colors, fonts, etc.), and it is always how custom templates have worked. However, the block template system is different. From a user viewpoint, I feel like my custom-created templates belong to me rather than the theme. I can see a user switching themes after a couple of years and building a dozen or so templates having a poor experience in this situation. If the feature remains the same, there should be more clarity.
@greenshady in this post.
Repeated Feedback: Settings Improvement and switching between editing modes (template vs page/post)
This section is dedicated to repeated items from previous calls for testing and solely focuses on new items that have come up in these same groupings. As has been noted across various tests, the placement of settings is not always intuitive with some options feeling hidden in the Block Settings under “Advanced” sections and others named/placed in a way that makes it hard to know what the option controls.
While lots of work has been done to improve the experience of switching between modes, this was still mentioned a few times as an experience that remains a bit confusing. As noted here, there are three ways that visually help indicate when you’ve switched into template editing mode:
At this point though, work remains to make the experience more seamless, including some updates to the Welcome Guide and some explorations around being able to view a template while editing content.
The “settings cog” is not actually a settings cog, it’s the show/hide for the main editor details side bar. The main Side-bar holds the very important Block and Page controls, including “publish”. These controls are not really the “cog type” broad ranging “settings” we might see in the main Settings section of WP-admin we see true settings. The sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. contents are more like “Edit controls” and we might more usually expect a pen icon, or a sidebar show./hide icon for this sidebar show/hide. The actual “settings” for the GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ editor are under 3 vertical dots. Titled (invisibly) “Options”. This terminology and labelling or lack of it is confusing.
@steve-pheriche in this comment.
When I’m in “Template Editing Mode” there’s this large black space around the canvas area of the editor. I think this is supposed to help me understand that I’m not editing a single document, but rather that I’m editing something more.
@shaunandrews in this post.
After taking care of the Advanced section, it took me several minutes to find the “Layout” area. I think it would be better to begin with the Layout and work our way down to the Advanced settings. With something as important as Title and Area, why don’t we move those 2 items up to the top of the Block section?
@askdesign in this comment.
It took me a while to find out where the Full Width settings were. I forgot it was under “Change alignment”, next to the block icon, but I didn’t even open it as I thought it had to do with text alignment (Left, Center, Right). This seems weird, is that the best button label we can have?
@piermario in this comment.
Crash Reports
There were multiple crashing reports including one due to an issue with Duotone, which has already been reported and fixed. For the other crashing reports, they were difficult to replicate and issues were not created but it’s worth noting that four people reported various crashes, the most of any call for testing.
Not until the very end, after I was finished. I went back to edit the template and got this error message a couple of times: “The editor has encountered an unexpected error.”
@askdesign in this comment.
Trying again I was not able to trigger the crash. So that suggests there needs to be some very specific order of events, or item selected. My main browser which I experienced the crash on is the FF Developer Browser, 89.0b13.
@steve-pheriche in this comment.
Initial Template Setup Improvements
Because this test involved setting up a brand new template rather than editing an existing one, there were a few items of feedback around the experience, including around just how “blank” the blank template is. The intention is for it to be fairly empty with basic blocks in place so a user can both get a sense of what can be done there and can easily make it their own without having to delete too many items. In time, this is where Patterns should be very advantageous to quickly build up desired content in a template.
When I first created a new custom template I was surprised by its content. I’d become used to the Site editor copying the index template.
@bobbingwide in this comment.
Usability feedback
Outside of the new items listed below, it’s important to note that having consistent dimension controls was mentioned repeatedly for a variety of blocks including the Columns Block and Template Part Block. The work to bring these controls is thankfully underway!
If you want to add a paragraph block to the Cover and that block includes a longer text, there seems to be no way to make that block of text narrow.
@agabu in this comment.
Confusing: adding the pages to Navigation. I wouldn’t have known I needed to use the Page Link block if you hadn’t instructed me to do so.
@agabu in this comment.
I think it would be good to add padding and margin options to the columns block.
@askdesign in this comment.
My initial impulse when attempting to rename the template part was to click on the block heading text “Untitled Template Part”. Instead of making the title editable, I was offered the option to transform the block.
@chthnc in this comment.
One part that was somewhat unclear is the alignment options aren’t reflected in the editor vs how they display on the site. Specifically for the custom footer section, when selecting full width I see it is applied when I view on the front-end, but in editor it was not full-width. I figured out that I needed to set the footer to full width as well as the columns within to full width to see it reflected in editor.
@circlecube in this comment.
Collection of Miscellaneous Bugs & Enhancements
As with every call for testing, it’s not just for finding bugs! It’s also important to hear about features that people reach for and find are missing. This section is a “catch-all” to cover all additional features and bugs that were reported that didn’t nicely correspond with a particular block or categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging..
When viewing the page on the front-end, it would be great to have the ability to open its template directly from the top admin bar.
@agabu in this comment.
While performing this test I thought, “Why can’t I pick the template upon which my new template should be modelled?” It makes sense to be able to use a template as a template. This would be a nice feature.
@bobbingwide in this comment.
Choosing the pages in the nav was strange. The search is essential of course but it feels it could have a scroll to show all the pages (lazyload if many?)
@ridesirat in this comment.
Teamwork makes the dream work, and in reflecting on the one year of the experimental FSE Outreach Program, it seems important to recognize those who have helped make this a success thus far. To that end, I wanted to open up a discussion around how we can incorporate profile badges into the FSE Outreach Program to recognize folks who have been deeply involved in responding to program efforts. I’m not sure how this is usually done with the test team, but I’d love to acknowledge the wonderful folks who have helped thus far!
Proposed qualifications
Currently, my thinking is that contributor badges could be given in two cases:
Share your questions/feedback by June 4th
At this point, these are the questions on my mind to discuss:
Based on what I see on other teams that do contributor badges, badges are associated closely with the mission of the team, so I am purposefully not including amplification efforts here. Tied to this, I’m not including wrangling group testing as I still see that as being consolidated into giving feedback on 5+ calls for testing. I don’t feel strongly about either aspect and would love to hear what others think!
Next steps
If this proceeds, I’ll do the following:
In the long run, I’d also love for this to set the groundwork for more people to get involved in the actual running of the program: writing calls for testing, triaging feedback, writing summary posts, etc. This might mean in the future that badges are given after you’ve helped with one round of testing.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. Thank you to everyone who joined in!
Attendance: @mkaz @annezazu @bobbingwide @bph
Video Recording:
Topics Covered:
Next Steps
Anne is going to follow up on a few issues found and a discussion point to include as a comment in an issue:
You must be logged in to post a comment.