Test Team meetings schedule – poll

Test Team is meeting twice a week:

  • Tuesdays 13:00 UTC bi-weekly for Triage Sessions
  • Tuesdays 13:00 UTC bi-weekly for Team meetings
  • Fridays 13:15 UTC for Test Scrubs

This schedule was discussed while ago, when the team wasn’t crystalized as it’s right now. This is why it’s a good time to ask members of this Team whether the time of meetings is okay, or shall we reschedule it.

Please post in the comments your opinion about that, we’ll be collecting votes until 30th of August and then make a decision.

Thank you for reading!

#core-meeting

Test Team Chat Summary: 3 August 2021

The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., here.

Announcement: Test Team Reps nominations

Announcement starts here.

@hellofromtonya mentioned that two testers were nominated for this position to work together, both of them accepted the nomination, @hellofromtonya @boniu91

Team had no objections to those nominations

@hellofromtonya published and shared previously prepared introductions of newly nominated Test Team Reps

Discussion: What is needed for Testers to make their contribution easier and better

The discussion starts here.

@francina shared her list, people present on the meeting agreed:

  • Clear testing instructions in tickets
  • Clear, unified, synched instructions to setup different testing environments
  • Streaming to go through tests together and learn by doing
  • A roadmap and action plan to automated testing

@lucatume added, that’s not clear for him:

  • Where and how he could contribute in a best way. He’s willing to help with what he’s the best in: coding.
  • It’s also not clear where we should collect discussions about the tickets and other important things.

@hellofromtonya replied to the second point, saying that:

  • If the discussion is related to ticket, we should collect them in ticket. If the discussion happens on Slack, it’s a good practice to link it inside the ticket.
  • Slack is great for adding collaboration, seeking guidance, Team chats, etc.

@lucatume expanded the previous (first) point and for the most effective contribution, the following things should be clear:

  • A clear indication of what tickets will need test code, not manual testing.
  • A clear path to how test code can be contributed. If tools are required, what tools are required.
  • A definition of “good” and “bad” friction.

@hellofromtonya answered, that tickets with needs-unit-tests keyword are the ones for the PHPUnit side of things. We don’t have anything for the e2e tests though.

@lucatume suggested adding needs-e2e-tests keyword and the Team agreed that’d be useful.

@francina mentioned, that the current test setup is Jest + Puppeteer. We need developers to set up a tool for the Team that will make creating e2e tests easy.

Team agreed that it’s not possible to have 100% of automated tests, with no human review.

@francina shared what the Team needs right now to kick off the e2e testing:

  • Skilled QA engineers to setup up the infrastructure
  • Documentation to teach people to write testing specs
  • People to write the tests
  • Automated test engineers to review and maintain

@hellofromtonya said, that most likely, the bottleneck will be the group of automation test engineers to do the review, tuning, and maintenance work.

@lucatume explained how the “autogenerating” of e2e tests looks like (here)

@hellofromtonya confirmed the workflow:

  • Human does the manual test
  • Tooling records those steps
  • Tooling converts the steps into code

Mai mentioned, that generated code can be worse in terms of quality than the one written by a human. Team agreed that it’ll need to be refined and maintained. @hellofromtonya shared improved workflow:

  • Human tester does the manual testing steps
  • Tooling records those steps
  • Tooling converts the steps into code
  • Test code is attached to the ticket
  • Code is reviewed by a human is skilled in e2e tests and the thing under test
  • Once approved, a coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. committer commits the test code into the project
  • Skilled e2e test humans maintain the tests (including tuning and refinement)

Team agreed to start a pilot initiative:

  • Build a prototype
  • Start small with a handful of impactful e2e tests
  • Get those tests stable
  • Learn
  • Iterate

@lucatume will start creating the prototype
@francina offered reviewing the test handbooks

#test-team

Test Team Chat Agenda for August 3, 2021

Here is the agenda for the upcoming Test Team Chat scheduled for 2021-08-03 13:00.

This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Agenda

  • Update on Test Team Reps
  • What do testers (like you) need?
  • Focal group updates
  • Open floor

Reading and Watching

A list of last week’s posts and live streams.

Posts:

Live streams:

Open Floor

Do you have something to propose for the agenda? Please leave a comment below.

Can’t make the meeting? Share anything relevant for the discussion in a comment below.

Props: @boniu91 for peer review.

#agenda

Hallway Hangout: Discussion on Full Site Editing Issues/PRs/Designs (30 July)

This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. Thank you to everyone who joined in!

Attendance: @bobbingwide @poena @jcasabona @richtabor @desrosj @optimizewordpress

Video Recording:

Topics Covered:

  • Happy birthday to @bobbingwide! Thanks for hanging with us for part of your special day.
  • Before recording, we talked a bit about various courses people are working on/have worked on related to FSE and the difficulty in keeping things up to date while so much is changing.
  • We started the call chatting about a TT1 blocks and global styles bug that Joe found. He’ll pass along the details. With the launch of 5.8, it’s been harder it seems to replicate problems and their sources.
  • We talked about various theme related topics including how to handle theme switches, different ways to set color options (particularly for patterns), a creative solution for having responsive font sizes (and later a PR exploring next steps to make this happen), and what else still requires custom CSSCSS CSS is an acronym for cascading style sheets. This is what controls the design or look and feel of a site..
  • Rich shared that he just converted his site’s theme to a blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. theme and we briefly went over his recent post on block templates. He also shared how he made older color slugs still work with new sequential ones while working with Ana on a recent block theme.
  • Once more, we talked through having a dedicated place for global settings/styles.
  • We went through the recent PRs on flex layout which led to a discussion around general confusion in setting layouts, including with template parts.
  • We checked out the site editor mosaic view designs and very recent PR to implement some of the functionality.
  • We discussed the difference between patterns, templates, template parts, and reusable blocks, including going over a discussion about having starter page templates/patterns.
  • Herb shared some struggles with a combination of the new widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user. screen and configuring multiple block.jsons. Rich shared this plugin that has multiple as an example to review.
  • Carolina gave a final plug after the recording stopped for anyone who might have any ideas about how best to solve this reported dark mode problem.

If you’re a theme author who has worked with theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. in some capacity, please fill out this theme survey.

#fse-hallway-hangout, #fse-outreach-program, #full-site-editing

Help shape the future of theme design

With WordPress 5.8 came the official introduction of theme.json to the project allowing you to configure both existing editor settings like enabling custom colors and adopt new ones as they are released. Since this new mechanism is an early step towards a comprehensive style system for the future of WordPress, it’s important to hear from everyone who is currently using theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. to learn more about how folks are using this tool and what might make sense to include in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. going forward. 

If you’re a theme author who has used theme.json or is interested in doing so, please fill out this survey by August 13th to help shape its future.

If you haven’t yet used theme.json, this survey won’t be applicable to you. Thanks in advance for taking the time to share your thoughts! The results will be shared in a recap post.

If you’re curious about theme.json and haven’t had a chance to explore just yet, check out this previous call for testing offering different difficulty levels of flows and the latest WordPress News post on Configuring Theme Design with theme.json.

#fse-outreach-program, #fse-outreach-survey, #themes

Test Team Chat Summary: 20 July 2021

The meeting took place here in Slack. @hellofromtonya facilitated.

Discussion: Team Reps

Discussion starts here in Slack.

  • @lucatume asked “Who is this call open to?” Answer: all Test Team members
  • @lucatume asked “What are the responsibilities?” Answer: mostly administrative reporting duties including:
    • Gathering feedback and then writing the agenda for the biweekly team chat (example)
    • Facilitating test triage (example) and team chat (example)
    • Writing regular team recaps and posting them in Updates
    • Keeping an eye on the moving parts of the team to be able to report for quarterly updates (example)
  • Team discussed qualifications and refined them to be:

A rep is an active team member who is reliable and trusted, advocates for and is knowledgeable of one or more areas of testing, and wants to represent, nurture, and grow the team to serve the WordPress project.

Action: @hellofromtonya to publish Call for Nominations once peer review is complete. Done.

Update: Here’s the post: Test Team Reps: Call for Nominations.

Discussion: Team and Contributor Badges

Discussion starts here in Slack.

  • Reference: FSE Outreach Program badges proposal
  • Team discussed and agreed to use CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. Team’s model for awarding Test Contributor badge: 1 contribution = a badge
  • What is a test contribution?
    • @lucatume suggested to first define what is not a test contribution?
    • Do Release Party and Contributor Days participation count?
    • @hellofromtonya noted:

Broadly testing entire release is hard to capture the feedback. Right? But testing something that has a ticket or Call for Test (CfT) is trackable for not only testers but the Core folks who are working on that thing under test. Sharing how you tested (the test environment, theme, plugins, etc.), steps you did, results, and maybe even a screenshare …. this type of info is valuable.

Commenting: “It works” is not as valuable as a feedback.

Action: @hellofromtonya will start a proposal for discussion:

laying out some guidelines for how testing is captured, what testing is not, and then that will guide everyone to know what a testing contribution is and isn’t.

Update: Autogenerate e2e Tests Initiative

Discussion starts here in Slack.

#meeting-notes, #summaries

Test Team Reps: Call for Nominations

The Test Team has been without reps for years (see slack thread). With the team’s revival, it’s time to nominate (and then elect) team reps (i.e. an abbreviation for team representative).

What are Team Reps?

tl;dr

  • They represent the team
  • Quantity: 1 to 2
  • Main tasks: mostly administrative
  • Weekly time commitment: ~1 hour per week
  • Term: 1 year
  • Full details: read here
  • Who: see the qualifications below

Team reps are responsible for communicating on behalf of the team to the other contributor groups and teams via weekly updates and cross-team chats.

In the WordPress open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. project, each team has on average one or two reps. While some teams have more than two, I propose we stick with electing one to two reps for this term. For the historians out there, this role goes way back to 2012.

It is not called “team lead” for a reason. While people elected as team reps will generally come from the pool of folks that people think of as experienced leaders, remember that the team repTeam Rep A Team Rep is a person who represents the Make WordPress team to the rest of the project, make sure issues are raised and addressed as needed, and coordinates cross-team efforts. role is designed to change hands regularly.

Team Rep is a leadership role that is mostly administrative in nature; it is not a Lead role. Letting go of the Team Rep title is not a loss of status, just a handing off of responsibilities. Someone who is a leader in a team can lead whether they are doing the team rep job or not.

from Team Reps on Updates

What are the main tasks?

  • Posting a weekly update of the team’s activity in Updates
  • Raising any issues or red flags that other teams should be aware of or discussing
  • Keeping an eye on the moving parts of the team to be able to report for quarterly updates: team’s current top priority, most recent priority, challenges, and big win(s) (example)
  • Writing the agenda for the biweekly team chat (example)
  • Facilitating the team’s weekly meetings: e.g. test triage (example) and team chat (example)

What are the qualifications?

A rep is an active team member who is reliable and trusted, advocates for and is knowledgeable of one or more areas of testing, and wants to represent, nurture, and grow the team to serve the WordPress open source project.

How Do I Nominate Someone?

tl;dr

  • Where: in this post’s comments or privately via DM in Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/.
  • Deadline: Monday, August 2
  • Do you have to accept a nomination? No

We nominate people in the comments of this post. You can write a comment as simple as “I nominate @the_persons_username.” Self-nominations are welcome by leaving a comment such as “I nominate myself.” If you want to nominate someone in private, please reach out to me (@hellofromtonya) on Slack.

The deadline for nominations is Monday, August 2.

Each person who is nominated will be contacted to discuss qualifications and to accept or decline their nomination.

If you get nominated, please do not feel like you have to say “yes”! It’s okay for you to decline the nomination if you don’t feel like this is the right fit for any reason. “Thank you, but no thank you” 😉

How Will the Election Work?

If there are enough accepted nominations that an election is necessary, then we’ll use an anonymous poll for the election (example). The poll will remain open for 2 weeks.

Once the results are done, the new team rep(s) will be announced in a post (example).

Call for Test Team Rep Nominations!

Ready? It’s time to nominate people to serve for a year as our Test Team Reps.

If you have any questions, please feel free to ask in the comments. Happy nominating!

#team-reps

Test Chat Agenda for July 20, 2021

Here is the agenda for the upcoming Test Team Chat scheduled for 2021-07-20 13:00.

This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Agenda

  • Test Team Reps: Call for Nominations
  • Test Contributor and Test Team badges
  • State of the Test Handbook
  • Focal group updates
  • Open floor

Posts

Open Floor

Do you have something to propose for the agenda? Please leave a comment below.

Can’t make the meeting? Share anything relevant for the discussion in a comment below.

#agenda

Hallway Hangout: Discussion on Full Site Editing Issues/PRs/Designs (14 July)

This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. It ended up just being me and @bobbingwide so I decided to not record and instead have a free flowing chat with Herb that was less structured and more in line with our interests. We chatted for just over an hour!

We mainly chatted about theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML., what’s coming to 5.8, a few individual blocks (Query LoopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop., Featured ImageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts., etc), responsive controls, and various explorations Herb has done including building a few blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. plugins. Of those, we went through the following links that I recommend spending time reviewing too:

Next Steps

I followed up on two issues that we found while chatting: The reset to default option being too close to X for global styles UI and a bug with an empty template name and trying to rename it.

#fse-hallway-hangout, #fse-outreach-program, #full-site-editing

Test Team Chat Summary: 13 July 2021

The meeting took place here in Slack. @hellofromtonya facilitated.

This was the team’s first team chat in a year or more.

Discussion: What’s the team’s purpose?

@lucatume asked “what’s the purpose of the group?” Great question!

A healthy discussion happened around the history of the team and what role we serve in the project. The discussion starts here in the slack thread.

Some highlights are captured here:

  • Tonya shared the team’s history:
    • “The team is deeply rooted in validating and surfacing issues with user experience and usability. That’s where it started and lived in the space of testing workflows, screens, results, etc. through manual testing with users. Over the years, more and more testing has come to WordPress in the form of automated tests.”
    • “rebooting it [the team] to be all things QA and testing including triage, education, outreach, documentation, and expanding to multi-dimensional testing (automated testing, manual testing, usability – user – testing, and environment testing).”
  • Charter statement:
    • Tonya then shared “we as a team can formulate the actual charter statement. But here’s what I’m thinking (open to discussion):
    • “to create a high quality feedback loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. into the project to continuously improve and elevate the quality and usability of WordPress”
  • Is the charter statement too generic?
    • @lucatume raised that it felt too generic as the terms “quality” and usability can different meanings.
    • @boniu91 shared “I think they need to be some kind of generic. We’re looking at WordPress from many different angles and point of views while testing.”

Action Item: Tonya to publish a vision of the team’s big picture of how the team can serve the project.

Discussion: Why did the team go dormant?

@lucatume asked why the team went dormant. The slack thread starts here.

Some of the reasons given include:

  • Testing and quality know-how
  • Interest in driving the team
  • Clear path for how QA and testing can serve the project

Discussion: autogenerating e2e tests

@lucatume raised a discussion about how to autogenerate e2e tests from written step-by-step processes or recording the actual testing process. The slack thread starts here.

The team agreed this is a great idea to help anyone contribute to testing.

Action item: @lucatume will work with @justinahinon and others to identify an auto-generator tool.

#meeting-notes, #summaries