Tech News issue #10, 2022 (March 7, 2022)

00:00, Monday, 07 2022 March UTC
This document has a planned publication deadline (link leads to timeanddate.com).
previous 2022, week 10 (Monday 07 March 2022) next

Getting all the government agencies of the world into Wikidata is obviously a big, hairy and audacious goal, which means it’s just in line with everything else we do on Wikidata!

There is a saying, there is only one way to eat an elephant, a bite at a time. The way we usually organize when eating these bites is in WikiProjects, and so, of course, we have them on Wikidata as well.

An introduction to WikiProject Govdirectory

Inspired by WikiProject EveryPolitician, and quite natural for this project, the WikiProject Govdirectory is divided per country. Getting to know the context and all nuances of a country can be quite tricky, and the quality and availability of sources vary wildly.

But even countries are too big bites to chew when even a small country like Sweden has over 600 government agencies. So within each country, we are usually breaking it down even further, often down to each specific type of agency, like municipalities or courts. On this level, it becomes possible to make progress in a short amount of time and make sure that each part is modeled with high quality without overly complex queries.

To easier get an overview of the state of each bite, we have adopted a rating system. We give from 0 to 5 stars, but already when getting to two stars, the data is starting to become usable.

Link to the project: https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory

Wikidata and beyond!

While we love Wikidata, and find editing and making queries a soothing and relaxing pastime, the interface of Wikidata isn’t really tailored for the general internet visitor. So to make the data about government agencies more accessible, and even more so, actionable, we have created an external website that extracts data from Wikidata and presents it in a very different way.

“The easiest way to contact your government.” is the catchphrase for the website.

Instead of focusing on the statements and editing and adding data, this interface is optimized for citizens to get in touch with their government officials to solve whatever need they have.

Find the website at: https://www.govdirectory.org/

There could, obviously, be many more uses beyond Wikidata itself, tailored for other groups or use cases. With Wikidata’s information being licensed CC0, imagination is indeed the limit!

Let’s make government agencies on Wikidata even better – together

First, check if someone already started working on the country you are interested in. You might be in luck and can join ongoing work. If not, we have created several guides to get you started with the project pages. 

But the real work, and an essential one, is to find high-quality sources of what agencies exist and how they are structured. This will help you figure out how to model the items on Wikidata. We also encourage looking at other countries and see how they have modeled items and made queries to be able to produce accurate and up-to-date lists of what is in Wikidata.

If you need any help, we have a centralized talk page that is pretty well watched. Every week on Fridays, we organize a collaboration hour in a video chat. You are welcome to join with questions about anything, or just to see what people are working on right now. It is a very informal and casual meeting and no commitments are needed to join.

Do you want to get involved? Add yourself to the participants and start looking around.

Collaboration hour every Friday.

Get URL from Git commit hash

13:47, Sunday, 06 2022 March UTC

SHA-1 Git hashes can be mapped to code review or code repository URL to offer a web visualization with additional context.

The resolve-hash command allows to get such URL from a Git hash, or another VCS reference. It can search Phabricator, Gerrit, GitHub and GitLab currently.

Ouf of the box, it will detect your ~/.arcrc configuration and use GitHub public API. You can create a small YAML configuration file it to add Gerrit and GitLab in the mix.

Install it. Use it.

The resolve-hash package is available on PyPI.

$ pip install resolve-hash

$ resolve-hash 6411f75775a7aa8db
https::⃫github.com/10up/simple-local-avatars/commit/6411f75775a7aa8db2ef097d70b12926018402c1

Specific use cases

Projects moved from GitHub to GitLab

GitLab requires any query to the search API to be authenticated. You can generate a personal access token in your user settings; the API scope is enough, so check only read_api.

Then you can add create a $HOME/.config/resolve-hash.conf file with the following content:

# GitLab
gitlab_public_token: glpat-sometoken

For Wikimedia contributors

Gerrit exposes a REST API. To use it, create a $HOME/.config/resolve-hash.conf file with the following content:

# Gerrit REST API
gerrit:
  - https://gerrit.wikimedia.org/r/

Gerrit will be then queried before GitHub:

$ resolve-hash 311d17f289470
https::⃫gerrit.wikimedia.org/r/c/mediawiki/core/+/768149

Note if you’ve configured Arcanist to interact with phabricator.wikimedia.org, your configuration in ~/.arcrc is used BEFORE the Gerrit one. Tell me if you’re in that case, we’ll allow to order resolution strategies.

What inspired this project?

Terminator allows plugins to improve the behavior of the terminal. Some plugins allows to expressions like Bug:1234 to offer a link to the relevant bug tracker.

What if we can detect hashes, especially VCS hashes, to offer a link to the code review system, like Phabricator or Gerrit, or at least to a public code hosting facility like GitHub?

What’s next?

We can add support for private instances of GitHub Enterprise and GitLab. Code I wrote in VCS package is already ready to accept any GitHub or GitLab URL, and is so prepared to accept a specific instance, so it’s a matter of declare new configuration options and add the wrapper code in VcsHashSearch class.

A cache would be useful to speed up the process. Hashes are stable enough for that.

Write a Terminator plugin, so we solve the root problem described above.

The code is extensible enough to search other kind of hashes than commits, but I’m not sure we’ve reliable sources of hashes for know files or packages.

References

weeklyOSM 606

11:13, Sunday, 06 2022 March UTC

22/02/2022-28/02/2022

lead picture

Measure with the camera (AR) [1] photo © StreetComplete | map data © OpenStreetMap contributors

Mapping campaigns

  • Andrés Gómez Casanova reported (es) > en on the progress of the ‘Note-a-thons’, an initiative of the MaptimeBogota group, which started this year and aims to resolve open notes in OSM in cooperation with other Latin American communities.

Mapping

  • The Polish OSM community presented a mapping project to help Ukrainian refugees by preparing a map (Menu in (en), (pl), (uk)) that shows refugee reception points and seeks to add Ukrainian names for Polish cities.
  • Fizzie41 reported on how the Australian mapper community has made progress on resolving open OpenStreetMap notes.
  • Richlv described a procedure to clean up untagged ways that are not members of any relations using JOSM.
  • Before-and-after micromapping examples have become popular, particularly on the OSM subreddit.
    User whatismoss has now extended
    the genre with a diary post about a shopping mall involving both 3D and indoor mapping.
  • Voting on the following proposals has closed:
    • transformer=*, to refine power transformers’ classification to make it clearer and remove redundancy with substation=*, was successful with 19 votes for, 0 votes against and 1 abstention.
    • boundary=forest, the latest version of a tagging scheme for forest boundaries (managed or unmanaged forests), was also successful with 18 votes for, 0 votes against and 0 abstentions.

Community

  • The next online meeting of the Latin American OSM community is on Saturday 26 March. OSM Mexico will be in charge of the coordination and more information is available on the event’s wiki page (es) > en.
  • OSM Fukushima members have released videos introducing weeklyOSM 601 and 602 (ja). They discover buildings in Tonga with extended roofs and discuss how to map them.
  • Ilya Zverev shared (ru) > en his personal view about mapping in difficult times and the risks of mapping objects under dispute, but also expresses the value of OSM in support of humanitarian activities.

OpenStreetMap Foundation

  • Mikel Maron shared his view on the draft resolution on membership prerequisites.
  • The OSMF board has rejected the application by MapUganda for local chapter status. A key reason was that there is insufficient separation between MapUganda’s commercial and community activities.
  • Heather Leson, in her diary, provided an update on how the HOT Governance Working Group, which she leads, plans to look at HOT governance moving from a governance model with membership control of the Board to a more diverse and inclusive model, and asked for volunteers to assist in this process of organisational change.The second part of the diary entry concerned current activities of the LCCWG (Local Chapters and Community) related to Codes of Conduct (CoC).

Education

  • Ever wanted to know what is under the hood of the OSM-driven route planner for cycle touring, cycle.travel? Richard Fairhurst presented an in-depth look at his site as part of the talk programme of the (virtual) 2022 Cycle Touring Festival.
  • raphaelmirc recommended (pt) > en a number of OSM Brazil videos (in Portuguese) created to improve mapping in OSM.
  • User unen reported on the aims and objectives of HOT’s ‘Asia Pacific OSM Help Desk’ and invited people to book an appointment for one of the sessions available every Monday (09:00 UTC) and Wednesday (06:00 UTC).

Maps

  • Anne-Karoline Distel described in her diary how she created a 3D-printed model, using OSM data, of the churchyard in Magorban for visually impaired people.
  • On gnulinux.ch (de) > en Ralf Hersel introduced readers to the osMap project, which displays OpenStreetMap data using names in selected Latin character set languages worldwide. For example English, French, Spanish

Programming

  • An OSM wiki page has been created for the 2022 Google Summer of Code. Compared with previous years, participants will be required to have a more in-depth understanding of both OSM and software components of the project. Mentor organisations will be informed by Google on 6 March.
  • Pieter Vander Vennet wrote about the new implementation of MapComplete’s language picker.
  • RadicallyOpenSecurity (ROS) has audited the security of Nominatim’s codebase and data.
  • Trufi’s lead developer gave a video guided tour of the Trufi code base to a group of new volunteer developers.

Releases

  • The February 2022 maintenance version (17.0.3) of Vespucci has been released.
  • A February 2022 version of OrganicMaps has been released for Android and iOS. The most important changes concern updated map data (4 February), fixes in routing and fixes in styles (e.g., parking, house numbers and railway stations).
  • StreetComplete v41 now allows users to measure the widths of streets and cycleways, as well as the height below overhead obstructions, using smartphone cameras with augmented reality. You can read about the results of the first tests regarding expected accuracy of AR measurement on GitHub.

Other “geo” things

  • Russian geographers have published (ru) > en an open letter against military operations in Ukraine, signed by more than 1800 geographers.

Upcoming Events

Where What Online When Country
Xexéu Mapathona na Cidade Xexéu – PE -Brasil – Edifícios, Estradas, Pontos de Interesses e Área Verde osmcalpic 2022-03-05 – 2022-03-06 flag
OSMF Engineering Working Group meeting osmcalpic 2022-03-07
Meatu Map Tanzania to help protect girls from FGM for International Womens’ Day osmcalpic 2022-03-08 flag
Marburg FOSSGIS 2022 osmcalpic 2022-03-09 – 2022-03-12 flag
Michigan Michigan Meetup osmcalpic 2022-03-10 flag
München Münchner OSM-Treffen osmcalpic 2022-03-10 flag
Berlin 165. Berlin-Brandenburg OpenStreetMap Stammtisch osmcalpic 2022-03-10 flag
Lyon EPN des Rancy : Technique de cartographie et d’édition osmcalpic 2022-03-12 flag
臺北市 OpenStreetMap x Wikidata Taipei #38 osmcalpic 2022-03-14 flag
San Jose South Bay Map Night osmcalpic 2022-03-16 flag
149.Treffen des OSM-Stammtisches Bonn osmcalpic 2022-03-15
Lüneburg Lüneburger Mappertreffen (online) osmcalpic 2022-03-15 flag
London Geomob London osmcalpic 2022-03-16 flag
Großarl 4. Virtueller OpenStreetMap Stammtisch Österreich osmcalpic 2022-03-16 flag
City of Subiaco Social Mapping Sunday: Shenton Park osmcalpic 2022-03-20 flag
Habay-la-Neuve Réunion des contributeurs OpenStreetMap, Habay-la-Neuve osmcalpic 2022-03-21 flag
Barcelona Geomob Barcelona osmcalpic 2022-03-22 flag
City of Nottingham OSM East Midlands/Nottingham meetup (online) osmcalpic 2022-03-22 flag
[Online] OpenStreetMap Foundation board of Directors – public videomeeting osmcalpic 2022-03-24
Perth Social Mapping Online osmcalpic 2022-03-27 flag

Note:
If you like to see your event here, please put it into the OSM calendar. Only data which is there, will appear in weeklyOSM.

This weeklyOSM was produced by Nordpfeil, PierZen, SK53, Sammyhawkrad, SomeoneElse, Strubbl, TheSwavu, YoViajo, derFred.

GitLab: Rethinking how we handle access control

23:44, Friday, 04 2022 March UTC

I'll start with a bit of general administrivia. First, our migration of Wikimedia code review & CI to GitLab continues, and we're mindful that people could use regular updates on progress. Second, I need to think through some stuff about the project, and doing that in writing is helpful for all involved. I'm going to try writing occasional blog entries here for both purposes.

Now on to the main topic of this post: Access control for groups and projects on the Wikimedia GitLab instance.

The tl;dr: We've been modeling access to things on GitLab by using groups under /people to contain individual users and then granting those groups access to things under /repos. This has been tricky to explain and doesn't work as well at a technical level as we'd hoped, so we're mostly scrapping the distinction, and moving control of project access to individual memberships in groups under /repos. This should be easier to think about, simpler to manage, and seems like it will suit our needs better. Read on for the nitty-gritty detail.

(Thanks to @Dzahn, @Majavah, @bd808, @AntiCompositeNumber, and @thcipriani for helping me think through the issues underlying this post.)

Background

During the GitLab consultation, when we were working on building up a model of how we'd use GitLab for Wikimedia projects, we wrote up a draft policy for managing users and their access to projects.

GitLab supports Groups. GitLab groups are similar to GitHub's concept of organizations, although the specifics differ. Groups can contain:

  • Other, nested groups
  • Individual projects (repositories & metadata)
  • Users as members; members of other groups can be invited to a group
    • A user who is a member of a top-level group is also a member of every group it contains

We've since changed the original draft policy in some small ways - in particular, we decided to move most projects into a top-level /repos group in order to offer shared CI runners (see T292094). You can read the policy we landed on at the latest revision of GitLab/Policy on mediawiki.org.

The basic idea was that we would separate groups out into:

  1. Sub-groups of /repos: Namespaces for projects, split up by functional area of code
  2. Sub-groups of /people: Namespaces for individual users, split up by organizational units like:
    • Volunteer group
    • Teams at organizations such as the WMF, WMDE, etc.

Groups in /people could then be given access to projects under /repos.

Our hope was that this would let us decouple the management of groups of humans from the individual projects they work on, and ease onboarding for new contributors. A new member of the WMF Release Engineering team, for example, could be added to a single group and then have access to all the things they need to do their job.

We intended for most /people groups to be owned by their members, who would in turn have ownership-level access to their projects under /repos, allowing for contributors to a project to manage access and invite new contributors.

As a concrete example:

Problems with this scheme

I've been proceeding under this plan as people request the creation of GitLab project groups, but there turn out to be some problems.

First, it doesn't seem like permission inheritance for nested groups with other groups as members works the way you'd expect & hope: See T300939 - "GitLab group permissions are not inherited by sub-groups for groups of users invited to the parent repo".

Second, users have concerns about equity of access and tight coupling of things like employment with a specific organization to project access. We didn't have any intention of modeling any group of users as second-class citizens within this scheme, but it seems to create the impression of one all the same. It's also striking that the set of projects people work on just isn't that cleanly mapped to any particular organizational structure. Once you've been a technical contributor for a while, you've almost certainly collected responsibilities that no org chart reflects accurately.

Finally, and maybe most importantly, this is a complicated way to do things. People have a hard time thinking about it, and it requires a lot of explanation. That seems bad for an abstraction that we'd like to be basically self-serve for most users.

Proposed solution

Mostly, my plan is to use groups closer to how they seem to be designed:

  1. Sub-groups of /repos will contain both individual contributor memberships and projects
  2. Except in occasional one-off cases, access should be granted at the level of a containing group rather than at the level of individual projects, so as to avoid micromanaging access to many projects.
  3. We'll keep /people in mind as a potential solution for some problems (for example, it might be a good tool for synchronizing groups of users from LDAP and granting access to certain projects on that basis), but not rely on it for anything at the moment.

There are some unanswered questions here, but I plan to redraft the policy doc, move existing project layouts to this scheme, and start creating new project groups on this basis in the coming week or so.

My main philosophical takeaway here is that I work with a bunch of anarchists, and it's always best to plan accordingly.

Originally, one of our goals for this migration was avoiding a repeat of the weird, nested morass that is our current set of Gerrit permissions. While it would be a good idea to keep the structure of things on GitLab flatter and easier to think about, I'm no longer that worried about it. Some of the complexity is inherent to any large set of projects and contributors; some of it just reflects a long-lived technical culture that's emergent and largely self-governing, tendencies that nearly always resist well-intentioned efforts to rationalize and map structure to things like official organizational layout.

Students improve articles on how bills became laws

18:42, Friday, 04 2022 March UTC

Wikipedia’s articles about US federal laws tend to have high numbers of pageviews — despite the fact that they’re often incomplete. Like all articles, those about public policy tend to focus in on what was of particular interest to the volunteers who have contributed to it in the past, leaving other sections missing or underdeveloped. That’s where Nicole Mathew’s Capstone Course in American Politics came in. Her Oakland University students each took a federal law from 1947 to 2012 and updated them. Since the students’ work this fall, the articles they edited have been viewed more than 300,000 times.

One popular article was that of the National Security Act of 1947, which restructured the military and intelligence agencies in the United States post-World War II. The student overhauled the article, which used to have only seven paragraphs and six citations. Today, thanks to the student’s work, it’s a fully developed article with twenty-one citations as well as detailed descriptions of its legislative history and provisions.

Another article the class expanded was the Civil Rights Act of 1960, which addressed voting rights and a number of other civil rights issues. The student editor added significant information about the background and legislative history of this act, helping place it in its historical context.

Another civil rights legislation article to get improved was the Civil Rights Act of 1968, the landmark law establishing hate crimes, the Indian Civil Rights Act, the Fair Housing Act, and the Anti-Riot Act. Since 2019, the article had included a warning banner asking readers to expand each section about the ten Titles in the act; the student editor finally did, extensively documenting what each one covers. The student also added sections to the background and legislative history of the article.

Like civil rights, immigration is still today a topic of legislative discussion. A student editor in the course expanded Wikipedia’s coverage of the Immigration and Nationality Act of 1965, significantly documenting its legislative history. The student also added information about its background and more effectively detailed the provisions in the legislation.

Before another student editor worked on it, Wikipedia’s article on the Fair Packaging and Labeling Act was what’s known on Wikipedia as a stub — a short article without much information or many citations. A student editor significantly expanded it, adding sections on the act’s background, legislative history, and provisions.

Other students in the course also added information to the Freedom of Access to Clinic Entrances Act, the McKinney-Vento Homess Assistance Act, the Public Health Cigarette Smoking Act, and the Health Insurance Portability and Accountability Act, or HIPPA, which has been in the news recently.

In all these examples, students are better enabling United States citizens to understand the laws of their country — the context they emerged from, the path they took to become law, and what they state. Having an informed citizenry requires having neutral, fact-based information about laws and other public policy topics — a key role Wikipedia can play.

If you’re a politics or public policy instructor interested in having your students help improve Wikipedia’s coverage, reach out! Visit teach.wikiedu.org for more information.

Image credit: Ralf Roletschek (talk) – Infos über Fahrräder auf fahrradmonteur.de Wikis in der Ausbildung, CC BY-SA 3.0, via Wikimedia Commons

Wikidata maxlag, via the ApiMaxLagInfo hook

16:21, Friday, 04 2022 March UTC

Wikidata tinkers with the concept of maxlag that has existed in MediaWiki for some years in order to slow automated editing at times of lag in various systems.

Here you will find a little introduction to MediaWiki maxlag, and the ways that Wikidata hooks into the value, altering it for its needs.

Screenshot of the “Wikidata Edits” grafana dashboard showing increased maxlag and decreased edits

As you can see above, a high maxlag can cause automated editing to reduce or stop on wikidata.org

MediaWiki maxlag

Maxlag was introduced in MediaWiki 1.10 (2007), moving to api.php only implementation in 1.27 (2016).

If you are running MediaWiki on a replicated database cluster, then maxlag will indicate the number of seconds of replication database lag.

Since MediaWiki 1.29 (2017) you have also been able to increase maxlag reported to users depending on the number of jobs in the job queue using the $wgJobQueueIncludeInMaxLagFactor setting.

The maxlag parameter can be passed to api.php through a URL parameter or POST data. It is an integer number of seconds.

If the specified lag is exceeded at the time of the request, the API returns an error (with 200 status code, see task T33156) like the following

Manual:Maxlag parameter

{ "error": { "code": "maxlag", "info": "Waiting for 10.64.48.35: 0.613402 seconds lagged.", "host": "10.64.48.35", "lag": 0.613402, "type": "db", }, "servedby": "mw1375" }
Code language: JSON / JSON with Comments (json)

Users can retrieve the maxlag at any time by sending a value of -1 (the lag is always 0 or more, so this always presents the error.

Generally speaking the maxlag on most sites with replication should be below a second.

Users of Wikimedia sites are recommended to use a maxlag=5 value, and this is the default in may tools such as Pywikibot. You can read more about this on the API Etiquette page.

Modifying maxlag

Back in 2018 in response to ongoing Wikidata dispatch lag related issues, we implemented a way to modify how and when the maxlag error was shown to users from extensions. Conveniently the maxlag error already included a type value, and we had the plan to add another!

The new hook ApiMaxLagInfo was born (gerrit change, documentation)

Receivers of the hook will get the MediaWiki calculated $lagInfo, and can decide if they have a system that is more lagged than the current lag they have been passed. If they do, they can overwrite this $lagInfo and pass it on.

In the diagram below we can see this in action

  • MediaWiki determins the maxlag to be 0.7s from its sources (replication & optional jobqueue)
  • Hook implementation 1 determines its own maxlag to be 0.1s, and decides to not change the already existing 0.7s
  • Hook implementation 2 determines its own maxlag to be 1.5s, so it replaces the lower 0.7s
  • The final maxlg is 1.5s and this is what is used when checking the maxlag parameter provides by the user, or when displaying the lagged value to the user

Factors

As with the optional MediaWiki job queue maxlag integration, all usages of the ApiMaxLagInfo hook generally come with their own configurable factor.

This is due to the expectation that ~5 seconds of lag is when automated clients should back off.

For some systems, we actually want that to happen at say 2 minutes of lag, or when the job queue has 1000 entries. The factor allows this translation.


// 5 = 1000 / 200 $lag = $jobs / $factor;
Code language: PHP (php)

Dispatching

Dispatching or change propagation has been part of Wikibase since the early days. This mechanism keeps track of changes that happen on Wikibase, emitting events in the form of MediaWiki Jobs to any clients (such as Wikipedia) that need to be notified about the change for one reason or another.

Historically dispatching has had some issues with being slow, which in turn leads to updates not reaching sites such as Wikipedia in a reasonable amount of time. This is deemed to be bad as it means that things such as vandalism fixes take longer to appear.

Before adding dispatch lag to max lag the Wikidata team already had monitoring for the lag, and would often run additional copies of the dispatch process to clear backlogs.

You can find numerous issues relating to dispatch lag issues, and before this was added to maxlag Wikidata admins would normally go around talking to editors making large numbers of edits, of blocking bots.

Dispatch lag was origionally added as a type of maxlag in 2018, and was the first usage of the ApiMaxLagInfo hook.

The value used was the median lag for all clients with a factor applied. This was calculated during each request, as this median lag value was readily available in Wikibase code.


$dispatchLag = $medianLag / $dispatchLagToMaxLagFactor;
Code language: PHP (php)

Later in 2018 we inadvertently made dispatching much faster. The whole system was rewritten in 2021 as part of T48643: [Story] Dispatching via job queue (instead of cron script), and a decision was made to no longer include dispatch lag in maxlag.

Queryservice updates

The query service update lag was added as a type of maxlag on Wikidata since 2019 due to the ongoing issues that the query service was having staying up to date with Wikidata. You can find an ADR on the decision in the Wikidata.org MediaWiki extension.

The value used was calculated in a maintenance script that is run every minute. This script takes the lastUpdated values of query service backends as recorded in prometheus looking for the backend that is the next most lagged server from the median. This value is then stored in a cache with a TTL of around 70 seconds. (code)


public function getLag(): ?int { $lags = $this->getLags(); // Take the median lag +1 sort( $lags ); return $lags[(int)floor( count( $lags ) / 2 + 1 )] ?? null; }
Code language: PHP (php)

During every request, this cached value is then checked and a factor applied. (code)


$dispatchLag = $store->getLag() / $factor;
Code language: PHP (php)

In 2021 the Wikidata Query Service had fully switched over to its’ new streaming updater which should mostly tackle the lag issues.

The post Wikidata maxlag, via the ApiMaxLagInfo hook appeared first on addshore.

On 1 March 2022 the Wikimedia Foundation received a Russian government demand to remove content related to the unprovoked invasion of Ukraine posted by volunteer contributors to Russian Wikipedia. The Wikimedia Foundation and the movement we are part of have never backed down in the face of government threats to deny people their fundamental human right to access free, open, and verifiable information

The information available on Wikipedia is sourced and shared by volunteers who invest time and effort to ensure that the content is fact-based and reliable. Many continue to do so in adverse circumstances: As the invasion continues, Ukrainian volunteers have continued to add content and make edits to Wikipedia, even in face of deep hardship.

Tuesday’s takedown request threatened censorship. Denying people access to reliable information, at a time of crisis, can have life-altering consequences. We join Wikimedia Russia, the independent Russia based Wikimedia affiliate and the broader community of Russian Wikipedia volunteers in defending the right of  volunteers to continue their diligent work of editing Wikipedia with the most up-to-date and reliable information available related to Russia’s invasion of Ukraine. 

Wikipedia is a crucial second draft of history, freely and openly providing important historical and current context of Russia, Ukraine and many other topics. When the invasion of Ukraine began on February 23rd, volunteer editors immediately started to create Wikipedia articles to inform the world about unfolding events. As of March 3rd, the English Wikipedia article about the invasion has been viewed over 11.3 million times, and there are articles about the topic in over 99 languages. 

As ever, Wikipedia is an important source of reliable, factual information in this crisis. In recognition of this important role, we will not back down in the face of efforts to censor and intimidate members of our movement. We stand by our mission to deliver free knowledge to the world. 

2021 Coolest Tool Awards: Thank you for the tools!

18:51, Thursday, 03 2022 March UTC

The annual Coolest Tool Award celebrated software tools created and used by the Wikimedia community. Here are this year’s winners.

By André Klapper, Staff Developer Advocate, The Wikimedia Foundation

Wiki communities around the globe have diverse use cases and technical needs. Volunteer developers are often the first to experiment with new ideas, build local and global solutions and enhance the experience in our software. The Coolest Tool Award aims to acknowledge that, and make volunteer developers’ work more visible.

The third edition of the Coolest Tool Award took place on 14 January 2022. The event was live streamed on Youtube in the MediaWiki channel. The video is also available on Wikimedia Commons. The award was organized and selected by the Coolest Tool Academy 2021 and friends, based on nominations from our communities.

In total, ten tools were awarded in the categories listed below, and four more tools received honorable mentions. A tool is a piece of software used in the Wikimedia ecosystem. Examples for tools include gagdets, MediaWiki extensions, websites, web services, APIs, applications, bots, or user scripts.

Thank you for all your work! 🎉

2021 winners

Experience Intuitive and easy to use

WikiShootMe is a map of images and geographical locations that helps you find points of interest that need photos. It is both intuitive but also powerful, and it works well on both desktop and mobile.

To quote from a nomination, “I often show this tool to newcomers. It really helps them understand Wikidata and the relationship with Wikimedia Commons.”

Developer Tools that primarily serve developers

Quarry is a public interface to query replica databases of public Wikimedia wikis, via a web browser. You can also share your queries with others.

Newcomer New tools or tools by new developers

GlobalWatchlist shows your watched pages across all wikis on a single page.

Diversity Tools that help include a variety of people, languages, cultures

Humaniki allows to explore the gender gap in content of Wikimedia projects by dimensions such as country, language, or date of birth.

Quality Tools to improve content quality

Depictor is a game for filling in the gaps of Structured Data on Wikimedia Commons by adding depicts statements to images. It is mobile friendly and a fun way to make image search better.

Tiny Small tools and tools that do one thing well

diffedit is a user script that enables editing directly from viewing the differences between two versions of the same wiki page. It is very useful for patrolling.

Versatile Tools that support multiple workflows

Convenient Discussions is a user script that allows the user to post and edit comments without switching to a separate page. You do not need to search code for wiki markup, read talk pages through diffs, or deal with edit conflicts.

Reusable Serves many wikis and projects

Wudele lets you quickly schedule a poll or an event by setting up potential options and letting others vote on the best time or choice.

Editor Tools that augment editing

PAWS is a Jupyter notebook deployment. It allows to create and share documents which contain live code. You can run scripts that help with creating visualizations or write technical documentation and tutorials that help others.

Eggbeater Tools in use for more than 10 years

Earwig’s Copyvio Detector finds possible copyright violations in Wikipedia articles by searching the web, following the article’s links, and checking with external image databases. It is multilingual and an essential tool for maintaining Wikipedia’s quality.

Honorable mentions

  • CopyPatrol allows identifying recent copyright violations on several Wikimedia projects.
  • MediaWiki CLI makes it easy to set up a MediaWiki instance, services for development purposes, and to run many integration tools.
  • VizQuery lets you query Wikidata by property and property value. It offers autocomplete and the search results show images.
  • And for WBStack, let’s quote one of its users: “The true promise of Wikidata comes from the ability to build a federation of Wikibases all using the same data primitives, and WBStack is making that promise a reality by letting anyone create a fully functioning Wikibase at the press of a button.”

Big thanks to everyone who nominated their favorite tools, everyone who contributed code, and everyone who spread the word about the Coolest Tool Award!

For more details on the awarded projects, please see the Coolest_Tool_Award/2021 page.

andré — for the 2021 Coolest Tool Academy

Don’t Blink: Public Policy Snapshot for February 2022

10:00, Thursday, 03 2022 March UTC

The Wikimedia Foundation’s Global Advocacy team works to protect and promote a global regulatory environment in which everyone can freely share in the sum of all knowledge. The month of February 2022 has been a busy one when it comes to legislative and regulatory developments around the world that shape people’s ability to participate in the free knowledge movement. In case you blinked, we’re happy to catch you up. In this article we review the most important developments that have preoccupied us and the actions we’ve taken to advance fundamental rights online such as the freedom of expression, privacy, and access to knowledge. We’ll also highlight the team’s work to protect the public-interest Internet, and our vision of an online ecosystem in which everyone can freely produce, access, share and remix information. As a global movement, Wikimedians can uphold this vision together.  

To learn more about our team and the work we do, follow us on Twitter (@WikimediaPolicy) or sign-up to the Wikimedia public policy mailing list. The team’s Meta page is under construction.


US Legislative Developments

  • EARN IT Act: The Global Advocacy team opposed the EARN IT Act, which would significantly weaken protections that online intermediaries, like Wikipedia, have from liability concerning child sexual abuse material (CSAM) and undermine encryption. The Foundation has signed an open letter opposing the bill alongside a coalition of more than 60 other concerned organizations, which was published by the Center for Democracy and Technology. The team also published a blog post outlining the negative impact that the legislation, and others like it, would have on freedom of expression, privacy, and open knowledge projects like Wikipedia. Although the legislation passed out of the US Senate Judiciary Committee, the team’s blog post and open letter were introduced into the official record
  • Copyright and related rights: The Public Policy team advocated for copyright standards that would support the free and open nature of the internet on two counts. First, we signed an open letter against the Journalism Competition & Preservation Act (JCPA), which proposed granting media organizations the right to restrict third parties from linking to external content. These basic fair uses are essential to millions of websites and online services like Wikipedia, as they use links to connect content to online communities large and small. To further advocate for copyright norms that enable the free exchange of knowledge, we also submitted comments to the US Copyright Office ahead of upcoming proceedings evaluating technical standards for protecting copyrighted works online. 

Latin America and the Caribbean

  • Chile: The Chilean Congress has been considering a very concerning bill to regulate digital platforms since September 2021. If approved, the bill has the potential to become a misguided influence on similar regulations throughout the region. Both international and Chilean groups, including Wikimedia Chile and WMF, have expressed concern over the overly broad nature of the bill, lack of consultation with civil society, and last-minute hearings around the legislation. The Global Advocacy team has been supporting Wikimedia Chile to analyze the bill and its legal implications for open knowledge projects like Wikipedia.

Asia

  • SIM Card Registration Act in the Philippines. We endorsed a coalition letter to the President of the Philippines urging him to veto the SIM Card Act. The act targets telecommunications and social media entities in such a way that ultimately criminalizes pseudonymity online. The Act forbids the use of pseudonyms when registering accounts with social media, establishing a minimum of 6 years in prison and/or a harsh fine. It sets a worrying precedent of curtailing freedom of expression and privacy in a way that could apply to other online sites, including Wikimedia projects. 
  • Myanmar Draft Cybersecurity Law: A draft Cyber Security Law in Myanmar has been introduced one year after a military coup deposed the country’s democratically elected government. The law includes overly broad and poorly defined prohibitions and requirements that are inconsistent with international standards related to freedom of expression. Additionally, the military would have the legal authority to order content deleted, block digital platforms, seize individuals’ devices, and criminalize the use of VPN all without due process. Civil society and human rights organizations have supported a statement by the Global Network Initiative (GNI) calling for the withdrawal of the law. 

European Union

  • Our colleagues from the Free Knowledge Advocacy Group EU have been promoting the interests of EU Wikimedia Affiliates. The Digital Services Act (DSA) is in trilogue, i.e. the three main EU bodies have adopted their respective positions and are now negotiating over a  common version. We support language adopted by the Parliament that differentiates between a) terms of use of a platform operator and b) the rules set by the communities that use the platforms. In addition, we endorse the Parliament’s view on intermediary liability that leaves room for community-lead content moderation. 

Additional Developments

  • Online Safety Bills: the Global Advocacy team published a blog post on the common pitfalls of online safety regulations. Such bills generally address the symptoms rather than the causes of online harms and, in the process, threaten open knowledge communities and individuals’ fundamental human rights around privacy, freedom of expression, and access to knowledge.  Online safety bills have been proposed in Australia, the UK, and Canada, and we are closely following the debates around them.
  • Human Rights Policy: The Global Advocacy team participated in an Office Hour with the Board of Trustees on 17 February to brief the Movement on the recently adopted Human Rights Policy and to answer questions from the community. The briefing emphasized the Foundation’s commitment to protecting and upholding the human rights of Wikimedia users and consulting the community. 

The Wikimedia Movement is one of the most important, collaborative, and community driven initiatives in the world, it is also a meeting place to produce knowledge in a participatory way. But what happens when half of the population is underrepresented or not represented at all in these projects?

Currently, only 18% of the content in all Wikimedia projects, including biographies on Wikipedia, are of women. About 25% of movement organizers, who lead events, build community identity, and direct priorities for the movement, are also women. Historically, more men than women have edited Wikipedia. This has a direct result on the kind of information that is covered and how that information is presented.

Change first begins with knowledge and on International Women’s Day this year, the Foundation aims to shed light on the gender bias across the information landscape by sharing the facts and highlighting the Wikimedia movement’s efforts in closing these gaps.

On  March 8th, the Wikimedia Foundation will be hosting a virtual panel discussion #BreaktheBias – Celebrating women’s achievement and taking action for equity with movement leaders who are working to improve gender diversity in the Wikimedia projects and moderated by our very own CEO, Maryana Iskander. It will take place virtually at 16:00 to 17:00 UTC and you can join the event by clicking here

Join us to learn more about the movement’s achievements, challenges, and opportunities in closing the gender gap. Together, we can build a more inclusive, equitable, and diverse community! 

Panelists will include

Carmen Alcazar – Wikimedian of the Year 2021; President of Wikimedia Mexico, Founder of Editatona.

Erina Mukuta – Project Lead, Wikimedia Community User Group Uganda

Mónica Bonilla-Parra – Project Coordinator, Wikimedia Colombia

Rosie Stephenson-Goodknight – Trustee, Wikimedia Foundation

On Friday, February 18, 2022, the Trust and Safety Policy Team hosted a Panel Q&A event with community members to discuss the relevance of the Universal Code of Conduct(UCoC) and Enforcement Guidelines, particularly to small and medium sized wikis. As you may be aware, the draft for the Enforcement Guidelines has been completed and is  moving to a community ratification vote from 7 – 21 March 2022. 

This panel discussion was organized as part of efforts to create awareness about the UCoC, its Enforcement Guidelines and the upcoming ratification vote. It had about 25 staff and volunteers attending with 4 panelists; Uzoma Ozurumba from the Igbo community, Ruby Brown from the Open Foundation West Africa, Chabota Kanguya from the Chichewa community  and Nahid Sultan from the Bengali community. The event, which lasted for over an hour, covered various issues ranging from the importance of the existence of a UCoC to the need for small/medium wikis to participate in the ratification vote and get their views/interests represented.

           

One of the main topics of discussion was whether or not the UCoC and its Enforcement Guidelines are important/necessary. Uzoma remarked that the UCoC is particularly important for smaller wikis because they tend to be fragile and vulnerable so having a UCoC will create a culture of accountability, trust, inclusivity and safety. She also added that having the UCoC without the Enforcement Guidelines is like having a “toothless security/guard dog that can’t bark or bite to protect you”. Isaac mentioned that it will create a safer environment for contributors to collaborate in harmony with people from diverse cultures and backgrounds. Nahid also added that smaller wikis have fewer active users and thus fewer resources and are at risk for higher levels of harassment and toxicity so the UCoC Enforcement Guidelines would be useful at creating equitable standards across all communities. 

                       

On why it is necessary for (small/medium sized) communities to vote on the UCoC Enforcement Guidelines, panelists agreed that it is particularly important for community members from smaller wikis to vote to get their voices/views represented in the vote. Uzoma and Isaac particularly mentioned that since the UCoC and EG are going to be binding on everyone, consensus is very important hence the need for everyone to participate. Ruby also added that smaller wikis may feel that their votes could be insignificant because of their size but their opinion and vote still matters and are equally important.

                              

We rounded up the discussion with questions on the U4C and UCoC related training for administrators and functionaries. Panelists echoed the need for that to exist especially for smaller wikis that may not have the existing structures and systems to such a committee to support them with dealing with these issues. 

                The ratification vote for the UCoC Enforcement Guidelines will be happening from March 7, 2022 to March 21, 2022. You can read more about how to vote, the UCoC and the Enforcement Guidelines to feel empowered to vote! Also, you can find a full recording of the panel discussion here.

Art+Feminism Call to Action Art Commission

12:02, Wednesday, 02 2022 March UTC

Art+Feminism is pleased to announce three new pieces hosted on Wikimedia Commons as part of the Call to Action Art Commission program by artists Gucora Andu (Kenya), Aditi Abhijit Kulkarni (India), and Thamires Fortunato Martins (Brazil). The new pieces by Gucora Andu and Aditi Kulkarni are available now on Wikimedia Commons. The work of Thamires Fortunato Martins will be available in mid-March 2022. Read more about the artists and their work below.

Art+Feminism team members and Co-curators Zita Ursula Zage, Medhavi Ghandi, and Juliana Montiero selected these three artists out of 85 applications to create original pieces of work that visualize Art+Feminism. The work will be used by the global Art+Feminism community in future Art+Feminism campaigns.

In our most recent open call, we celebrated the Global South, exclusively inviting applications from artists located in regions where the curators of the call reside:  African countries, Brazil, India, Bangladesh, Pakistan and Sri Lanka. Each commission came with a 2000$ (USD) artist fee.

Call to Action art commissioning program was established by Art+Feminism in 2017 with the goal is to both highlight the work of contemporary artists and to expand the body of images available that represent the project.

Please join us in celebrating these artists and these new works! Join us in a remix workshop on March 28 at 4pm UTC led by Sara Clugage, where we’ll be adapting these works into new ones (like embroidered patches!) and putting the remixed works back on Wikimedia Commons.


An illustration by Gucora Andu of two figures symbolizing internalized misogyny.

Internalized Misogyny (2022) As feminists, we are convinced that we see everyone equally, yet most of us have internalized misogyny. One of the most challenging parts of becoming a feminist has been the sometimes startling disparity between my current beliefs and how I’ve been conditioned to think and behave. Internalized sexism might be difficult to identify. Regardless of how independent we feel we are, we have a plethora of preconceived notions about how a woman should live that stem from societal expectations and gender conventions. It is vital to be conscious of this, and your own thoughts and opinions, not only about other women but also about yourself. The solution, I believe, is that empowerment is not found in the perfect unison of belief and action, but rather in the chaotic mix between the two. Learning, in some ways, implies acknowledging we were wrong at one point, so perhaps we can take solace in the often contradictory process of becoming the woman we want to be.

Photo of Gucora Andu smiling at the camera wearing a white camisole.

Gucora Andu means ‘To Draw People’ in Kikuyu, a Kenyan Language. My work focuses on illustrating people, most especially women, and issues that concern them. In a country where aspects of African tradition contribute to widespread chauvinistic attitudes, Gucora Andu is a platform that promotes and encourages feminist values. Gucora Andu has worked with NGOs, feminist and female-focused platforms to raise awareness of feminist realities throughout Africa and the world. | Instagram: @gucora.andu | Twitter: @gucora_andu


Black and white image by Aditi Abhijit Kulkarni of five figures in the compartment reserved for women in the Mumbai local trains with a black border with white letters that repeat the text ‘Take Space, Make Space'

Ladies Dabba (2022) represents the compartment reserved for women in the Mumbai local trains. It provides a safe space in public transport and bears witness to a diverse group of women of all ages, from different walks of life. There is a sense of familiarity among strangers as they sit or stand shoulder to shoulder and are free to exist without inhibitions given the lack of the male gaze. To journey in this compartment is to experience average everyday interactions between women – both compassionate and harsh. It is a space that is as flawed and gentle as any woman travelling in it. ‘Take Space, Make Space’ hopes for women claiming and thereby creating bigger and healthier spaces for their kind.

Photo of Aditi Abhijit Kulkarni smiling at the camera wearing a blue and white button down shirt.

Aditi Kulkarni is based out of Mumbai, India and enjoys creating art about mundane everyday activities through her observations of the surroundings. She endeavours to showcase her interpretation of the world around in a personal and sometimes humorous way. | @aditi_koolkarni


Photo of Thamires Fortunato Martins smiling looking off camera wearing a burgundy sweater with a blue collared shirt underneath.

Thamires Fortunato lives in the North Zone of Rio de Janeiro. A multidisciplinary artist, she works with digital collage and video art. Her research reconstructs the peripheral and Black narrative, with aspects of the struggle involving incredible Black women in the context of resistance in the African Diaspora. She participated in the exhibition A Zero na Alfaiataria, in Curitiba, in the show focused on the publication of artists from the Residência Artística with the work “Reconheça-me Marli Coragem”. She also participated of the Festival Deixa a Gira Girar Exposição Olope, with a mapped projection in Vitória – Espírito Santo, with the work “Deus é Preta” and of the edital Cenas DELAS held by the Observatório de Favelas, focused on promoting artistic work produced by women. In December 2021, she exhibited her work “Negritude Viva” at Escola de Artes Visuais do Parque Laje – Rio de Janeiro. She is currently pursuing a BA in Cinema and Audiovisual at the Universidade Estatual do Sudoeste da Bahia. | @thamiresfortunato

Thamires Fortunato’s commission is forthcoming in March 2022.

Another docs-first win (didn't happen)

01:55, Wednesday, 02 2022 March UTC

I was writing some user documentation for RedirectManager just now, and wrote this sentence-and-a-bit:

If you provide a name of an existing page to create as a redirect, an error message will be shown and no redirect will be created. Similarly, if you

I was going to say something about how it's not possible to create a redirect to a page that doesn't exist. This isn't a limitation of MediaWiki, but when I was writing the RedirectManager API I thought it would be good to prevent these "dangling redirects". It wasn't until I came to write the documentation that I realised the most obvious use-case: creating a redirect to a page that you're in the process of creating! As in, while writing a new page, you want to add a shortcut to it — hardly a rare thing, I think.

This is why I really like "documentation-driven development", where one writes the docs first and pretends that they're describing features that already exist. It really does help focus the mind on what's required of the code, and (as in today's example) highlights things that might otherwise be overlooked.

So I'll now go and change the API error to a warning, and not show it at all in the UI (although it might be worth having some indication that the target page doesn't exist).

https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Extension:RedirectManager

Этот пост также доступен на русском языке.
Цей пост також доступний українською мовою.

Update: As the conflict has continued, the Wikimedia Foundation received a Russian government demand to remove content from Russian Wikipedia. You can read the latest update and our response.

The Wikimedia Foundation supports a global movement of volunteers and communities around the world. We stand with our volunteers in Ukraine, Russia, and around the world who are calling for an immediate and peaceful resolution to the conflict.

The invasion of Ukraine has resulted in the senseless loss of life and has also been accompanied by information warfare online. The spread of disinformation about the ongoing crisis affects the safety of people who depend on facts to make life-and-death decisions and interferes with everyone’s right to access open knowledge. Ukrainian volunteers have heroically continued to add content and make edits to Wikipedia and other Wikimedia projects ensuring that people everywhere continue to have access to neutral, reliable information in times of crisis, and demonstrating our shared belief in knowledge as a human right.

The Wikimedia Foundation is actively working with affected communities to identify potential threats to information on Wikimedia projects, and supporting volunteer editors and administrators who serve as a first line of defense against manipulation of facts and knowledge.

We join those calling for a peaceful end to the conflict, and will continue to support the efforts of those contributing to a strong digital commons that keeps knowledge open, neutral and free.

Telling the story of Filipinos in Alaska

17:19, Tuesday, 01 2022 March UTC
Image of Gio Renegado
Gio Renegado.
Image courtesy Gio Renegado, all rights reserved.

University of Alaska Southeast freshman Gio Renegado has always known about Wikipedia. But until he took a class with Professor Jonas Lamb, he didn’t know anyone could edit it. How did he learn this? By editing it himself, as a class assignment, with support from Wiki Education.

“I didn’t know that I was going to contribute to a free and massive library of information, he says. “I learned that there was a lot that goes into adding information to Wikipedia. There’s certain aspects when writing a Wikipedia article. There were a lot of rules and guidelines for writing on Wikipedia. For example, citations and plagiarism were taken very seriously. Misinformation can be spread across Wikipedia. But with the help of contributors and watchers, this can be prevented.”

Gio worked on two articles. One was personal: Filipinos in Alaska. As a Filipino who lives in Alaska, Gio learned about his community through working on the article.

“Contributing to this article opened my eyes on the history of my heritage and Alaska,” Gio says. “I didn’t know that Filipinos had an impact on Alaska, or the different Filipino community organizations across Alaska.”

Gio also worked on creating the start of an article on Indigenous connectivity in the United States.

“Gio’s contributions to the Filipinos in Alaska article demonstrated how the assignment enabled him to feel connected to his culture, family, community and history,” his professor, Jonas Lamb, says. “Additionally, through his work on a draft article on Indigenous Connectivity in the United States I could truly sense his increasing awareness of some equity gaps in our state (Alaska) and in society.”

Gio says he enjoyed writing for Wikipedia because it felt interactive, compared to a traditional assignment. Everything he wrote was available for the public to view — and help edit.

“My favorite part of Wikipedia is the fact that someone in the future may use my information for their own need,” he says. “For example, it helps them learn something new about a particular topic. I hope that someday someone will take my place and continue adding information to the articles I contributed to.”

This learning is exactly what Professor Lamb was hoping Gio and his classmates would get out of the Wikipedia assignment.

“Throughout the course we examined the rhetorical element of audience,” he says. “We identified and discussed the anticipated audience(s) for each piece of writing and asked how that particular audience might influence the posture taken by the writer, what information is given privilege, what is left out? Too many assignments ask undergraduates to write for a singular audience, the instructor. I wanted students to consider the impact their writing could have when shared with a broader audience while also recognizing the privileged position they occupy as college students with access to information, library collections, journal subscriptions, etc. and considering how their writing could contribute to the knowledge commons.”

To teach with Wikipedia, visit teach.wikiedu.org.

Image credit: The Alaska Landmine, CC BY 2.0, via Wikimedia Commons

In a previous blog post, I mentioned that Wikidata could help us “Find manuscripts that are in some way related to Shah Jahan (commissioned by him, owned by him, depicting him, about him),” and that its answers would be more complete as more collections shared their catalogue data. This is a follow-up with examples, going beyond manuscripts to other kind of object. As I work on enriching Wikidata’s representation of the Khalili Collections, I’m finding more and more connections to other collections around the world. This process makes those connections visible and suggests educational visualisations we can create.

It’s easy for Wikidata to generate lists of objects with a connection to a specific person. Here is the example for Shah Jahan, the 17th century Mughal emperor, which right now lists 29 objects commissioned by, formerly owned by, or depicting him. I wanted to make this kind of exploration more visually interesting, so I built on an idea written up in another previous blog post; using Wikidata’s graph tool.

Here are some objects connected to the Ottoman Sultan Abdülmecid I. Click on the image to enlarge it.

Objects created by, commissioned by, or depicting Abdülmecid I

The interactive version of the above graph lets you drag nodes and double-click on them to get more information. It will also update over time as more collection data are added to Wikidata.

In the graph above, the same image is used to represent “Abdülmecid I” and “Sultan Abdülmecid I”. This is potentially confusing, but they are not the same thing. The “Abdülmecid I” in the centre is the man, and “Sultan Abdülmecid I” is a painting that depicts him. An image of the painting is used to represent the man because that is the best visual representation Wikidata/Wikimedia have for him.

My query asks for objects in collections with a connection to the individual, then gets the collections those objects are in, then gets the type of object. By substituting one identifier in the query, we ask the same question about a different person: here, Ottoman Sultan Suleiman the Magnificent:

Objects dedicated to, commissioned by, or depicting Suleiman the Magnificent

Here’s the link to the stretchy interactive version. Seven different collections, in multiple countries, are represented here, and this is part of the excitement of exploring art on Wikidata: These graphs aren’t an ideal interface for the general public, but they expose connections that we would never find by browsing a single collection, or even a national aggregator.

Can we do better than link seven collections? The graph for Timur, founder of the Timurid Empire, links nine different collections and includes a letter he wrote alongside art works depicting him:

Objects connected to Timur

Here’s the link to the stretchy interactive version.

When more objects and collections turn up, the graph gets crowded and difficult to read, so I deactivate the code that shows the “instance of” properties. Here’s the graph for the Mughal emperor Aurangzeb:

Objects connected to Aurangzeb

Here’s the link to the interactive query.

Previously in my career I’ve added catalogue data from the Bodleian Library and the Ashmolean Museum to Wikidata and more currently I’m building a data set of works from the Khalili Collections and know people doing the same for other institutions including the Metropolitan Museum of Art, so it’s satisfying to see all this work coming together.

Let’s finally see the rather crowded graph for my initial suggestion, Shah Jahan, presently with 29 objects in ten collections:

Objects connected to Shah Jahan with the collections they are part of

This is the link to the stretchy interactive version. Note that some art works appear on these graphs as a thumbnail image (e.g. the Khalili Astrolabe at the top of the image) and some appear just as a name (e.g. the Ashmolean Museum’s objects). Images appear in Wikidata queries if they are available on Wikimedia Commons; one of many benefits of bulk-uploading images to that platform is to improve this kind of visualisation.

All these graphs are almost certainly incomplete in their coverage, and can be improved as more institutions openly share their catalogue data.


Historical people and modern collections: a Wikidata exploration was originally published in Wiki Playtime on Medium, where people are continuing the conversation by highlighting and responding to this story.

Tech/News/2022/09

11:12, Tuesday, 01 2022 March UTC

Other languages: Bahasa Indonesia, Deutsch, English, español, français, italiano, norsk bokmål, polski, português, português do Brasil, suomi, svenska, čeština, русский, українська, עברית, العربية, فارسی, বাংলা, ไทย, 中文, 日本語, ꯃꯤꯇꯩ ꯂꯣꯟ

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

  • When searching for edits by change tags, e.g. in page history or user contributions, there is now a dropdown list of possible tags. This was a request in the 2022 Community Wishlist Survey. [1]
  • Mentors using the Growth Mentor dashboard will now see newcomers assigned to them who have made at least one edit, up to 200 edits. Previously, all newcomers assigned to the mentor were visible on the dashboard, even ones without any edit or ones who made hundred of edits. Mentors can still change these values using the filters on their dashboard. Also, the last choice of filters will now be saved. [2][3]
  • The user group oversight was renamed suppress. This is for technical reasons. You may need to update any local references to the old name, e.g. gadgets, links to Special:Listusers, or uses of NUMBERINGROUP.

Problems

  • The recent change to the HTML of tracking changes pages caused some problems for screenreaders. This is being fixed. [4]

Changes later this week

  • The new version of MediaWiki will be on test wikis and MediaWiki.org from 1 March. It will be on non-Wikipedia wikis and some Wikipedias from 2 March. It will be on all wikis from 3 March (calendar).

Future changes

  • Working with templates will become easier. Several improvements are planned for March 9 on most wikis and on March 16 on English Wikipedia. The improvements include: Bracket matching, syntax highlighting colors, finding and inserting templates, and related visual editor features.
  • If you are a template developer or an interface administrator, and you are intentionally overriding or using the default CSS styles of user feedback boxes (the classes: successbox, messagebox, errorbox, warningbox), please note that these classes and associated CSS will soon be removed from MediaWiki core. This is to prevent problems when the same class-names are also used on a wiki. Please let us know by commenting at phab:T300314 if you think you might be affected.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

When Russia launched its large-scale invasion of Ukraine on 24 February, volunteer editors on Wikipedia leapt into gear to document developments as they unfolded. A look at the record of their edits provides a window behind the scenes of breaking news editing.

The coverage built on Wikipedia’s existing information on Russo-Ukrainian relations. On 21 February, as tensions rose, editors began a discussion at the talk page of the preexisting “2021–2022 Russo-Ukrainian crisis” article about whether to split off a separate article for an invasion should one occur. Seven supported, and three opposed.

Consensus model: agreeing on article names

Wikipedia uses a consensus model but gives editors wide latitude to act boldly. On 23 February at 17:12 UTC, one went ahead and created the invasion article. Over the next several hours, a brief edit war ensued as several editors alternately removed and restored the page, debating whether the discussion had established sufficient agreement.

At 03:07 UTC 24 February, minutes after Putin’s speech announcing the invasion, an editor restored the article and others immediately flocked to it. Then 1800 words long with 113 references, it consisted almost entirely of background information.

Editing & vandalism protection

Over the next 3 hours, the article was edited 178 times, once every minute on average. At 04:10 UTC, an administrator restricted edits from new users to prevent vandalism. Another administrator later raised the protection further to extended-confirmed level, allowing only editors with 500 prior edits and a month’s tenure, citing precedent from Wikipedia’s Arbitration Committee for Eastern Europe articles. Editors who cause disruption or violate Wikipedia’s commitment to neutrality are blocked.

Discussions about improving the article immediately began on its associated talk page. To date, there have been more than 350 threads with more than 500 participants, seeking consensus on issues like which parties to list as belligerents in the infobox. More than 600 editors have subscribed to track updates to the article and its talk page on their personal watchlist.

Reliable sources

Reliable sources have been crucial to the article’s development. As a tertiary work, Wikipedia’s coverage is built directly on the news reports from journalists in Ukraine risking their lives to bring us this moment in history in real time. Wikipedia does not allow citations to outlets like the state-funded Russian TV network RT, which editors have agreed by consensus is unreliable. Scholarly histories are the best sources, and someday, when those are written, they’ll be added to the article. For now, breaking news reports are the only sources available, so those are used.

Often, journalists themselves are forced to cite questionable sources, such as the casualty counts from the Ukrainian government (which has an incentive to minimize their own losses). In these cases, Wikipedia provides in-text attribution of information to help readers interpret it.

When early news reports get information wrong, Wikipedia generally does, too. For instance, journalists initially reported that 13 soldiers defending Snake Island had been killed, and the Ukrainian government confirmed, and Wikipedia mirrored that claim. When Russia disputed the account, saying they had only been captured, their perspective was added with attribution, and when Ukraine admitted the error, the article was updated again. A banner at the top of articles on current events cautions readers that “information may change rapidly as the event progresses, and initial news reports may be unreliable.”

What is the right level of detail?

As the invasion progressed, further spinoff articles were created, such as one on the international reaction (at 18:38 UTC 24 February). There are now more than 50 such articles.

One aspect of breaking news editing that can be difficult is writing at the right level of detail. Wikipedia aspires to take a long-term world-historical view, rather than indiscriminately covering every breaking development as a newspaper would. In the heat of the moment, it’s hard to tell what will or won’t become important in the long run. For instance, it is currently unknown whether the Ghost of Kyiv will ultimately become a significant part of the narrative of the invasion or just a momentary internet rumor. The natural momentum right now is toward creating many articles, but as the world gains hindsight, some will likely be merged or deleted, making the coverage easier to maintain at a high level of quality.

Freely licensed multimedia

Wikipedia editors have searched for images and videos depicting the invasion. This is difficult due to the site’s strict licensing requirements, but freely licensed videos from Voice of America, the U.S. external broadcaster, were particularly helpful early on.

Images can be vectors for misinformation, and unlike article text, they generally can’t be cited to a reliable source. Still, editors have taken steps to verify their authenticity. For instance, in deciding to use this image of a shell strike in Kyiv uploaded by a pseudonymous new editor, Wikipedians checked its metadata, which provided details on when, where, and how it was taken, and matched it with an image of the same scene taken by a professional photojournalist.

reach

The publicly available statistics on the present invasion article reveal the remarkable scale of Wikipedia collaborations. It has been edited more than 3800 times by more than 700 editors. It’s more than 12,000 words long, with more than 600 references and incoming links from 2700 other articles. Independent editions exist in nearly 100 languages. It’s been viewed more than 7.5 million times, making it currently the most popular article on the site.

Wikipedia has a long history of covering current events. The September 11 attacks that took place when the encyclopedia was only months old had a formative impact on its approach, as described by Brian Keegan in his 2019 account “An Encyclopedia with Breaking News”. Last year, Wikipedian Molly White documented the site’s coverage of the January 6 attack on the U.S. Capitol in a viral Twitter thread.

Wikipedia’s coverage of the Ukraine invasion will continue to evolve as the invasion unfolds and is ultimately written into history. The world is watching.

About the author of this post: Samuel Breslow is an American media journalist and veteran Wikipedia editor. He tweets @sdkb42.

This post is also available in English.
Этот пост также доступен на русском языке.

Фонд Вікімедіа підтримує глобальний рух волонтерів і спільнот по всьому світу. Ми підтримуємо наших волонтерів в Україні, Росії та в усьому світі, які закликають до негайного і мирного врегулювання конфлікту.

Вторгнення в Україну призвело до безглуздої загибелі людей, а також супроводжується інформаційною війною в інтернеті. Поширення дезінформації щодо актуальної ситуації напряму впливає на безпеку людей, які спираються на факти для прийняття життєво важливих рішень. Дезінформація порушує право кожного на доступ до відкритих знань. Українські волонтери героїчно продовжують додавати контент і вносити правки у Вікіпедію та інші проекти Вікімедії, гарантуючи, що люди в усьому світі матимуть доступ до нейтральної та надійної інформації в часи війни. Вони демонструють нашу спільну віру в те, що знання – це право кожної людини.

Фонд Вікімедіа активно працює зі спільнотами, на які вплинула війна, щоб виявляти потенційні загрози інформації на проектах Вікімедіа. Ми підтримуємо редакторів-волонтерів та адміністраторів, які тримають першу лінію захисту від маніпулювання фактами та знаннями.

Ми приєднуємося до тих, хто закликає до мирного завершення конфлікту, і продовжуємо підтримувати зусилля всіх, хто робить свій внесок у створення потужного спільного цифрового надбання, яке зберігає знання відкритими, нейтральними і вільними.

Outreachy report #29: February 2022

00:00, Tuesday, 01 2022 March UTC

🏆 First-time achievements I dealt with Feedback #3 completely on my own: I followed up with extension requests, processing them or arguing against them when necessary. I pinged community coordinators and mentors about non-individualized feedback and/or missing feedback. All feedback was processed by February 15. All interventions made after Feedback #2 were successful. Only 1 intern needed an extension. ✨ Team highlights We saw a decrease in extension requests in the December round: We made adjustments to the intern chat topics to talk about the ins-and-outs of working remotely.

weeklyOSM 605

10:16, Sunday, 27 2022 February UTC

15/02/2022-21/02/2022

lead picture

New responsive design for openstreetmap.de? [1] | © miche101 | map data © OpenStreetMap contributors

Mapping campaigns

  • Polish contributors have mapped over 2000 AEDs. Cristoffs proposed (pl) > en normalising the access tags applied and is waiting for feedback.
  • On Saturday 26 February the first Latin American Notathon (es) > en was held to promote map note resolution in all Latin American countries.
  • Raphaelmirc reported (pt) > de that UMBRAOSM (pt) > en will hold a mapathon on Saturday 5 March. The city of Xexéu, in the state of Pernambuco (Brazil), will be mapped.

Mapping

  • Mateusz Konieczny has proposed stopping the use of opening_hours:covid19=*.
  • On the Tagging mailing list, Dian Ågesson suggested splitting the list into multiple lists.
  • kubahahaha suggested (pl) > en distinguishing between different paid parking zones by using the colour tag.
  • The Data Working Group has blocked the Map Builder user account permanently. The profile claims to be a proxy account of Microsoft for anonymous contributions using the experimental Map Builder platform. See the block message for further details.
  • Trufi Association and Crowd2Map Tanzania held a webinar: Introduction to Mapping Public Transport.

Community

  • Users can now delete their OpenStreetMap accounts themselves without needing to contact the sysadmins (via GitHub).
  • BudgieInWA reflected on a social mapping meetup held in Claremont, Western Australia (we reported earlier), by a group of seven OSM enthusiasts. The diary entry describes and illustrates the planning and execution process, which may be useful to others planning similar events.
  • ealp recalled that 8 years ago, Valentine’s Day was celebrated in Mexico by having a meet-up to talk about OSM and HOT in Mexico City. The result was that OSM people found each other to initiate and advance the now intense and useful work in OSM through the creation of the Mexican Community.
  • We reported in December that Florian, Adrien and Donat had launched their YouTube channel ‘Tropicana’. So far two live nights have been broadcast. One about mapping toys shops for Christmas, and the other on mapping bakeries for galettes des Rois. They welcome you to join every first Wednesday of the month at 20:30 CET. Events are beginner-friendly, and the next will be on 2 March at Tropicamap (fr). Tropicamap can also be followed on Twitter.

Local chapter news

  • Have a look at the 2022 OpenStreetMap US Board Election Results!

Events

  • State of the Map 2022 will take place in Florence, Italy from 19 to 21 August as a hybrid conference. The scientific committee is looking for members to be responsible for the academic track.
  • The State of the Map Working Group is calling for logo proposals for SotM 2022, to be held in Florence, Italy.
  • Ilya Zverev pointed out (ru) > de, as a member of the organising team, that after two online events the next SotM from August 19–21, 2022 in Florence should on the one hand take place as a face-to-face event, but on the other hand should also make online participation possible.

Education

Maps

  • b-unicycling has created a uMap of vacant buildings in Kilkenny and written a blog entry about his reasons for it and the reactions she received.
  • [1] After a discussion (de) > en about redesigning the German OpenStreetMap.de website, miche101 created a showcase page (de), which includes a lot of tools and links around OpenStreetMap.

switch2OSM

  • Pedal Me, an e-cargo-bike company for student and freight transport in London, has used OSM data to argue for the superiority of cargo bikes over cars in cities.

Programming

  • The code powering the OpenStreetMap.org website has been updated to Rails 7.

Releases

  • StreetComplete users are going to have to do a manual app update starting with v41. The reason for this is a new feature: you’ll be able to measure widths and heights with your camera. This will require additional permission and as a result neither Google Play nor F-Droid will automatically update to this major version.

Did you know …

  • … Bruno Duyés’ OpenPianosMap? The goal of this project is to create an open source map of accessible pianos. Doudouosm said on Mastodon: ‘… with its companion MapContrib theme, it’s never been easier to add the ones you run across’.
  • … there is a calculator for distances and driving routes? In the bottom of the right sidebar on the website there are other localised versions of the calculator.
  • … how to properly map traffic lights?
  • … the overview of specialised online maps that use OpenStreetMap as a data source?

OSM in the media

  • Guillaume Rischard checked the claim from Lydie Polfer (Mayor of Luxembourg City) that 87% of Luxembourg’s streets are speed limited to 30 km/h. In fact only 35%, by length, of Luxembourg’s streets have speed limits of 30 km/h or less, according to OpenStreetMap data.

Other “geo” things

  • Alastair Otter presented nine essential mapping tools for journalists.
  • Google has launched (fr) > en its Detailed Street Maps feature in Brussels (Belgium), offering further details about pavements and other pedestrian-oriented features.

Upcoming Events

Where What Online When Country
Puerto López Notathon en OpenStreetMap – resolvamos notas de Latinoamérica osmcalpic 2022-02-26 flag
IJmuiden OSM Nederland bijeenkomst (online) osmcalpic 2022-02-26 flag
Bremen Bremer Mappertreffen (Online) osmcalpic 2022-02-28 flag
London Missing Maps London Mapathon osmcalpic 2022-03-01 flag
San Jose South Bay Map Night osmcalpic 2022-03-02 flag
Landau an der Isar Virtuelles Niederbayern-Treffen osmcalpic 2022-03-01 flag
Berlin OSM-Verkehrswende #33 (Online) osmcalpic 2022-03-01 flag
Stuttgart Stuttgarter Stammtisch (Online) osmcalpic 2022-03-01 flag
Olomouc březnový olomoucký mapathon osmcalpic 2022-03-03 flag
Xexéu Mapathona na Cidade Xexéu – PE -Brasil – Edifícios, Estradas, Pontos de Interesses e Área Verde osmcalpic 2022-03-05 – 2022-03-06 flag
OSM Africa March Mapathon: Map Sierra Leone osmcalpic 2022-03-05
Bogotá Distrito Capital – Municipio OpenDataDay – Mapeando el barrio Nicolás de Federmán de Bogotá, Colombia osmcalpic 2022-03-05 flag
OSMF Engineering Working Group meeting osmcalpic 2022-03-07
Meatu Map Tanzania to help protect girls from FGM for International Womens’ Day osmcalpic 2022-03-08 flag
Marburg FOSSGIS 2022 osmcalpic 2022-03-09 – 2022-03-12 flag
Michigan Michigan Meetup osmcalpic 2022-03-10 flag
München Münchner OSM-Treffen osmcalpic 2022-03-10 flag
Berlin 165. Berlin-Brandenburg OpenStreetMap Stammtisch osmcalpic 2022-03-10 flag
Lyon EPN des Rancy : Technique de cartographie et d’édition osmcalpic 2022-03-12 flag
149.Treffen des OSM-Stammtisches Bonn osmcalpic 2022-03-15
San Jose South Bay Map Night osmcalpic 2022-03-16 flag
Lüneburg Lüneburger Mappertreffen (online) osmcalpic 2022-03-15 flag
London Geomob London osmcalpic 2022-03-16 flag
Großarl 4. Virtueller OpenStreetMap Stammtisch Österreich osmcalpic 2022-03-16 flag

Note:
If you like to see your event here, please put it into the OSM calendar. Only data which is there, will appear in weeklyOSM.

This weeklyOSM was produced by Lejun, Nakaner, Nordpfeil, PierZen, SK53, Sammyhawkrad, SeverinGeo, Strubbl, TheSwavu, YoViajo, conradoos, derFred.

Offline WikiFundi closes the digital divide

07:00, Saturday, 26 2022 February UTC

The Offline editing solution is now available in Spanish, English and French

For most people reading this, being online has become an integral part of our lives. It is how we function, work, communicate, collaborate, and connect to our families, friends, communities, and colleagues. 

Yet nearly 37% of the global population is not connected at all. For most people, being offline is a mild inconvenience, but for these 2.9 billion people it is their daily reality as the digitally excluded. 

The connectivity canyon

A child in Cameroon reads WikiFundi as part of the WikiChallenge Ecoles d’Afrique annual article drive. Pic: Geugeor 

In early 2022, 95% of the world is covered by mobile broadband. However, due to the expense of data, devices, lack of skills, lack of content, etc., only 1/3rd of the global population chooses to access the internet via mobile broadband coverage. The 5% lack of coverage affects the most vulnerable in developing countries, such as nearly 30% of the rural population of Africa that has no mobile broadband coverage. 

These 2021 statistics from the ITU reveal what they term a “connectivity ‘grand canyon’” separating the digitally empowered from the digitally excluded. In 2021 it was estimated that 96% of the 2.9 billion humans who remain offline live in the developing world.

An offline solution to closing the digital divide

What if we could use an offline solution to make people comfortable with and skilled in using the digital and online world? 

In 2015, WikiFundi was designed, developed, and created to bridge this ‘connectivity grand canyon’ and provide a way for those who cannot access the internet – either permanently or temporarily – to learn key digital and content creation skills whilst creating articles about their context, surroundings and interests. Once the content is created, it can either be accessed locally or can be transferred and added to Wikimedia projects or other MediaWiki-based sites. 

The WikiFundi 1.0 editing interface with a Wikipedia offline library and hosts of training resources and materials was released in English and French in 2017. 

In January 2022, Wiki In Africa is proud to announce that WikiFundi is now available in Spanish, along with updated French and English versions. Each language version is supplemented with updated, expanded and entirely new teaching, training and reading materials and resources. 

Collaborative learning whilst doing

WikiFundi is accessible via the digital device of learners or trainees through the closed network (similar to wifi) powered by a Raspberry Pi device plugged into the mains or a battery pack. The WikiFundi interface simulates the experience of editing Wikipedia. Entire classes, groups or communities can work together whilst learning how to engage with a digital interface as they read quality articles, navigate the device and write articles collaboratively.

It’s in the name …The “Fundi” (/ˌfʊndɪ/, not /ˌfʌndɪ/) part of WikiFundi is based on a southern African word that means expert, authority or guru. It is used as an informal label, as in “you’re such a maths fundi”. The use of the term Fundi is a play on words. WikiFundi teaches you a skill, and it also gives access to knowledge. It is a clever platform that teaches you how to teach others by writing articles about things that you are passionate about.

An online future through offline training

Insufficient skills are the first culprit blamed as an impediment to meaningful connectivity. WikiFundi provides the interface, tools, and materials needed to introduce and train people in skills while they are also contributing to local or global knowledge. There are so many ways for people to engage, the limit is only the trainers’ imagination. 

Since launching in 2017, Wiki Fundi has been used in K-12 (primary (aged 6-12) and secondary (aged 13-18 years) classrooms, as part of adult-education and afterschool programs, for Wikimedia outreach and training by Wikimedia communities and digital skills acquisition and coding development by fledgling digital entrepreneurs. Its most notable impact has been through the annual WikiChallenge Ecoles d’Afrique, run annually since 2017 by Wiki In Africa in collaboration with Fondation Orange and Wikimedia Usergroups. 

WikiFundi benefits …

1. Educators teaching students (K-12, primary, and secondary), or running in school clubs, after-school or adult-education programs.

2. Wikimedia organizers and trainers in South America, Africa, or any rural situation or place that faces offline challenges (refugee camps, etc.).

3. Entrepreneurs interested in exploring and using a useful new digital tool.

The 2021 contest took place in 100 primary schools across 9 African Francophone countries. The contest resulted in 138 articles being written, the contribution of over 800 photos. Twenty primary schools won WikiChallenge prizes.

Further, in 2021 WikiFundi received the Open Asset award from the Open Education Awards for Excellence

Sounds good? Here’s how to get WikiFundi for your community

When considering installing WikiFundi, think of it more like installing a website, and not like downloading a mobile app. Like all digital interfaces, it is not a simple process. 

If you are feeling technically confident and you have a Raspberry Pi and SD card, you can download WikiFundi online from the image files available here. If you are not feeling technically savvy or wish to add offline Wikipedia, Wiktionary, or any other offline resources to your WikiFundi package then contact us directly.

WikiFundi is a Wiki in Africa project that was created and designed by Florence Devouard and Isla Haddow-Flood and developed by Florence Devouard, Emmanuel  Engelhard (Kelson), Florent Kaisser, Renaud Gaudin, and other incredible members of the Wikimedia global community. 

Since its inception, it has been funded by the Orange Foundation, Wikimedia Foundation, and Wikimedia CH. Version 3.0 was funded by Wikimedia CH and supported through an organisational grant from the Wikimedia Foundation. 

Try out WikiFundi today!

Useful links:

Diving Into Our Deployment Data

22:11, Friday, 25 2022 February UTC

If you’ve ever experienced the pride of seeing your name on MediaWiki's contributor list, you've been involved in our deployment process (whether you knew it or not).

The Wikimedia deployment process — 🚂🌈 The Train — pushed over 13,000 developer changes to production in 2021 . That's more than a change per hour for every single hour of the year—24 hours per day, seven days per week!

As you deploy more software to production, you may begin to wonder: is anything I've been working on going to be deployed this week? What's the status of production? Where can I find data about any of this?

🤔 Current train info

Bryan Davis (@bd808) created the versions toolforge tool in 2017. The versions tool is a dashboard showing the current status of Wikimedia's more than 900 wikis.

Other places to find info about the current deployment:

📈 Past train data

There's an aphorism in management: you can't manage what you can't measure. For years the train chugged along steadily, but it's only recently that we've begun to collect data on its chuggings.

The train stats project started in early 2021 and contains train data going back to March 2016.

Now we're able to talk about our deployments informed by the data. Release-Engineering-Team partnered with Research late last year to explore the data we have.

We're able to see metrics like Lead time and Cycle time

We measured product delivery lead time as the time it takes to go from code committed to code running in production.

– Accelerate (pg. 14, 15)

Our lead time — the time to go from commit in mainline to production — is always less than a week. In the scatter plots above, we can see some evidence of work-life balance: not many patches land two days before deployment — that's the weekend!

For the software delivery process, the most important global metric is cycle time. This is the time between deciding that a feature needs to be implemented and having that feature released to users.

– Continuous Delivery (pg 138)

Our cycle time — the time between a patch requesting code review and its deployment — varies. Some trains have massive outliers. In the chart above, for example, you can see one train that had a patch that was five years old!

It is now possible to see what we on Release Engineering had long suspected: the number of patches for each train has slowly been ticking up over time:

Also shown above: as the number of patches continues to rise, the number of comments per patch — that is, code-review comments per patch — has dropped.

The data also show that the average number of lines of code per patch is slightly going up:

🔥 Train derailment

The train-stats repo has data on blockers and delays. Most trains have a small number of blockers and deploy without fanfare. Other trains are plagued by problems that explode into an endless number of blockers — cascading into a series of psychological torments, haunting deployers like the train-equivalent of ringwraiths. Trainwraiths, let’s say.

The shape of the histogram of this data shows that blockers per train follows a power law — most trains have a few blockers:

Surprisingly, most of our blockers happen before we even start a train. Bugs from the previous week that we couldn't justify halting everything to fix, but need to be fixed before we lay down more code on top.

The data also let us correlate train characteristics with failure signals. Here we see that the number of patches (“patches”) per train (trending ↑) positively correlates with blockers, and lines of code review (“loc_per_train_bug”) per patch (trending ↓) negatively correlates with blockers — more patches and less code review are both correlated with more blockers:

Contrast this with Facebook's view of train risk. In a 2016 paper entitled "Development and Deployment at Facebook," Facebook's researchers documented how their Release Engineering team quantified deployment risk:

Inputs affecting the amount of oversight exercised over new code are the size of the change and the amount of discussion about it during code reviews; higher levels for either of these indicate higher risk.
– Development and Deployment at Facebook (emphasis added)

In other words, to Facebook, more code, and more discussion about code, means riskier code. Our preliminary data seem to only partially support this: more code is riskier, but more discussion seems to lessen our risk.

🧭 Explore on your own

This train data is open for anyone to explore. You can download the sqlite database that contains all train data from our gitlab repo, or play with it live on our datasette install.

There are a few Jupyter notebooks that explore the data:

An audacious dream for the future of this data is to build a model to quantify exactly how risky a patchset is. We keep data on everything from bugs to rollbacks. Perhaps in future a model will help us roll out code faster and safer.


Thanks to @Miriam, @bd808, and @brennen for reading early drafts of this post: it'd be wronger without their input 💖.

Wiki Loves Africa 2022 focuses on ‘Home’

18:03, Friday, 25 2022 February UTC

The 8th edition of Wiki Loves Africa – one of Africa’s largest photographic competitions – that takes place across Africa is open for entries. This year, professional and amateur photographers, videographers, filmmakers, audiophiles, and Commonists are invited to submit entries under the theme of Home and Habitat.

Wiki Loves Africa has challenged the distorting lens of Western media stereotypes since 2014. It encourages people across the continent to change the visual narrative, close the gaps and alter global perceptions of “Africa ” on Wikipedia (and beyond). 

“A single image holds immense power. This power has been welded with dire consequences to paint a negative ‘single story’ of Africa. That power can also be used to positive effect. Wiki  Loves Africa ensures that the myriad stories of ‘Africa’ are captured and shared. Not just with Wikipedia’s global audience but, more importantly with the people from the continent.”

Isla Haddow-Flood, Wiki Loves Africa co-creator

Focus on Home & Habitat

The focus on Home and Habitat this year is a natural progression after last year’s Health + Wellness. This year’s theme invites photographers and artists to visually explore what home means, how it reflects our identity, and how it integrates with the larger environments that we inhabit. In the era of COVID the concept of home as either a physical or emotional space, a place of permanence, safety or solace, has been both challenged and reinforced by enforced lockdowns. 

Competitors are asked to explore the many aspects of home and habitat, including, but not limited to, types and uses of residence, locations, institutions, and communities, as well as the traditional or contemporary architectural structures, aesthetic choices, design details and construction materials that make up and distinguish a house or community. Habitat is not limited to community or humans, and we are excited to receive reflections of animals at ‘home’ or in their natural environments.

a slide show of Wiki Loves Africa information in 4 languages
Wiki Loves Africa 2022 is themed ‘Home & Habitat’

A new visual narrative for Africa

Since 2014 Wiki Loves Africa has facilitated the addition of 73,000 alternative ‘African’ perspectives to the Wikipedia free licenced media library, Wikimedia Commons. Wikipedia’s more than 55 million articles (in 2021) can be accessed in over 300 languages, for free, and without advertisements, all created by volunteers.  Not every image that is entered for Wiki Loves Africa illustrates a Wikipedia article.

Only 1 in 7 distinct images submitted to the contest – just over 14% – have yet to make it onto a Wikipedia page, but those images that are on articles have been collectively viewed 942 million times. This equates to nearly 1 billion opportunities where a previously negative or biased perspective of Africa or Africans has been revised, reviewed, or altered.

“The relatively easy task of uploading images to an information mega portal, like Wikipedia, provides Africans with an opportunity to share their ‘Africa’, their identity, contemporary reality, and rich heritage. The lack of African representation online can only be changed by Africans. Wiki Loves Africa is a fun, easy way to do that,”

Florence Devouard, Wiki Loves Africa co-creator.

Wiki Loves Africa not only changes how Africa is perceived – the contest also provides a very necessary visible platform to grow the skills of African professional photographers who are eager to present their visual artistry to the world. Over 9,000 photographers from 53 countries have submitted entries throughout Wiki Loves Africa’s history. This is no mean feat in a region where access to and access for professional photographers is notoriously low. World Press Photo’s annual competition laments that 2% of its global entries are submitted from Africa.

Wiki Loves Africa is open for entries until 15 April.

Entries are accepted from anywhere on the globe but must represent ‘Africa’ and the theme of Home & Habitat. There are multiple events being hosted by 30 local volunteer Wikimedia communities across Africa. All entries must be released under a free licence and uploaded via Wikimedia Commons, the media library for Wikipedia and other Wikimedia knowledge portals.

Get involved! Here’s how …

  1. Enter photographs, videos, or audio files to the competition: Wiki Loves Africa 2022 competition portal.

2. Attend a local event or find out more about Wikimedia in your country. Here’s whats happening across Africa’s Wikimedia community (and even in Hiati!).

3. Share – let your friends and family know about the Wiki Loves Africa competition and the photos you have submitted on social media using #WikiLovesAfrica, and tag @WikiLovesAfrica in your posts.

With the Spring 2022 term underway, we’ve finally had the chance to take a deep breath and reflect on Fall 2021. At this point, it seems like an overused refrain, but we know that the pandemic has continued to pose unique challenges to higher education. It hasn’t been easy for students and faculty alike, and we just want to acknowledge these hardships and thank our program participants for their commitment and hard work in a difficult time.

The work our students did would have been impressive at any time, let alone during an unrelenting pandemic. Nearly 6,000 students from roughly 330 courses:

  • added 4.72 million words to Wikipedia.
  • cited almost 50,000 references.
  • edited more than 6,000 articles.
  • created 500 entirely new entries.

From improving articles on Molecular biotechnology to Ancient Greece, our students had a major impact on Wikipedia, but in doing so, Wikipedia also had a major impact on them.

Life skills and skills for life

We often think about the Wikipedia assignment in loftier terms. Students, with their unparalleled access to information behind paywalls for most of the population, make the world a better place by making knowledge freely available to the public. As one instructor wrote, “Wikipedia enables me and my students to indulge in our love of ideas while always challenging us to learn new things and to present scholarship to the world.” We sometimes, however, gloss over the very real and practical skills students develop while learning to contribute to Wikipedia. Indeed, the broader goals of the Wikipedia assignment go hand-in-hand with its more pragmatic outcomes.

Over 90% of instructors regularly report that their students gain critical digital literacy skills from the Wikipedia assignment. As one instructor noted, the project is a “Perfect assignment for teaching how to critically evaluate online information.” Another remarked, “Students gained much more awareness of how to verify information and work through policies.” Sourcing is at the heart of the Wikipedia assignment. Students learn how to evaluate the quality of a source and to quickly judge whether a source is reliable or not. Armed with their newly honed digital literacy skills, students are far more prepared to take on the barrage of misinformation thrown at them daily.

Digital literacy is the first step though toward the more amorphous goal of digital citizenship. 98% of instructors report that the project engenders a sense of digital citizenship in their students. As one instructor wrote, “The students came out feeling that they had contributed substantively to scholarship about the region we are studying, and they felt indeed a sense of digital citizenship.” In other words, students not only develop the ability to discern reliable information, but they see themselves as knowledge producers with a responsibility toward a greater community to ensure information they share is accurate and accessible.

Alongside media literacy, instructors report that the Wikipedia assignment also helps students to develop writing and research skills and to better master the course content. Writing for a public audience raises the stakes and often helps to make the material they’re studying take on new relevance and meaning. As one instructor described, “The Wikipedia assignment offers external reinforcement of many of the learning outcomes in the class, so students can see the real-world utility of research skills.” “Wikipedia has provided [wrote another instructor] an opportunity for students to dig into the scientific literature while also making their research widely available. A term paper is great but only the instructor sees the final product. Having student work widely available is incredibly useful for both the students and the scientific community.”

Many instructors often comment that the real pedagogical value of the Wikipedia assignment lies in the processes students undertake while learning how to contribute to the world’s largest online encyclopedia. Put simply, “Students work for themselves and learn how to learn.”

Who’s teaching who?

Nearly all of the instructors we support have no prior experience editing Wikipedia. This means that instructors are often learning along side their students as they embark on their Wikipedia adventures. “Because this was my first time teaching a Wikipedia assignment,” noted one instructor, “and because I was upfront with my students that I was learning alongside them, they appreciated seeing me as a learner as well as a teacher.”

In contributing to a public-facing site, students develop a sense of authority that comes from being able to share expertise. As one instructor remarked, the project “works as a way to transform advanced undergrads from a student mindset to an authority in the field mindset.” Often for the first time, students understand the types of issues faculty face in their own research. “I think it helped them to realize,” offered one instructor, “that the professor grapples with the same kinds of research and writing challenges that they face and that such challenges are a natural and expected part of scholarship.”

Pedagogically speaking, the project offers a rich landscape of opportunities. One instructor wrote that, “I gained a great knowledge of my students’ deficiencies in research from this project. I had made assumptions about their prior knowledge and skills that were inaccurate, and this assignment helped me diagnose the gaps in their skills and knowledge. It was not always pretty to see what I discovered, but it will be helpful to me in the future.” Another noted, “It is the main learning tool I use. The work from the textbook is less creative. The Wikipedia assignment solidifies the learning.” And yet another declared that “Wikipedia helped me improve my confidence and teaching skills!”

More simply, the Wikipedia assignment often engenders a sense of excitement and energy among students and faculty alike that leads to greater engagement. As one faculty member wrote, “It was invigorating as an instructor to talk about the democratization of knowledge with my students, and it was empowering for them to put their research and writing skills into action.” Another commented, “This assignment significantly enhances teaching and learning in the course overall by creating excitement in the students, a greater sense of belonging and viewing themselves as scientists with something to offer, a greater appreciation of the purpose of all of the course material, and a much deeper relationship between student and instructor.”

Taking pride in a job well-done

In the account of one instructor, “A senior indicated that it was the most difficult research project they ever had to do, but also the most satisfying.” This is a common refrain among faculty and students alike. The Wikipedia assignment isn’t easy. There’s a lot to learn before students are able to make their contributions, but the pride and satisfaction students feel at the end of the project is without parallel.

Students understand that the work they put on Wikipedia has real world implications, and as a result, feel more accountable for the information they put out into the world. As one instructor wrote, “Students were very engaged. In final presentations at the end of the semester, many said, “I was surprised how invested I became in this assignment.” Another explained, “Students for the most part came to see their work as meaningful beyond the classroom. They understood the concept of ‘making knowledge’ as something that actually happens more so than just an ideal.”

The pride and satisfaction students experience from seeing their work live on Wikipedia is infectious! As one instructor shared, the value of this project lies in “Witnessing students stepping up to accomplish a difficult task and sharing in their joy and celebration!” Another declared, “Don’t be afraid! I was nervous and I was awe struck by how amazing my students’ contributions were.”

Thank you again to all of the instructors and students who made Fall 2021 an outstanding term for us at Wiki Education and for helping to make the world a better place during challenging times!

Small commits

16:04, Thursday, 24 2022 February UTC

There are many blog posts and articles out there about making small git commits. I’m sure most people (including me) bring up the same few topics around why small commits are good and why we should all probably be making smaller commits.

In this post, I’ll look at some of the key topics from my perspective, and try to tie these topics to concrete examples from repositories that I have worked on. The topics are in no particular order, so be sure to give them all a read.

One thing to note is that “small” doesn’t necessarily mean small in terms of lines of code. Small here is also relative. Also, small commits can benefit you in many different places, but to stand the test of time they must end up in your main branch.

Git features during development

Git only takes full responsibility for your data when you commit

Commit Often, Perfect Later, Publish Once: Git Best Practices

You can’t take advantage of any of the features that git has to offer unless you make commits. So commit early, and commit often, even if you later squash your changes either to submit them to a single commit based review system such as Gerrit or aim to squash them when merging a pull request.

Some IDEs do retain a history of individual file changes that can help you in some regards here, such as Local History which is part of IntelliJ. But git of course offers similar functionality IF you make commits.

Git commits will enable you to track named versions of your changes, as well as go back and follow your own thought process. You can even share this thought process (the commits) with others. Multiple commits will also allow you to take advantage of things such as revert and diff for parts of your change.

Take the example below, which includes 8 commits, but ultimately only ends up adding ~120 lines of code. The set of commits is a good example and communication tool showing how I personally tackled the problem that I was looking at, even if the resulting value is the ~120 lines of code.

Separation of concerns

Separation of concerns is primarily discussed as a design principle in software development, where generally speaking code should do one thing, and do it well.

All of the benefits that can be seen with code implementations can also be seen with git commits when those commits are small, targetted and deliberate, atomic if you like, just trying to do a single thing. The points below were based on a separation of concerns post by nalexn on Github.

  • More clarity: It’s much easier to understand what is going on in commits, when each commit just does one thing.
  • Better reusability: When if work in situations where you may need to apply a change to multiple branches, or package it up as a patch file, it’s easier to extract what you want with small distinct commits.
  • Better testability: It’s easier to see what behaviours small changes have had by looking at small test changes. Or even be able to see that a particular change is untested, or results in no behaviour change. Tests and continious integration workflows can also be run only for the small commit, rather than many things together.
  • Faster evolution: It’s much easier to get small changes merged into the code base and thus propell the code base forwards. If you tie changes together they may get blocked things that are totally unrelated, and thus not make it into the code base quickly, meaning the code base can’t benefit from the change.
  • It is easier to have simultaneous development by multiple engineers. (There are many sub topics here)

Splitting a change which does many things into smaller changes which each do only one thing can decrease the total complexity associated with accomplishing the same goal.

Writing Reviewable Code

An example of clarity and faster evolution from my work today would be this 5 line change altering some code that is used during the current Wikibase release process. I found that I wanted to make this change while rewriting documentation in another pull request which adds 100 lines and removes 41, however, I created a separate small commit, publishing it for review separately.

At the time of writing this, the small fix is already merged, and the documentation change is still pending. If I had submitted them together neither would have made their way into the codebase yet.

Reusability of commits, testability and clarity really shines through when it comes to maintaining multiple releases of software. For Wikibase we maintain multiple release branches alongside our main branch and often want to backport fixes and small improvements that have been made by developers in the main branch.

An example would be this change that alters how a single SQL query is formed (a small +3 -1 line change). This change appears to have been made alongside a larger change to the code, which is also generally doing other things. As these are separate commits we can see what is happening, the impact that it should have, the fact that the tests all pass in the commits standalone form and we can easily cherry-pick the change onto the release branch, which was done.

Easier review

Your reviewers’ number one challenge is to understand what you’re doing and why, and the way to help someone understand is to manage their cognitive load.

Cracking the code review (Part 2): Make them seem small

Keeping the cognitive load low during code review is important to increase the quality of the review, and also get through the review process faster, thus merging the code faster. You want to avoid reviewers’ minds wandering, and you can do this by keeping things small, and focused, particularly keeping refactorings and behavioural changes separately. The post I quoted above comes in 2 parts (part 1 & part 2) and I highly recommend reading both.

The example of small commits making easy review that I bring is this change of 9 commits that were reviewed and merged into Wikibase in 2020. You can see the relation chain in the screenshot below in the top right.

The first 8 changes here touched a total of 1,600 lines of code. The commits were concise and stated clearly what they were doing, and each commit only did that one thing. For example, remove class X or deprecate thing Y.

The final change still touched around 4,000 lines of code, but everything was much easier for the reviewers to digest, due to small/atomic focused commits. Having the commits combined would have given reviewers more things to think about at once, prolonged review and prevented merging early.

The example may be thousands of lines of code, but the same principles apply with 10 lines of code.

Conflicts, Rebasing & Reverts

The bigger a patch is, the longer review takes, and the more people that are working on any given codebase, the more likely you are to have merge conflicts. Merge conflicts in turn prolong the time before code can be merged, introduce a new round of review and as the size of a change increases the time to resolve conflicts probably increases non-linearly.

So avoid making changes that could be split out into smaller commits together. If you notice some unrelated documentation that has a typo, or some indenting that is wrong, a small bug that could be fixed, or some optimization that could be made, do it in a separate commit!

Conflicts don’t only happen during the review cycle, but also if reverts are needed for any reason, such as a bug in a change that is only found after merging to main has already happened. If a commit is small it will likely be easy to revert, but if it is doing many things as time progresses the revert will become harder, need more manual intervention and likely lead to more issues.

This is also a reason to do cleanup and refactorings around a change before and not after making behaviour changes. If you need to revert a behaviour change that has already been refactored, you’ll likely also need to revert the refactoring. If refactoring came first you can simply revert the behaviour change.

The example I will use while looking at reverts will be a rather large change to MediaWiki core back in 2015 introducing a new feature called catwatch.

The original patch “Enable users to watch category membership changes” (+614 -57) needed to be reverted 6 days later. The revert had a few conflicts that needed to be manually resolved (an opportunity to introduce more errors) before it could be merged.

This patch was later re-applied in a few smaller commits, aiding in review, and also, if needed in the future aiding in future reverts. The patches were:

Though not a perfect example, in the case of a future revert, the behavioural change was kept to the third commit, and only this one commit would need to be reverted if there was an issue.

Teamwork & Future you

The commits that end up being merged into a repository are only kept around to help the group of people working on a codebase, and that includes future you. A lot of other git principles come into play here, such as writing meaningful commit messages.

Other folks that were not involved in any code review process, or in writing code that has been merged start with a disadvantage in terms of understanding what has happened and why. This should be apparent from the commit and the commit message, and things should not be hidden beneath the surface. Commits are guaranteed to stand the course of time in your repository, code review and external discussions not so much.

Even if you wrote the code, if you have a reason to look back through weeks, months or even years of code, then the footprints that you leave in the form of commits end up being the main useful signposts. You’ll primarily be looking at the first line of commit messages, and hoping that this message represents what the commit is actually doing.

I can’t find a concrete example link for this, but imagine that you write a commit called “implement feature X”, but this commit also fixes bug Y. If you are looking at commits, you won’t see this hidden change.

The post Small commits appeared first on addshore.

Wikisource Triage meetings

11:50, Thursday, 24 2022 February UTC

In a bit over three weeks' time we're going to have the first of what will hopefully become a series of Wikisource "triage meetings", in which we'll go through the backlog of Phabricator tickets relating to Wikisource tech. It's basically an idea to get some clarity around what needs doing, what is being worked on, and probably what things are out of date and can be closed. Read more and sign up here: https://meta.wikimedia.org/wiki/Wikisource_Triage_meetings

Wikisource technical contributors are not vast in number, but there are still quite a few of us! So hopefully through some judicious collaboration we can continue to slowly and steadily (and without too much disruption!) improve Wikisources' software.

Robert Harper Building

06:47, Thursday, 24 2022 February UTC

I'm working from a new office now, and just uploaded a 1950 photo of it to Commons: https://commons.wikimedia.org/wiki/File:Robert_Harper_building,_1950,_slwa-b2384977-1_(cropped).png (at some point I guess I'll be able to embed photos in the text of my blog…).

Emma Schell headshot
Emma Schell
Image courtesy Emma Schell, all rights reserved.

In the spring 2020 term, College of Wooster student Emma Schell took her first college-level history class, with Professor Katie Holt. The class was about modern Latin America and it included the assignment to improve Wikipedia’s coverage of Latin American topics, through Wiki Education’s Wikipedia Student Program.

“I was nervous when I learned I was going to be writing for Wikipedia as a part of my first class with Professor Holt,” Emma says. “Since I had little experience with college-level history, I was concerned about my ability to informatively write about history on a public website.”

Emma worked on two articles: one on Frida Kahlo’s painting The Two Fridas and another on Afro-Cuban artist Wifredo Lam. She chose these articles because of her interest in visual art, and particularly in multi-racial artists.

In the fall 2021 term, Emma signed up for another of Dr. Holt’s classes, Latin American Revolutions. Once in class, she discovered she’d be writing a Wikipedia article again. This time, she chose a topic from a list curated by her instructor: the Tzʼutujil people, an Indigenous group in Guatemala.

“I chose to research this article because I feel that Indigenous people are underrepresented in the media, even though they make up a significant portion of the Latin American population,” Emma explains. “I wanted to help to close this gap in representation and available knowledge by writing about multiple aspects of the Tz’utujil people, such as their history, indigenous religion, and how they were impacted by the Guatemalan Genocide of the late twentieth century.”

Emma says she enjoyed working on the articles. Writing for Wikipedia helped her develop skills in non-argumentative communication and digital media. She likes working on Wikipedia articles that are underdeveloped — “stubs”, in Wikipedia parlance — because she could contribute in a meaningful way, adding factual information.

“My favorite part about writing for Wikipedia was writing factually,” she says. “Although this was hard to do sometimes, especially for interpreting artworks, I feel like it improved my academic communication skills, which is useful as someone who hopes to pursue scientific research after graduating college.”

Emma, a chemistry major minoring in Spanish and Latin American Studies, says she’d be open to adding more information to the articles she worked on. She values the assignment not only for the research she did, but also for what she gained from reading classmates’ Wikipedia contributions.

“I appreciate that Professor Holt encouraged my class to research Latin American people, groups, and events that have not been highly represented in the classroom or media,” Emma says. “Researching for my project and reading other students’ Wikipedia edits helped to broaden my knowledge of diverse perspectives in Latin America.”

For more information on teaching with Wikipedia, see teach.wikiedu.org.

Image credit: Rod Waddington from Kergunyah, Australia, CC BY-SA 2.0, via Wikimedia Commons