Greg Ross 6:17 am on July 12, 2017

GlotPress and Locale Variants

An often requested feature in GlotPress has been to be able to support locale “variants”. This has been tracked as issue #226 on GitHub and has been open since January 2016.

Locale variants allows for a “root” language to have one or more “variants” of the language in which the variants automatically inherits the translations of the root for any missing translations.

This is useful for many popular languages, for instance English, Spanish, French, etc. where the vast majority of the translations are likely to be identical with minimal differences for local dialects.

PR #747 has recently been created with a solution for locale variants.

Ok so what does it look like?

The first thing to note that all of the below is based on the current PR, which may change by the time it is actually merged in to GlotPress.

Let’s say you have a project called “Labels”, with three locale’s defined:

English (Unites States)
Spanish (Spain)
English (Canada)

One of these things isn’t like the others, but two of them are very close to being the same.

In the above situation, English (Canada) would inherit the translations form English (United States) for any translations that were not defined.

Going to the translation set page you would see something like this:

You’ll notice a new entry in the legend at the bottom of the page, “Root translation”, this indicates any translation shaded in this color (slightly darker green) is inherited from the root language instead of being marked as untranslated (white).

If you select one of these translations to edit you will see the following editor displayed:

Notice the addition of the “Root Translation” information under the Meta heading along with a link back to the root translation.

If you change the translation here and save it, a new translation entry for the locale will be created and the root translation will no longer be inherited.

Looks good so far, so now what?

The PR is mostly complete, however we are looking for some input on one part of the system; the relationship between root and variant languages.

As part of the PR an initial pass at defining the relationships between languages has been made but we are looking for input from the community to ensure they are correct and for any missing relationship that make sense.

One item to note is that root to variant relationships can only go one level deep, so you cannot define a variant of one root as the root of another one. So in our English example, you cannot do English (United States) -> English (Great Britain) -> English (Canada). This was done on purpose to ensure translations did not degrade in quality as they moved farther away from the original source language.

The current root/variant relationships in the PR (see the locales.php in the PR for full details) are as follows (the root is the top level name and the variants are in the bullet list):

Arabic

Algerian Arabic
Moroccan Arabic

Azerbaijani

South Azerbaijani
Azerbaijani (Turkey)

German

German (Switzerland)
Swiss German

Greek

Greek (Polytonic)

English (Unites States)

English (Australia)
English (Canada)
English (UK)
English (New Zealand)
English (South Africa)

Spanish (Spain)

Spanish (Argentina)
Spanish (Chile)
Spanish (Colombia)
Spanish (Costa Rica)
Spanish (Guatemala)
Spanish (Mexico)
Spanish (Peru)
Spanish (Puerto Rico)
Spanish (Venezuela)

Persian

Persian (Afghanistan)

French (France)

French (Belgium)
French (Canada)
French (Switzerland)

Dutch

Dutch (Belgium)

Norwegian

Norwegian (Nynorsk)

Portuguese (Portugal)

Portuguese (Brazil)

Tamil

Tamil (Sri Lanka)

Chinese

Chinese (China)
Chinese (Hong Kong)
Chinese (Singapore)
Chinese (Taiwan)

A brief note about the slightly strange English root/variants, English (United States) was chosen as the root due to the vast majority of code that is written with it as the language of the source code. The root/variants relationships are assigned during the load of the locales file and can be altered for individual sites either through the use of a custom locales.php (using the GP_LOCALES_PATH define) file or through a plugin that hooks in to the GP_INIT hook.

Feedback on the above relationships (or any other part of the PR) is most welcome as always, you can visit the GitHub Issue or PR to participate.

Pascal Birchler 11:13 am on July 12, 2017

That’s great news, Greg! I didn’t know that #747 was already that advanced!

German (Switzerland) as a variant of German definitely makes sense, but Swiss German (gsw) doesn’t. I just left a comment on the PR about that. Hope you’ll consider removing that.

Cheers!
Pedro Mendonça 11:22 am on July 12, 2017

Hi,

First of all, I’m very glad that real Variants with fallback are coming to see the light of day.

Second, as pt_PT Translation Editor, I must say that the proposed relationship root/variant between pt_PT/pt_BR doesn’t make any sense. These are completely different locales, with independent guidelines and translation teams, and I believe that a fallback between it’s not the apropriate scenario.

I believe that this will probably happen with other root/variant relations proposed, like spannish speaking locales.
I think the Variants are much more a feature inside the same locale than between locales.

As discussed in WCEU, Polyglots, etc, we do need variants for pt_PT locale, but all pt_PT variants, not related with any other locale. Example: The AO90 Variant already requested in this post.

Example for pt_PT:
(root) pt_PT default
—-(variant) pt_PT AO90

At last, I think the color difference between root/variant isn’t very much clear, I suggest it should be much more different so there is no doubt about it.

Thanks!
Vitor Madeira 2:21 pm on July 12, 2017

One question: All strings on the “variant” would be open for translation, independently of what the “parent” locale strings might contain, or am I getting this wrong?

If a user translates the strings on the “variant”, then those strings would not inherit any kind of future translations on the “parent”, is this right?

If this would be the case, then I can only see that this feature would help getting a bit closer the people who speak the same language but are from different countries, and speed up plugins, themes and all other WordPress translations.
- Greg Ross 4:13 pm on July 12, 2017
  
  Correct, all strings in the variant can be uniquely translated independent of the root translation. Once a unique translation for the variant is saved, it will no longer inherit the root’s translation of that string.
  - Vitor Madeira 4:18 pm on July 12, 2017
    
    Great! I have no doubt that such feature would suite either Portuguese and Brazilian translator folks (as much as some people dislikes this opinion).
    
    That would mean concentrating efforts from both teams and not keeping things as we have nowadays: each one on it’s own.
Vitor Madeira 3:09 pm on July 12, 2017

As a Portuguese native speaker and a frequent Wikipedia editor, this move reminds me of what is going on with the Wikipedia editing features, as in the Portuguese Wikipedia, it is now possible to have a kind of “common root” text and have something one could call a “fallback” when the user switches to the locale she/he wants to. I wonder if that concept might be somehow inspirational for translating projects such as WordPress, who knows?

I know that actually there are many differences between the way that Portuguese and Brazilian teams decide to translate the original American English strings into their locales, but I suppose that in some near future it would be great to see a kind of approach on both sides of the Atlantic and try to get Portugal and Brazil translations a bit closer – even considering that many differences would have to remain, as there are some real differences between both ways of speaking and writing the same language that must remain different.

I know that many Portuguese and Brazilian users do not think like this, but I really wanted to leave this statement, as after all, the Portuguese language is the same, and I totally presume that teams could work on a common base and then local teams could modify the most important details that would need more attention. (kind of what en_GB already has to do when they “translate” their strings from American English.)

As for the colours, yes I too would like to remark that it needs attention there.

As for the existing translations on pt_PT (Portugal) and pt_BR (Brazil) it is quite impressing that if you open two windows side by side of both pt_PT and pt_BR translations, you will quickly see by yourself that in most translations strings to not differ that much from each other. Sometimes, the use of different words are simply and plain synonyms that a given translator decided to use in the time she/he made the translation.

It’s as easy as opening, for instance:
https://translate.wordpress.org/projects/wp/dev/pt-br/default
and
https://translate.wordpress.org/projects/wp/dev/pt/default
And compare by yourself, page, after page.

The slight differences are quite minimal and would totally be under the idea that is described on this ticket. The idea, if I understand well, would be to fall back only the non translated strings on the “variant”, right?

Correct me if I’m wrong, but if a string on the “parent” locale could be identical on the “variant” locale, nothing would have to be done, but if the “variant” users need to change given strings (or even all the strings on the whole given project) then those strings/whole project would overwrite and/or not inherit the “parent” locale strings, right?

No single doubt that this concept would be a tremendous help for everyone who translates WordPress from English into Portuguese, no matter what people might think about the differences between Brazil’s and Portugal’s Portuguese.

I totally understand that GTEs can have a final word on this matter, but for the sake of truth, this would work on more that some 75 to 80% of strings. See for yourself on the links I left above.

I understand that in the specific case of this ticket, this feature would mostly benefit the users of the “variant” locale, but I see that as a major giving from whoever works on the first strings that would then serve as “gift” to the “variant” users.

Variant users would only have to change the strings that would need ‘retouch’, right?

Now the other issue would be to convince Brazilian and Portuguese translators to work a bit more together, side by side either using Slack and/or P2/O2, etc., but (again, for the sake of truth) that is something that I don’t see much of both users trying to make efforts to work together. As a manner of fact, I’ve seen quite the opposite and that is quite a sad, very sad aspect to see.

I’m only one person and understand that those things are decided by the majority but at least this statement had to be left here, for future generations of WordPress / GlotPress users to read.

As a manner of fact, if this concept (on the “parent” strings) could stay off the nationality / country aspect and then create the variants for the needed nationality / country locales, I guess that WordPress would become one of the best examples of democratic translation around the IT communities in the world.

I mean, instead of considering “national” locales as parents, you could just consider the language name as parent and then, all national/county locales as variants.

In Portuguese, it would be something like this:
PORTUGUESE (parent)
pt_PT (variant)
pt_BR (variant)
(etc.)

For English, it would be something like this:
ENGLISH (parent)
en_AU (variant)
en_BZ (variant)
en_CA (variant)
en_CB (variant)
en_GB (variant)
en_IE (variant)
en_JM (variant)
en_NZ (variant)
en_PH (variant)
en_TT (variant)
en_ZA (variant)
en_ZW (variant)

So anyone wanting to translate a plugin / theme / whatever on WordPress.org could contribute on the ‘parent’ strings and then, local teams would approve / modify the variant strings.

For all other languages, you could set it according to the ISO language coding table:
http://www.lingoes.net/en/translator/langcode.htm
Gary Jones 3:17 pm on July 12, 2017

Looks promising.

Question regarding percentages for completeness – do the strings that are currently marked as inherited from the parent, count towards the percentage complete? i.e. a new plugin is added to translate.wordpress.org, with the default English (United States). Would that mean it gets automatically set at 100% complete for all of the English variants?

Is there any way for English GTEs to filter / view the inherited strings, in the same way that Waiting strings are flagged up? If not, it means that all of a sudden, all of the plugins will be marked as 100%, without us having implicitly checked the strings. It would be good to be able to view the inherited strings, and then bulk apply what would be Copy from Original, so that it confirms that they have been checked, even though that won’t affect the percentage complete (and everyone can pull down en_ in the meantime anyway).

Also, in terms of accessibility, we need to move away from just distinguishing information with colour – those colours should have different patterns on them. No doubt a different ticket, but the “slightly darker green” doesn’t improve this issue in the slightest and actually makes it worse.
- Greg Ross 4:17 pm on July 12, 2017
  
  That is still an open question, at the moment, root translations are not counted towards completness at the moment, but if they should be or if there should be a separate column for it has yet to be determined.
  
  For the accessibility issue, I do agree the colour highlights need some work.