666: Add CLDR support #784

toolstack · 2017-07-07T17:32:11Z

Resolves #666.

toolstack · 2017-07-07T17:34:56Z

A couple of notes on the current state of this:

I have only added CLDR plurals support to the android format, the Apple strings format remains to be done.
The test cases haven't been updated to do importing of CLDR plurals yet and the export test case is a bit of a fudge ;)
~~There's probably a few more docblocks to be written.~~

yoavf

Really happy to see this PR. Left a few comments on a few very minor issues spotted while taking a quick look

yoavf · 2017-07-10T14:44:23Z

bin/locale-generator/locale-generator.php

+/*
+ * Get the CLDR Data.
+ *
+ * By defualt it is read from the plurals.json file in the current directory, you can download


typo defualt

yoavf · 2017-07-10T14:44:23Z

bin/locale-generator/locale-generator.php

+// Create a working locales object.
+$locales = new GP_Locales;
+
+// Run through the locales and see if we can find a mathcing CLDR locale.


Typo in mathcing

yoavf · 2017-07-10T14:44:23Z

tests/phpunit/testcases/tests_formats/test_format_android.php

@@ -24,11 +27,22 @@ function test_export() {
 		$entries_for_export = array();

 		foreach( $this->entries as $sample ) {
-			list( $context, $original, $translation ) = $sample;
+			list( $context, $original, $translation, $comment ) = $sample;


Is the addition of the comment intentional in this patch?

Yes, but only because it fixes an existing bug in the test case.

ocean90 · 2017-10-03T16:08:37Z

gp-includes/template.php

+ *
+ * @return string HTML markup for a select element.
+ */
+function gp_plurals_dropdown( $name_and_id, $selected_plural_type = null, $attrs = array() ) {


Let's name this gp_plural_types_dropdown().

ocean90 · 2017-10-03T16:08:37Z

gp-templates/project-form.php

@@ -12,6 +12,9 @@
 	<dt><label for="project[description]"><?php _e( 'Description', 'glotpress' ); ?></label> <span class="ternary"><?php _e( 'can include HTML', 'glotpress' ); ?></span></dt>
 	<dd><textarea name="project[description]" rows="4" cols="40" id="project[description]"><?php echo esc_html( $project->description ); ?></textarea></dd>

+	<dt><label for="project[plurals_type]"><?php _e( 'Plurals Type', 'glotpress' ); ?></label></dt>
+	<dd><?php echo gp_plurals_dropdown( 'project[plurals_type]', $project->plurals_type, array() ); // WPCS: XSS ok. ?></dd>


gp_plurals_dropdown() should be added to customAutoEscapedFunctions in phpcs.xml.

ocean90 · 2017-10-03T16:08:37Z

bin/locale-generator/locale-generator.php

+		} // End foreach().
+
+		// Add some space between locales for easier reading.
+		echo "\n"; // WPCS: XSS ok.


Can we ignore the variants part in this PR for now?

Not easily without breaking one or both of the patches :)

They both do a lot of work on the locales file so separating them is hard.

ocean90 · 2017-10-03T16:08:37Z

bin/locale-generator/locale-generator.php

+					}
+				}
+			}
+		} // End foreach().


These are no longer required and can be removed.

ocean90 · 2017-10-03T16:08:37Z

bin/locale-generator/locale-generator.php

+		$root_var_name = str_replace( '-', '_', $locale->variant_root );
+
+		// Output the first line to 'create' the GP_Locale object for this locale.
+		echo "\t\t\${$var_name} = new GP_Locale();\n"; // WPCS: XSS ok.


Remove all // WPCS: XSS ok. comments and add the bin directory as a exclude-pattern to phpcs.xml.

ocean90 · 2017-10-03T16:08:37Z

bin/locale-generator/locale-generator.php

+					}
+				}
+			}
+		} // End foreach().


These are no longer required and can be removed.

ocean90 · 2018-04-08T14:52:36Z

gp-templates/project-form.php

@@ -12,6 +12,9 @@
 	<dt><label for="project[description]"><?php _e( 'Description', 'glotpress' ); ?></label> <span class="ternary"><?php _e( 'can include HTML', 'glotpress' ); ?></span></dt>
 	<dd><textarea name="project[description]" rows="4" cols="40" id="project[description]"><?php echo esc_html( $project->description ); ?></textarea></dd>

+	<dt><label for="project[plurals_type]"><?php _e( 'Plurals Type', 'glotpress' ); ?></label></dt>
+	<dd><?php echo gp_plural_types_dropdown( 'project[plurals_type]', $project->plurals_type, array() ); // WPCS: XSS ok. ?></dd>


The // WPCS: XSS ok seems to be obsolete.

ocean90

The current locale alignment doesn't match our code styling. Example:

<?php
$av = new GP_Locale();
$av->english_name                         = 'Avaric';
$av->native_name                          = 'авар мацӀ';
$av->text_direction                       = 'ltr';
$av->lang_code_iso_639_1                  = 'av';
$av->lang_code_iso_639_2                  = 'ava';
$av->slug                                 = 'av';
$av->nplurals                             = '2';
$av->plural_expression                    = 'n != 1';

produces:

-----------------------------------------------------------------------------------------------------------------
FOUND 0 ERRORS AND 9 WARNINGS AFFECTING 9 LINES
-----------------------------------------------------------------------------------------------------------------
  2 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 22 spaces but found 1 space
  3 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 8 spaces but found 25 spaces
  4 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 9 spaces but found 26 spaces
  5 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 6 spaces but found 23 spaces
  6 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 1 space but found 18 spaces
  7 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 1 space but found 18 spaces
  8 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 16 spaces but found 33 spaces
  9 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 12 spaces but found 29 spaces
 10 | WARNING | [x] Equals sign not aligned with surrounding assignments; expected 3 spaces but found 20 spaces
-----------------------------------------------------------------------------------------------------------------
PHPCBF CAN FIX THE 9 MARKED SNIFF VIOLATIONS AUTOMATICALLY
-----------------------------------------------------------------------------------------------------------------

It needs to be:

$av                      = new GP_Locale();
$av->english_name        = 'Avaric';
$av->native_name         = 'авар мацӀ';
$av->text_direction      = 'ltr';
$av->lang_code_iso_639_1 = 'av';
$av->lang_code_iso_639_2 = 'ava';
$av->slug                = 'av';
$av->nplurals            = '2';
$av->plural_expression   = 'n != 1';

ocean90 · 2018-04-08T15:09:27Z

locales/locales.php

+		$af->cldr_code                            = 'af';
+		$af->cldr_nplurals                        = '2';
+		$af->cldr_plural_expressions['one']       = 'n = 1 @integer 1 @decimal 1.0, 1.00, 1.000, 1.0000';
+		$af->cldr_plural_expressions['other']     = ' @integer 0, 2~16, 100, 1000, 10000, 100000, 1000000, … @decimal 0.0~0.9, 1.1~1.6, 10.0, 100.0, 1000.0, 10000.0, 100000.0, 1000000.0, …';


Is the leading whitespace required?

I don't think so but it's in the original JSON file from the CLDR repo so I think it's safer to leave in.

GaryJones · 2018-04-09T07:15:41Z

gp-includes/formats/format-resx.php

@@ -162,6 +274,11 @@ private function add_schema_info() {
 		$this->line('-->', 1 );
 	}

+	/**
+	 * Cteate the .resx schema declaration.  .resx file have a standard scheam declaration.


Several typos here.

GaryJones · 2018-04-09T07:16:04Z

gp-includes/formats/format-resx.php

 	private function escape( $string ) {
 		$string = str_replace( array( '&', '<' ), array( '&amp;', '&lt;' ), $string );
 		return $string;
 	}

-
+	/**
+	 * Cteate the .resx schema info.  .resx file have a standard scheam block.


Several typos here.

GaryJones · 2018-04-09T08:11:06Z

phpcs.xml

+		in the current version of phpcs (see https://github.com/squizlabs/PHP_CodeSniffer/issues/1731)
+		which breaks this when a single file name is passed to be checked (for example with TravisCI).
+
+		A fix has been committed and will be included in phpcs version 3.2, however this is no release


PHPCS 3.2 was released in December 2017, and the latest release is 3.2.3, with 3.3.0 not too far away. It would be reasonable to make ^3.2 the minimum version of PHPCS required for devs to contribute to GlotPress.

This should have automatically taken care of itself, we just have to remove the extra lines now as they were only needed before 3.2 was released.

…als script.

And add a minor comment to the locale generator.

Rename dropdown function and PHP CS rule additions. Fix error.

Import not yet implemented.

Now appends the code around the locales list to make life easier.

To create no-variants version of the locales file and also write both out to files instead of the console. Also make a backup just in case before deleting the old file.

toolstack requested a review from ocean90 on Jul 7, 2017

toolstack force-pushed the 666-cldr-support branch from 451a901 to f3492ba on Jul 7, 2017

yoavf reviewed on Jul 10, 2017

View changes

toolstack mentioned this pull request on Jul 10, 2017

666: Android Plurals Support #667

Closed

toolstack added this to the 3.0 milestone on Jul 11, 2017

akirk mentioned this pull request on Aug 30, 2017

666: Add Android Plurals support #692

Closed

toolstack force-pushed the 666-cldr-support branch from 16af3e9 to df61191 on Aug 31, 2017

toolstack force-pushed the 666-cldr-support branch from 6cfe241 to 8890fcf on Oct 2, 2017

ocean90 requested changes on Oct 3, 2017

View changes

toolstack force-pushed the 666-cldr-support branch 2 times, most recently from 2a41a26 to 5a3ec10 on Oct 9, 2017

toolstack force-pushed the 666-cldr-support branch 4 times, most recently from 44f8127 to e171d3a on Oct 17, 2017

toolstack force-pushed the 666-cldr-support branch from 4094482 to e1860d8 on Nov 23, 2017

toolstack mentioned this pull request on Nov 27, 2017

Allow plugins to control the number of translations entered #855

Open

ocean90 reviewed on Apr 8, 2018

View changes

ocean90 requested changes on Apr 8, 2018

View changes

ocean90 reviewed on Apr 8, 2018

View changes

GaryJones reviewed on Apr 9, 2018

View changes

toolstack force-pushed the 666-cldr-support branch from 2ad7e51 to db99fc2 on May 17, 2018

toolstack force-pushed the 666-cldr-support branch 3 times, most recently from dcbfaf9 to 83b45fe on Oct 30, 2018

toolstack force-pushed the 666-cldr-support branch from 192f0bc to b0530f1 on Jan 2, 2019

Add CLDR plurals to the locales, formats and projects.

18a82e4

toolstack added 26 commits on Jul 7, 2017

Update the CLDR plurals script.

341c054

Update the locale.php file based on the output from the new CLDR plur…

26e551c

…als script.

Make sure the CLDR number list honours the $how_many parameter.

28f98c4

Fix the android export test case.

df667bd

And add a minor comment to the locale generator.

Add docblocks and comments to locales.php.

d159e18

Coding standard updates.

479fafa

Add plurals type to ngx format.

c5ef05f

Add downloading of the CLDR plurals file if a local copy does not exist.

582275f

Add columns to the locales page to indicate plurals support.

8942d1a

Coding standard updates.

3d4aae3

Various updates.

b7a383c

Rename dropdown function and PHP CS rule additions. Fix error.

Update phpcs exclusion.

494ec44

Fix inverted logic check.

8001bf1

Add file format definition link to comments.

8e75433

First cut at stringdict support.

7f2e593

Import not yet implemented.

Coding standard updates.

c1433d0

Add padding to locale object creation lines.

8f6edfa

Update locale file.

6cd992d

Fixed typo's in comment.

4601f1f

Regerenate locales file.

0455636

Update locale generation script

6aa918c

Now appends the code around the locales list to make life easier.

Coding standard updates.

d9c73fb

Bump DB version after notes merge.

ffb0b0c

Update android test cases.

890fe40

Update locale generator.

d80ce09

To create no-variants version of the locales file and also write both out to files instead of the console. Also make a backup just in case before deleting the old file.

Coding standard updates.

e7c0296

toolstack force-pushed the 666-cldr-support branch 2 times, most recently from 7264bcb to e7c0296 on Jan 3, 2019

ocean90 added [Type] Locales [Type] Build Tooling labels on Apr 13, 2020

GlotPress / GlotPress-WP

666: Add CLDR support #784

666: Add CLDR support #784

toolstack commented on Jul 7, 2017

toolstack commented on Jul 7, 2017 •

edited

yoavf left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ocean90 left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

GlotPress / GlotPress-WP

666: Add CLDR support #784

Are you sure you want to change the base?

666: Add CLDR support #784

Conversation

toolstack commented on Jul 7, 2017

toolstack commented on Jul 7, 2017 • edited

yoavf left a comment

This comment has been minimized.

yoavf on Jul 10, 2017 Member

This comment has been minimized.

yoavf on Jul 10, 2017 Member

This comment has been minimized.

yoavf on Jul 10, 2017 Member

This comment has been minimized.

toolstack on Jul 10, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Oct 3, 2017 Member

This comment has been minimized.

toolstack on Oct 3, 2017 Author Contributor

This comment has been minimized.

ocean90 on Apr 8, 2018 Member

ocean90 left a comment

This comment has been minimized.

ocean90 on Apr 8, 2018 Member

This comment has been minimized.

toolstack on May 17, 2018 Author Contributor

This comment has been minimized.

GaryJones on Apr 9, 2018

This comment has been minimized.

toolstack on May 17, 2018 Author Contributor

This comment has been minimized.

GaryJones on Apr 9, 2018

This comment has been minimized.

toolstack on May 17, 2018 Author Contributor

This comment has been minimized.

GaryJones on Apr 9, 2018

This comment has been minimized.

toolstack on May 17, 2018 Author Contributor

toolstack commented on Jul 7, 2017 •

edited

yoavf on Jul 10, 2017
Member

yoavf on Jul 10, 2017
Member

yoavf on Jul 10, 2017
Member

toolstack on Jul 10, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Oct 3, 2017
Member

toolstack on Oct 3, 2017
Author Contributor

ocean90 on Apr 8, 2018
Member

ocean90 on Apr 8, 2018
Member

toolstack on May 17, 2018
Author Contributor

toolstack on May 17, 2018
Author Contributor

toolstack on May 17, 2018
Author Contributor

toolstack on May 17, 2018
Author Contributor