Web Development on Windows in 2018

Foreword

I have been developing web apps on Windows for the last 10+ years. I ran PHP, Apache then Nginx, MySQL and Postgres, Redis and others. I dabbed in VMs at some point but was always dissatisfied with the experience so mostly this was all running natively on Windows. There were quirks for sure, I have used a lot of duct tape over the years.

All my hosting and sysadmin work is done on Linux though, so I have always felt more comfortable using command line tools from the Linux world. I have tried a variety of solutions to get those running on Windows, Cygwin was interoperability hell, Msys/MinGW was better but also had shortfalls.

Then a couple years ago Microsoft announced WSL (Windows Subsystem for Linux). In short it is a Windows-level implementation of the Linux system calls, allowing us to run Linux-compiled software on top of the Windows kernel. The coverage of syscalls isn't 100% yet, but as of the Windows 10 April Update released last month most things work really well for my purposes. As I recently got a new laptop I figured it was time to switch over.

Hardware considerations

In the last year, more and more of my Mac-using colleagues and friends have stopped making fun and started complaining, missing Esc keys and whatnot. The reasons for their anguish do not matter much, but it seems like lots of people are considering switching to Windows and/or Linux. I figured I would do a write up as it can probably help smoothen the migration process.

If you are looking to get a new laptop, I can only recommend Lenovo's T-Series (I just got a T480s) or the X1 Carbon or Microsoft's own Surface Book 2 if portability is really important to you. Dell and Asus also seem to have a few decent machines but I have no experience there, and I would stay away from any other cheaper manufacturers if you're going to use this all day every day.

Setting up a new machine

First of all, depending on how you configured your machine to start with, check what your username is. If you open a terminal (open up the start menu and type PowerShell or cmd.exe/Command Prompt) you will see the prompt contains C:\Users\XXX - that last bit is your username, and if it doesn't please you or contains spaces or non-ASCII characters I would highly recommend renaming your user account as it will likely save you some trouble later.

Then the first step is to install Chocolatey which is a Windows package manager. Once you have that, you can install a bunch of things you will need on the Windows side.

I would recommend the following:

choco install -y firefox googlechrome dropbox slack skype thunderbird notepad2-mod sysinternals xnviewmp 7zip.install foobar2000 cyberduck wox heidisql sublimetext3 vcxsrv
choco install -y clipx --version 1.0.3.9-beta --pre
choco pin add --version 1.0.3.9-beta -n=clipx

Go through the list and remove those you don't want obviously, but a few that you might not recognize if you haven't been using Windows are:

• notepad2-mod overwrites the good ol' notepad with something that's not a full blown editor but does a decent job of quickly editing a file
• sysinternals is a set of utilities for monitoring and debugging windows
• xnviewmp is a great image viewer, IMO anyway, the one shipping with windows isn't all that bad these days depending on what you do
• 7zip handles all possible archive formats
• foobar2000 is the best music player with the worst name, nerdy business, skip it if you gave in to music streaming
• cyberduck does S/FTP just in case the 90s call, but also S3 browsing
• wox is like Alfred for OSX, an app launcher, you can also use the windows key then type but I find Wox faster and more configurable
• heidisql is a nice way to browse your MySQL/PostgreSQL database, definitely beats phpMyAdmin or the like
• sublimetext3 my editor of choice but if you have opinions please remove it and move on, I don't care
• vcxsrv is an X server which needs to run to be able to run GUIs from Linux, which I really only use for gitk. You can also look at X410 which is a paid alternative with a bit better support if you want to run many linux apps.
• clipx is a multiple clipboard storage which looks horrendous but is functional, if you know a better option please do tell

This gives you a decent basis to get stuff done, and for backup and whatnot, please note that most apps will (and should) store their settings in C:\Users\...\AppData\Roaming. When migrating to a new laptop copying that over is the best way to carry over settings. Some stuff will end up in AppData\Local but in theory that is only made for caches and temp files.

If you then want to upgrade apps, run choco upgrade -y all in an admin shell.

Installing Ubuntu, git/ssh and a web server

First of all, it's not exciting but it's a good idea to set some environment variables. Search for "Edit the environment variables" in the start menu, then in the dialog add the following User env vars:

WSLENV set to HOME/p - this says that WSL should receive your HOME env var, /p means it's a path and should be converted to linux style path.

HOME set to C:\Users\XXX - or whatever you want your home dir to be, but this is what makes sense from a Windows perspective.

Add a few paths to the Path environment variable (it is exported to WSL automatically). I recommend adding C:\Users\XXX\bin to have a place to drop exe files or scripts you want to be able to execute from anywhere, C:\Users\XXX\.yarn\bin if you are going to use Yarn, C:\Users\XXX\.config\composer\vendor\bin if you use Composer to and want to install global utils.

Then as it is 2018, you can get Ubuntu 18.04 right from the Microsoft store. What a world we live in!

Now you can install ConEmu, which is my terminal emulator of choice. This will do the color support etc. If you install it once Ubuntu is present I think it'll default to the right thing, but if not make sure you pick {Bash::bash} as its startup task. That way you get to Linux-land as soon as you open it.

Alternatively open a new cmd.exe window and type bash or ubuntu1804 to start it.

Ubuntu / WSL configuration

Once in a bash prompt, run sudo nano /etc/passwd to edit your home dir. By default it will be /home/XXX but I rather not use that as it is under the Linux filesystem and I want to be able to store as little as possible in there, so I can manage my files from Windows using my editor of choice and also make sure I have backups running etc. The Linux filesystem I treat as disposable. Anyway once in that /etc/passwd file you can look for a line like XXX:x:1000:1000:,,,:/home/XXX:/bin/bash with your username, change the /home/XXX at the end to be /mnt/c/Users/XXX instead, you'll thank me later. Save and exit (Ctrl-X then Y then Enter), then go on to the next file: sudo nano /etc/wsl.conf to edit the WSL config, and make sure that it contains this at the end:

[automount]
enabled = true
options = "metadata"

Save and exit (Ctrl-X then Y then Enter), close all bash windows to make sure it "shuts down" Ubuntu and then reopen it. This metadata option means that it will let you use chmod to modify Windows files permissions from bash. That is invaluable to set up SSH correctly. You will want to have your ssh key set up just like you would on OSX or in Linux at ~/.ssh, or in Windows terms %USERPROFILE%/.ssh, so C:\Users\XXX\.ssh. Put your public/private keys in there, and ssh_config if you have one. You can create the folder from the Ubuntu terminal if needed, or create it as .ssh. from Explorer otherwise it won't let you save it as .ssh, but the trailing dot will get trimmed. Once you got those files set up you want to make sure SSH does not flip out about file permissions. By default all files in the Windows filesystem have a 777 mode for Linux, so SSH will refuse to read your key as it is insecure. chmod -R go-rwx ~/.ssh && chmod 0400 ~/.ssh/id_rsa* should fix that. Try to ssh in to some server if you can to verify.

Web server

Once that is done, time to install a web server, PHP and whatnot. You can run the commands in the Setup script at https://gist.github.com/Seldaek/fe3676f8b1ade7b9eb65abd5f5508b3a. I tried to split up the steps so it can be edited easily, but I think that should be a good base for most PHP devs at least. If you want Apache.. don't ask me :)

The other file accessible in the gist linked above is a .bash_profile, which you can drop in ~/.bash_profile. It adds a few aliases, and ssh-agent so that your encrypted SSH key doesn't prompt for a password every single time but only once when you open the terminal. It also contains a few helpful functions like n/subl/exp which will start notepad (or nano if editing a linux file)/sublime text/explorer but do path translation first so that if you give them a linux-style path they will receive the equivalent windows path and open the right file/directory. Note that you have to open a new terminal for that file to take effect. It's not the end all of bash configs, but it's a good start and inspiration to build upon.

To configure PHP and nginx, there are a few minor things I would recommend to make it run smoothly. In /etc/php/7.2/fpm/pool.d/www.conf you should set the following to make sure the php processes are running as your user and you don't have permission issues:

listen = /var/run/php-fpm.sock
user = XXX
group = XXX

Then in /etc/nginx/nginx.conf inside the http {} block make sure you add fastcgi_buffering off;. That is required to make it run on WSL for now. Hopefully a future Windows version will fix it. In that same file I also add at the end of the http {} block an include so that I can manage all my vhost files in windows and not loose them if I decide to reset Ubuntu: include /mnt/c/Users/XXX/nginx-vhosts/*.vhost; - check nginx_example.vhost in my gist to see how these vhosts can look like for hosting a Symfony app. What matters really is that root has to be a Linux-style /mnt/c/.. path and fastcgi_pass should match what FPM listens to.

For starting up nginx, php etc you use sudo service php7.2-fpm restart for example. Now there is a catch here one of the issues left of WSL is that it has no init system really as it doesn't "boot" an OS per se. So you can't easily make it start services automatically in the background. You could make sure they start at the end of your .bash_profile, but that slows down opening new terminals so I took the approach of adding a new file in ~/bin/srv containing the following:

#!/bin/bash

sudo service php7.2-fpm start
sudo service nginx start
sudo service redis-server start

Now as I added this bin dir to my PATH earlier, all I need to do when I start my machine is open a new terminal, run srv and all is well.

One last thing I would recommend if you care about gitk is to run echo 'start "" "C:\Program Files\VcXsrv\vcxsrv.exe" -multiwindow' > /mnt/c/Users/$USER/AppData/Roaming/Microsoft/Windows/Start\ Menu/Programs/Startup/VcXsrv.bat to make sure VcXsrv starts automatically when you start windows, otherwise gitk won't be able to run without X server.

Gotchas

Since Windows is case insensitive and contains lots of uppercase paths I would also recommend adding a ~/.inputrc file with set completion-ignore-case On in it. That makes bash tab completion ignore case sensitivity, which makes life much easier IMO.

Speaking of differences, if you create files from Windows, make sure you use LF line endings and not CRLF, or bash will trip up. Notepad2 has an easy way to change that by double clicking on LF/CR/CR+LF in the bar at the bottom.

FAQ

What about MySQL/PostgreSQL?

I install those in Windows and not Ubuntu, because I want the data to live in the Windows filesystem as explained above. Sockets work just fine across the OS boundary so you can connect from your Ubuntu PHP.

Which Antivirus do you need?

Just don't go there IMO. Windows Defender has shipped with Windows for ages now, and it's really good enough if you are not opening random .exe attachments from dodgy emails. Other antivirus software tends to hook so deep into the system that they cause instability or even introduce vulnerabilities.

Conclusion

That's it for the first steps. Hopefully this gets you up and running quicker than if you had to figure it all out. If anyone has specific tips please do write in the comments below. I am sure some of this can be improved.

May 16, 2018 // News, Random // Post a comment

PHP Versions Stats - 2018.1 Edition

It's stats o'clock! See 2014, 2015, 2016.1, 2016.2, 2017.1 and 2017.2 for previous similar posts.

A quick note on methodology, because all these stats are imperfect as they just sample some subset of the PHP user base. I look in the packagist.org logs of the last month for Composer installs done by someone. Composer sends the PHP version it is running with in its User-Agent header, so I can use that to see which PHP versions people are using Composer with.

PHP usage statistics

May 2018 (+/- diff from November 2017)

All versions			Grouped
PHP 7.2.4	7.54%		PHP 7.1	35.02% (-1.61)
PHP 7.1.16	7.41%		PHP 7.0	23.02% (-7.74)
PHP 7.0.28	5.54%		PHP 7.2	20.18% (+20.18)
PHP 7.1.15	4.11%		PHP 5.6	16.48% (-6.8)
PHP 7.2.3	3.85%		PHP 5.5	3.50% (-2.61)
PHP 7.1.14	3.79%		PHP 5.4	1.04% (-0.47)

A few observations: PHP 7.1 is still on top but 7.2 is closing real quick with already 1/5th of users having upgraded. That's the biggest growth rate for a newly released version since I have started collecting those stats. Ubuntu 18.04 LTS ships with 7.2 so this number will likely grow even more in the coming months. 78% of people used PHP 7+ and almost 95% were using a PHP version that is still maintained, it sounds too good to be true. PHP 5.6 and 7.0 will reach end of life by late 2018 though so that's 40% of users who are in need of an upgrade if we want to keep these numbers up!

Here is the aggregate chart covering all my blog posts and the last five years.

PHP requirements in Packages

The second dataset is which versions are required by the PHP packages present on packagist. I only check the require statement in their current master version to see what the latest requirement is, and the dataset only includes packages that had commits in the last year to exclude all EOL'd projects as they don't update their requirements.

PHP Requirements - Recent Master - May 2018 (+/- diff from Recent Master November 2017)

5.2	1.16% (-0.12)
5.3	15.9% (-2.85)
5.4	16.59% (-3.7)
5.5	15.52% (-3.55)
5.6	19.57% (-0.83)
7.0	19.47% (4.62)
7.1	11.15% (5.83)
7.2	0.64% (0.61)

This is as usual lagging behind a little but PHP 7 is finally seeing some real adoption in the OSS world which is nice.

May 14, 2018 // News, PHP // Post a comment

PHP Versions Stats - 2017.2 Edition

It's stats o'clock! See 2014, 2015, 2016.1, 2016.2 and 2017.1 for previous similar posts.

A quick note on methodology, because all these stats are imperfect as they just sample some subset of the PHP user base. I look in the packagist.org logs of the last month for Composer installs done by someone. Composer sends the PHP version it is running with in its User-Agent header, so I can use that to see which PHP versions people are using Composer with.

PHP usage statistics

November 2017 (+/- diff from May 2017)

All versions			Grouped
PHP 7.1.10	11.63%		PHP 7.1	36.63% (+18.99)
PHP 7.0.22	7.95%		PHP 7.0	30.76% (-5.36)
PHP 5.6.31	7.38%		PHP 5.6	23.28% (-8.16)
PHP 5.6.30	7.23%		PHP 5.5	6.11% (-4.5)
PHP 7.0.24	5.45%		PHP 5.4	1.51% (-1.6)
PHP 7.1.11	4.55%		PHP 5.3	0.76% (-0.22)

A few observations: I find it quite incredible that PHP 7.1 is now the most used version, even though Ubuntu LTS does not yet ship with it. I don't know if it means people use Docker or alternative PPAs but regardless it is good news! For the first time since I started these blog posts, the version usage actually matches the order in which they were released, with the older ones having less and less usage. That's also great news. We have a total of 90% of installs done on PHP versions that are still maintained, which is awesome. If you are still on 5.6 or 7.0 though you only have one year of security fixes left so consider upgrading to 7.2 which should come out in the next week or two.

Here is the aggregate chart covering all my blog posts and the last four years.

PHP requirements in Packages

The second dataset is which versions are required by the PHP packages present on packagist. I only check the require statement in their current master version to see what the latest requirement is, and the dataset only includes packages that had commits in the last year to exclude all EOL'd projects as they don't update their requirements.

PHP Requirements - Recent Master - November 2017 (+/- diff from Recent Master May 2017)

5.2	1.28% (-0.24)
5.3	18.75% (-4.4)
5.4	20.29% (-4.12)
5.5	19.07% (-4.63)
5.6	20.4% (3.59)
7.0	14.85% (6.12)
7.1	5.32% (3.65)
7.2	0.03% (0.03)

This moves at a decent pace with EOL'd versions slowly being abandoned. I still think it could go faster though ;) Please consider bumping to PHP 7.0 at the very least when you update your libraries.

November 13, 2017 // News, PHP // Post a comment

PHP Versions Stats - 2017.1 Edition

It's stats o'clock! See 2014, 2015, 2016.1 and 2016.2 for previous similar posts.

A quick note on methodology, because all these stats are imperfect as they just sample some subset of the PHP user base. I look in the packagist.org logs of the last month for Composer installs done by someone. Composer sends the PHP version it is running with in its User-Agent header, so I can use that to see which PHP versions people are using Composer with.

PHP usage statistics

May 2017 (+/- diff from November 2016)

All versions			Grouped
PHP 5.6.30	14.73%		PHP 7.0	36.12% (+1.11)
PHP 7.0.15	9.53%		PHP 5.6	31.44% (-6.02)
PHP 5.5.9	6.12%		PHP 7.1	17.64% (+16.28)
PHP 7.0.17	6.00%		PHP 5.5	10.61% (-8.32)
PHP 7.1.3	5.88%		PHP 5.4	3.11% (-2.29)
PHP 7.1.4	3.65%		PHP 5.3	0.98% (-0.62)

A few observations: With a big boost of PHP 7.1 installs, PHP 7 overall now represents over 50%. 5.3/5.4 are really tiny and even 5.5 is dropping significantly which is good as it is not maintained anymore since last summer. That's a total of 85% of installs done on supported versions, which is pretty good.

And because a few people have asked me this recently, while HHVM usage is not included above in the graph it is at 0.36% which is a third of PHP 5.3 usage and really hardly significant. I personally think it's fine to support it still in libraries if it just works, or if the fixes involved are minor. If not then it's probably not worth the time investment.

Also.. since I now have quite a bit of data accumulated and the pie chart format is kind of crappy to see the evolution, here is a new chart which shows the full historical dataset!

It is pretty interesting I think as it shows that 5.3/5.4/5.5 had people slowly migrating in bunches and the versions peaked at ~50% of the user base. On the other hand 5.6/7.0/7.1 peak around 35% which seems to indicate people are moving on faster to new versions. This is quite encouraging!

PHP requirements in Packages

The second dataset is which versions are required by all the PHP packages present on packagist. I only check the require statement in their current master version to see what the latest is.

PHP Requirements - Current Master - May 2017 (+/- diff from November 2016)

5.2	2.13% (-0.22)
5.3	37.6% (-3.65)
5.4	28.38% (-1.74)
5.5	17.11% (+0.13)
5.6	9.37% (+3.15)
7.0	4.61% (+1.53)
7.1	0.81% (+0.81)

A few observations: This is as usual moving pretty slowly. I think I can give up trying to advise for change, it doesn't seem to be working.. On the other hand it looks like Symfony is going to use 7.0 or 7.1 for it's v4 to come out later this year, so hopefully that will shake things up a bit and make more libraries also realize they can bump to PHP 7.

PHP Requirements - Recent Master - May 2017 (+/- diff from Current Master November 2016)

In response to Nikita's comment below I ran the requirements numbers for packages that had some sort of commit activity over the last year. This excludes all stale/done packages and looks much more encouraging, but the difference points are probably overly large because they compare to the old numbers which included everything, therefore take those with a pinch of salt, and in the next six months update I'll have more trusty numbers.

5.2	1.52% (-0.83)
5.3	23.15% (-18.1)
5.4	24.41% (-5.71)
5.5	23.7% (+6.72)
5.6	16.81% (+10.59)
7.0	8.73% (+5.65)
7.1	1.67% (+1.67)

May 06, 2017 // News, PHP // Post a comment

PHP Versions Stats - 2016.2 Edition

It's stats o'clock! See 2014, 2015 and 2016.1 for previous similar posts.

A quick note on methodology, because all these stats are imperfect as they just sample some subset of the PHP user base. I look in the packagist.org logs of the last 28 days for Composer installs done by someone. Composer sends the PHP version it is running with in its User-Agent header, so I can use that to see which PHP versions people are using Composer with.

PHP usage statistics

I have two datasets, from May 2016 and today, which shows the progression of various versions.

May 2016

All versions			Grouped
PHP 5.5.9	11.87%		PHP 5.6	39.67%
PHP 7.0.6	10.39%		PHP 5.5	29.56%
PHP 5.6.20	8.41%		PHP 7.0	20.24%
PHP 5.6.21	7.69%		PHP 5.4	7.64%
PHP 5.6.19	4.71%		PHP 5.3	2.43%

November 2016

All versions			Grouped
PHP 7.0.12	8.58%		PHP 5.6	37.46%
PHP 5.5.9	8.25%		PHP 7.0	35.01%
PHP 7.0.11	7.62%		PHP 5.5	18.93%
PHP 7.0.8	6.92%		PHP 5.4	5.40%
PHP 5.6.26	6.12%		PHP 5.3	1.60%
PHP 5.6.27	4.49%		PHP 7.1	1.36%

A few observations: 5.3 and 5.4 at this point are gone as far as I am concerned! 5.5 still has a good presence but lost 12% in 6 months which is awesome. 5.6 basically stayed stable as I suspect people jumped from 5.5 to 7 directly probably when upgrading Ubuntu LTS. 7.0 gained 15% and is now close to being the most deployed version, 1 year after release! That should definitely encourage more libraries to require it IMO, and I hope it is good encouragement to PHP internals folks as well to see that people actually upgrade these days :) Interestingly 7.1 is almost passing 5.3 already and it isn't even released. That is probably coming from CI installs mostly but for example I already run 7.1 on my local dev environment and I hope others do too.

PHP requirements in Packages

The second dataset is which versions are required by all the PHP packages present on packagist. I only check the require statement in their current master version to see what the latest is.

PHP Requirements - Current Master - November 2016 (+/- diff from May 2016)

5.2	2.35% (-0.16)
5.3	41.25% (-4.01)
5.4	30.12% (-1.57)
5.5	16.98% (+1.5)
5.6	6.22% (+2.7)
7.0	3.08% (+1.54)

A few observations: I don't know how else to say this but PEOPLE COME ON! This is moving waaaay slower than people are migrating their servers, and it doesn't make any sense to me. I guess there are a lot of projects out there that are somewhat stale or stable and not really changing and that makes sense, but if you still maintain a library, don't hesitate to require 7 and bump the major release at this point. You will have more fun using all the cool features of the language instead of being stuck writing array().

As I wrote in the last update: I would like to encourage everyone to be a bit more aggressive in bumping PHP requirements when tagging new major releases of their libs. Don't forget that the old code does not go away, it's still there to be used by people using legacy PHP versions.

However it seems that a lot of people here do not read and just look at the pictures, so allow me to illustrate this last point.

November 18, 2016 // PHP // Post a comment

Typo Squatting and Packagist

Earlier this month an article was published summarizing Nikolai Philipp Tschacher's thesis about typosquatting. In short typosquatting is a way to attack users of a package manager by registering a package with a name similar to a popular package, hoping that someone will accidentally typo the name and end up installing your version of it that contains malware.

The thesis mentions https://packagist.org as a good example as we use vendor namespaces:

[...] it is much more secure, if a package is named ntschacher/GoogleScraper instead of just GoogleScraper. The reason is: If the package name is misspelled and not the author name, this will not have any consequences, because the typo version cannot be registered in this namespace, since this author name is already reserved. [...] Because package names are much longer with two attributes, it is more likely that users will copy and paste the package name instead of remembering it.

Despite this mitigating fact, it is still technically possible to squat the vendor name, so I wanted to take a look at our repository data and see if I could spot any bad actors. I wrote a script that basically does the following:

• Read the list of all vendor names which have packages with at least 1000 downloads, as the others are unlikely targets or at least low value targets.
• Check the levenshtein distance of every vendor name against all others.
• If the distance is 1, then it checks for package names within those two vendors to see if they have any intersecting names. Those are then candidates for being typosquatters.

What did I find? 21 vendor pairs that conflict to some degree. Only one that looked like an actual typosquatting attempt, momolog/monolog, and it even had in the package description that it was a demonstration of typosquatting. I deleted it along with 5 others packages that were useless, but the others are still in place. A lot of it is just due to people renaming their vendor names, or simply people that picked similar names but don't seem to be abusing anything.

In the future it would be nice to automate this, or prevent the creation of vendors that are too similar to popular ones. However it is reassuring to see that there is no widespread abuse going on.

June 29, 2016 // PHP // Post a comment

PHP Versions Stats - 2016.1 Edition

Last year I posted stats about PHP versions, and the year before as well, both time in November. However this year I can't wait for November as I am curious to explore the PHP7 uptake!

A quick note on methodology, because all these stats are imperfect as they just sample some subset of the PHP user base. I look in the packagist.org logs of the last 28 days for Composer installs done by someone. Composer sends the PHP version it is running with in its User-Agent header, so I can use that to see which PHP versions people are using Composer with.

PHP usage statistics

I have two datasets, from November 2015 and today, which shows the progression of various versions. Note that the previous dataset was checking for Composer updates only, while the new one includes installs as well.

November 2015

All versions			Grouped
PHP 5.5.9	29.63%		PHP 5.5	50.68%
PHP 5.6.14	5.63%		PHP 5.6	22.09%
PHP 5.3.3	4.60%		PHP 5.4	15.86%
PHP 5.4.45	3.94%		PHP 5.3	9.90%
PHP 5.6.13	3.39%		PHP 7.0	1.17%

May 2016

All versions			Grouped
PHP 5.5.9	11.87%		PHP 5.6	39.67%
PHP 7.0.6	10.39%		PHP 5.5	29.56%
PHP 5.6.20	8.41%		PHP 7.0	20.24%
PHP 5.6.21	7.69%		PHP 5.4	7.64%
PHP 5.6.19	4.71%		PHP 5.3	2.43%

A few observations: 5.3 dropped to almost nothing which is great news! 5.4 is also down by almost 10% and is definitely on the way out. 5.5 is still big but less so, while 5.6 got a huge boost to become the main version. The big surprise is that we have 20% of PHP7 already! That is great news only six months after this major release came out.

PHP requirements in Packages

The second dataset is which versions are required by all the PHP packages present on packagist. I only check the require statement in their current master version to see what the latest is.

PHP Requirements - Current Master - May 2016 (+/- diff from November 2015)

5.2	2.51% (-0.3)
5.3	45.26% (-6.43)
5.4	31.69% (-1.76)
5.5	15.48% (+5.29)
5.6	3.52% (+1.84)
7.0	1.54% (+1.34)

A few observations: 5.3/5.4 are declining slowly, 5.5 is taking the bulk of it though which makes me a bit sad :) I wish there was more love for 7 now that it shipped in Ubuntu 16.04.

All in all, it seems like package requires are way behind actual version usage, so I would like to encourage everyone to be a bit more aggressive in bumping PHP requirements when tagging new major releases of their libs. Don't forget that the old code does not go away, it's still there to be used by people using legacy PHP versions.

June 06, 2016 // PHP // Post a comment

Goddamn it.

It's not often that one messes up really bad. But today is my day apparently.

TL;DR: I accidentally wiped a github organization that had a few popular repos on it. But it's all fixed now.

How the heck did this happen?

I was trying to remove a private repository, called nelmio, which incidentally has the same name as the organization it was in, so nelmio/nelmio. Then this happened:

• I wanted to check repo permissions so I went to https://github.com/nelmio/nelmio/settings/collaboration then followed a team link which led to https://github.com/orgs/nelmio/teams/foo
• Then I was like OK let's go back in the settings tab to delete this repo, except at this point the settings tab points to https://github.com/organizations/nelmio/settings/profile (i.e. the org settings not the repo)
• So at the end of the settings tab I see the familiar red delete button, hit it, it tells me to type the repo name (nelmio) as usual, but obviously I've done this many times so I don't read the fine print. It turns out in this case it wanted me to confirm the org name and not repo name.
• As I click "Confirm Delete" I saw that something in the message wasn't quite familiar, but then it reloaded the page and I find myself on the github home. I'm like "That's odd!", then more or less 2 seconds later this horrible feeling in my guts is confirmed, the entire org was wiped.

Mitigation

I immediately emailed GitHub support, and am still waiting for an answer. I kinda wish there was a hotline for such cases, even if it was billed 10 bucks a minute :)

After doing so, I started re-pushing repos into a new organization (nelmiobackup). I then changed the packagist.org package URLs to point to this new org, so that at least package installs should continue working relatively normally for the time being.

I did not want to re-create a nelmio org as I thought this might hamper any recovery effort by the github support folks, but someone had the great idea to do that for me, so now that the potential damage is done and they added me as owner, I forked everything from nelmiobackup to nelmio so that it's present at the old URL for people doing installs using composer.lock files that point to the old URL.

I hope GitHub will be able to fix this, but if not apparently http://ghtorrent.org/ has a ton of github data. We'll see what can be done.

Updates

Update1: GH support answered, it seems they can restore. I had to rename nelmio to nelmio-old for now to make room for the restore, so clones with old URLs will temporarily fail again.

Update2: All restored, I only have to re-create teams within the org, no biggie :) Total time was a bit over 1h from deletion, which isn't too bad in the grand scale of things.

P.S: If you feel like comparing this to the left-pad incident, this is quite different because I fucked up accidentally while the guy in question did it intentionally. I guess I can't stop you though.

P.P.S: If you want to laugh or say anything mean, please go away. I don't want to hear it right now. It is stressful enough as it is, and I am doing what I can.

May 31, 2016 // News // Post a comment

Common files in PHP packages

This one started in a peculiar way. Paul M. Jones announced a new version of his Producer tool, I had a look at it and saw that it recommended having a changelog called CHANGES.md by default. This irked me a bit because I always use CHANGELOG.md and hardly ever see that as a file name (it's the little things that matter, right?).

My first thought was to report an issue asking to change the default, but then I thought it's Paul, he will not just take my word for it, he will want hard facts. So here I am two days later. I queried GitHub's API for the file listing (only the root directory) of all PHP packages listed on packagist.org.

Continue reading...

April 21, 2016 // PHP // Post a comment

Composer goes Gold

Five years ago today, Composer was born. In some ways it feels like yesterday, at least it doesn't feel like five years went by. In other ways it seems like a lifetime ago, and I can barely remember what it was like to write PHP code without having a whole ecosystem at my fingertips.

Composer 1.0.0

Today I have the pleasure of announcing that the first 1.0.0 stable release is out and available for immediate download!

It has been a long time coming, but we fixed a few last critical issues in the last few months that finally allow me to take this step. Going forward I plan on releasing more frequently as well ;)

Update channels

One big change that happened recently is that by default the Composer installer and composer self-update both install stable releases by default. This is great to avoid bad surprises if you run self-update as part of your deployment, but it also means that the feedback loop gets longer for us when we do changes. Therefore I really hope that we can get enough people running frequent self-updates using the preview (alpha/beta/..) and especially snapshot (dev builds) channels.

My recommendation would be to run regular updates for deployment/builds to have stability, run self-update --preview in CI if you can to make sure you test at least pre-release versions. And on dev environments composer self-update --snapshot would give you the latest and shiniest Composer has to offer. This will ensure we spot regressions or mistakes as early as possible, and thus avoid breaking things in stable releases.

Composer Gold Edition

Finally, in an attempt to mark the fact that Composer has finally gone gold, I wanted to do something special.

My girlfriend had a brilliant idea, and a few days and a couple express deliveries later here we are. We made an actual Composer gold master copy of the 1.0 release, on a floppy!

Collector items are no fun if you can't collect them though, so you can head to eBay now to bid on it if you'd like to own it!

Here's to the next five years (for the 2.0, hah.)!

April 04, 2016 // News, PHP // Post a comment