Resource control with systemd
I’m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups.
My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary.
I created a new slice, and set a lower CPU share weight:
$ cat /etc/systemd/system/debomatic.slice
[Slice]
CPUAccounting=true
CPUShares=512
$
Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive.
That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:
This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I’m open to suggestions how to properly solve this issue 🙂
I’ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more 🙂
Adventures in cross-chroot creations
I’ve been playing with qemu-user-static a bit to create a set of porterboxes for my Deb-o-Matic build farm. After reading gregoa’s post on how to create cross-chroot with qemu-debootstrap, I was immediately able to create armel, armhf and powerpc boxes with very little efforts.
I tried to extend the number of porterboxes available by adding mips* and s390x, in order to have all the Linux-based architectures supported in Jessie, but with no luck. Here’s a summary of my attempts.
MIPS*
Chroot creation fails under both mips and mipsel trying to configure libuuid1. The problem is due to the fact libuuid1’s postinst script calls groupadd and useradd. Those two utilities rely on NETLINK sockets, which apparently are not handled by QEMU at the moment. I raised the question upstream to see whether it is possible to solve this problem.
s390x
Chroot creation used to fail with a SIGSEGV. This particular bug has been fixed recently, but it seems it’s not enough to have a working chroot. It fails with some gzip errors, probably because some portions of dpkg-deb are not fully covered by qemu-s390x-static.
Preparing to unpack .../base-files_7.3_s390x.deb ...
Unpacking base-files (7.3) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:5>: incorrect data check'
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/base-passwd_3.5.33_s390x.deb (--install):
subprocess dpkg-deb --control returned error exit status 2
Cross-architecture Linux containers in Debian
I wanted to create a Deb-o-Matic environment to testbuild packages for a different architecture. Taking inspiration on Stéphane’s excellent blog post, I tried to replicate the creation of a cross-architecture Linux container in Debian. Here are the steps I made:
Load binfmt_misc module:
# modprobe binfmt_misc
Install the required packages:
# apt-get install lxc debootstrap rsync qemu-user-static binfmt-support
Mount the cgroup virtual filesystem:
# mkdir /cgroup
# echo "none /cgroup cgroup defaults 0 0" >> /etc/fstab
# mount -a
I needed a specific template to create a cross-architecture container. I used the excellent one by Laurent Vivier. Download it, rename it as lxc-cross-debian, mark it executable, and store it under /usr/share/lxc/templates.
Create the cross-architecture container, an armhf one in this case:
# lxc-create -t cross-debian -n debian-armhf -- --arch armhf --suite sid --interpreter-path /usr/bin/qemu-arm-static
After a while, the container was created and I enjoyed my brand new armhf test machine 🙂
So long, and thanks for all the packages!
I’ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can’t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years.
This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I’ll take care of removing the debian.net domains I created for them.
Deb-o-Matic development won’t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.
On participation
While reading the platforms for the upcoming DPL election, I read some statements about hearing from Project members about their experience within the Debian Project, so I took the chance to write some thoughts about my experience in one of our core teams, in which I have the privilege to be part of: the FTP Team.
During NM process, my Application Manager asked me about some activities I could have involved into once I became a Debian Developer. I was fascinated by the hard work of the FTP folks when processing NEW packages (and I guess everybody reading these lines agree with me, remembering the happines of reading that wonderful ACCEPTED mail for their very first contribution to Debian, I still have it in my mailbox), so I decided I wanted to help out those folks to share a portion of their hard work.
I committed errors, sometimes they were so silly I’d be better to hide, but the team helped me to solve the mess I created. The “thank you” and “kudos” messages the FTP Team receives greatly overtake the frustration for my mistakes, and give the strength to keep up the good work 🙂
FTP Team is just one of the dozens teams which do an awesome work for the Debian sake, every day. Don’t be shy, or worried of being unsuitable or not skilled enough for the role. You’ll never know until you try. And it could happen you really like the job, and you could feel sorry for have waited so long 😉
Spare Parts TV Set
I don’t like watching videos sitting in front of a computer, I usually store them into a USB key to plug into my Samsung LE37 TV set. It’s not an easy task, though, as I have to fight with other components of my family who usually like watching something completely different than DebConf videos 😉 I could have invested 200 € to buy a new one, but I tried a different approach first.
Recycling a netbook
I received a Tweety device as Christmas gift, and I tried to connect it to my netbook to see how it worked. I was quite impressed by the good quality of the sound (if compared to the really poor quality of the netbook speakers), and I realized I could have recycled the netbook as a brand new TV set with some little adjustments. I plugged my SyncMaster 920n PC monitor into the netbook VGA port, switched Totem to fullscreen mode, and here it is my TV set!
Remote control
OK, that was quite straightforward to accomplish. A real TV set has a remote control too, but my netbook is not equipped with an IrDA port, so I had to think about a different solution. My BlackBerry is equipped with a cool app called BBSSH, which allows you to connect to any SSH server, this could be a good starting point to implement a remote control device. Then, I had to write an interface to Totem using Python and DBus binding to allow my “SSH remote control” to issue commands to start and pause the player. I published a first working implementation: at the moment the only supported operations are play, pause, stop and volume adjustments, but there are a lot of opportunities by implementing full MPRIS specification. Just to avoid typing a long command each time, I finally created a set of bash aliases, one for each command, so I just have to type one letter to have the desired function, as much as a real remote control 🙂
FTP Team stats during Wheezy development
Already chilled by Wheezy freeze? It’s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes.
Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:
| Login | Accepts | Rejects | Comments |
|---|---|---|---|
| ansgar | 407 accepts (0.800 per day) | 71 rejects (0.139 per day) | 53 comments (0.104 per day) |
| dak | 12 accepts (0.024 per day) | 1 rejects (0.002 per day) | 0 comments (0.000 per day) |
| dktrkranz | 4319 accepts (8.485 per day) | 381 rejects (0.749 per day) | 104 comments (0.204 per day) |
| joerg | 100 accepts (0.196 per day) | 12 rejects (0.024 per day) | 1 comments (0.002 per day) |
| mhy | 214 accepts (0.420 per day) | 14 rejects (0.028 per day) | 5 comments (0.010 per day) |
| stew | 67 accepts (0.132 per day) | 16 rejects (0.031 per day) | 7 comments (0.014 per day) |
| tolimar | 1480 accepts (2.908 per day) | 93 rejects (0.183 per day) | 84 comments (0.165 per day) |
| twerner | 278 accepts (0.546 per day) | 53 rejects (0.104 per day) | 26 comments (0.051 per day) |
Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:
- Debian Perl Group (559 accepts)
- Debian Haskell Group (491 accepts)
- Debian Ruby Extras Maintainers (285 accepts)
- Debian Java Maintainers (257 accepts)
- Debian Med Packaging Team (164 accepts)
- Debian Multimedia Maintainers (160 accepts)
- Debian Fonts Task Force (156 accepts)
- Debian Javascript Maintainers (137 accepts)
- Debian Python Modules Team (129 accepts)
- Debian Qt/KDE Maintainers (98 accepts)
That doesn’t reflect the real developers, though. Here’s our Changed-By top ten:
- Clint Adams (216 accepts)
- Jonas Smedegaard (208 accepts)
- Ben Hutchings (203 accepts)
- Joachim Breitner (153 accepts)
- TANIGUCHI Takaki (112 accepts)
- Alessio Treglia (101 accepts)
- David Paleino (95 accepts)
- Nicholas Bamber (76 accepts)
- Mathieu Parent (68 accepts)
- Jeff Breidenbach (63 accepts)
Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place 😉
How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not 🙂
liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never 😉
FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:
- Source packages removed: 2159
- Binaries for architecture all: 8773
- Binaries for architecture alpha: 3646
- Binaries for architecture amd64: 14653
- Binaries for architecture arm: 3
- Binaries for architecture armel: 13283
- Binaries for architecture armhf: 3423
- Binaries for architecture hppa: 1418
- Binaries for architecture hurd-i386: 6923
- Binaries for architecture i386: 15655
- Binaries for architecture ia64: 12365
- Binaries for architecture kfreebsd-amd64: 10972
- Binaries for architecture kfreebsd-i386: 10998
- Binaries for architecture mips: 12478
- Binaries for architecture mipsel: 12479
- Binaries for architecture powerpc: 13945
- Binaries for architecture s390: 12374
- Binaries for architecture s390x: 3884
- Binaries for architecture sparc: 12295
FTP Team also took care of override changes:
- Number of priority changes: 853
- Number of section changes: 975
Invitation, part II
Wheezy is frozen! Hooray!
This also means you’re probably too late in uploading NEW packages into the archive, so why don’t you fix some RC bugs instead, contextually helping FTP Team keeping the queue low? Thanks, and happy bug squashing! 🙂
Spam review: mass-tagging nominations
If you are one of those brave souls chasing off spam from Debian mailing lists, you will probably get bored of clicking on the “Spam” radio buttons a lot of times, and will probably look for a quick shortcut. I submitted a patch in bug #640309, but as of today, it has not been applied yet.
Waiting for the patch to be applied, I prepared a little Greasemonkey script called Debian Mass Spam that allows to mark every message as “Unsure”, “Ham”, “Inappropriate”, or “Spam” by clicking the correspondent key on the keyboard:
Just press Z (for “Unsure”), X (for “Ham”), C (for “Inappropriate”), or V (for “Spam”), and you will see every radio button to change its status. I tested it on Chromium, Iceweasel (with xul-ext-greasemonkey package) and Epiphany (with epiphany-extensions package), and it seems to work properly.
Too many words and no actions so far, go and kill some spam now! 😉
Kranz Korner 
