Artificial intelligence is not willing to be correct

• Vincent Lequertier's blog
• —
• 09:00, Sunday, 29 January 2023

As deep learning models get better at representing human language, telling whether a text was written by a human being or a deep learning model becomes harder and harder. And because language models reproduce text found online (often without attribution); the risk of considering their output as if they were written by a human changes the reading experience for the reader.

The last year has been incredible for natural (and programming) language processing. GitHub’s Copilot has been out of technical preview since June, and ChatGPT was released in November. Copilot is based on OpenAI Codex and acts as a source code generator (which raises several issues of its own). ChatGPT is a language model built for dialogue, where a user can chat with the AI, ask questions and have them answered. Both are trained with data from web scrapping, with source code for Copilot and webpages for ChatGPT. Those models work particularly well for their respective purposes, and can thus be used to generate seemingly convincing source code or prose.

Because AI-generated texts are convincing, the fact that they were generated by an AI is not obvious to the careless reader. This is problematic, as there is no guarantee that the text is factually correct and that the human leveraging the AI checked it for mistakes. When reading, this may create a discomfort, as the reader has to determine whether a text was generated by an AI, and if so, if the publisher made sure that it is correct. Companies already started to use AI generated text for articles without clearly visible disclaimers and riddled with errors. The fact that text generated by ChatGPT may contain inaccuracies was acknowledged by OpenAI’s CEO. One might argue that humans make mistakes, too, and that prose or source code written by a human being can therefore also be wrong. This is true. However, the intent behind the text differs. In most cases, the author of a text tries their best to make it correct. But the language model does not understand the concept of correctness and will happily generate text containing wrong facts, which changes the tacitly assumed rules of writing and reading content.

Gaining trust in the text generated by an AI is thus a worthwhile objective. Here are partial solutions to this:

• Watermarking texts generated by GPT models is a work in progress. Among the possible ones, the words chosen by the AI would embed a proof (using asymmetric cryptography) in their probability distribution. While this does not alleviate the concern stated above, this allows the reader to avoid AI-generated text if he or she wants to.

• Connecting the text generated by the AI back to what lead it to generate the text might be another may offer a partial solution. If the readers can verify the trustworthiness of the sources, they might feel more confident about the AI-generated text they are reading.

• If citing the source is too involved computationally, weighting the learning process of the AI in such a way that would give more importance to the authoritative sources on a subject would be a good workaround. Counting the number of backreferences of a page would be a good indicator of whether the text it contains is authoritative (just like page rank).

Considering this perspective, using large language models raises trust issues. A few technical solutions are listed above. However, it would be too reductive to consider this only as a technical problem. AI generated text then looks akin to search engines, without the comfort of knowing that they merely redirect to a source website, whose content is presumably written by a human being who tried to make it correct.

PS: This article was not written by an AI.

Use Any SOP Binary With SOP-Java and External-SOP

• foss – vanitasvitae's blog
• —
• 19:20, Friday, 13 January 2023

The Stateless OpenPGP Protocol specification describes a shared, standardized command line interface for OpenPGP applications. There is a bunch of such binaries available already, among them PGPainless’ pgpainless-cli, Sequoia-PGP’s sqop, as well as ProtonMails gosop. These tools make it easy to use OpenPGP from the command line, as well as from within bash scripts (all of those are available in Debian testing or in the main repo) and the standardized interface allows users to switch from one backend to the other without the need to rewrite their scripts.

The Java library sop-java provides a set of interface definitions that define a java API that closely mimics the command line interface. These interfaces can be implemented by anyone, such that developers could create a drop-in for sop-java using the OpenPGP library of their choice. One such backend is pgpainless-sop, which implements sop-java using the PGPainless library.

I just released another library named external-sop, which implements sop-java and allows the user to use any SOP CLI application of their choice from within their Java / Kotlin application!

Let’s assume we have a SOP command line application called example-sop and we want to make use of it within our Java application. external-sop makes the integration a one-liner:

SOP sop = new ExternalSOP("/usr/bin/example-sop");

Now we can use the resulting sop object the same way we would use for example a SOPImpl instance:

// generate key
byte[] keyBytes = sop.generateKey()
        .userId("John Doe <[email protected]>")
        .withKeyPassword("f00b4r")
        .generate()
        .getBytes();

// extract certificate
byte[] certificateBytes = sop.extractCert()
        .key(keyBytes)
        .getBytes();

byte[] plaintext = "Hello, World!\n".getBytes(); // plaintext

// encrypt and sign a message
byte[] ciphertext = sop.encrypt()
        // encrypt for each recipient
        .withCert(certificateBytes)
        // Optionally: Sign the message
        .signWith(keyBytes)
        .withKeyPassword("f00b4r") // if signing key is protected
        // provide the plaintext
        .plaintext(plaintext)
        .getBytes();

// decrypt and verify a message
ByteArrayAndResult<DecryptionResult> bytesAndResult = sop.decrypt()
        .withKey(keyBytes)
        .verifyWithCert(certificateBytes)
        .withKeyPassword("f00b4r") // if decryption key is protected
        .ciphertext(ciphertext)
        .toByteArrayAndResult();

DecryptionResult result = bytesAndResult.getResult();
byte[] plaintext = bytesAndResult.getBytes();

The external-sop module will be available on Maven Central in a few hours for you to test.

Happy Hacking!

The KDE Qt5 Patch Collection has been rebased on top of Qt 5.15.8

• TSDgeos' blog
• —
• 23:43, Wednesday, 04 January 2023

Commit: https://invent.kde.org/qt/qt/qt5/-/commit/281044e2541c842f8d0b0bc1a199999bf9d9951c

Commercial release announcement: https://www.qt.io/blog/commercial-lts-qt-5.15.8-released

As usual I want to personally extend my gratitude to the Commercial users of Qt for beta testing Qt 5.15.8 for the rest of us.

The Commercial Qt 5.15.8 release introduced two bugs that have later been fixed. Thanks to that, our Patchset Collection has been able to incorporate the the fix for one of the issues [1] and revert for the other [2]  and the Free Software users will never be affected by it! 

P.S: Special shout-out to Andreas Sturmlechner for identifying the fix of the issue, since I usually only pay attention to "Revert XYZ" commits and this one is not a revert but subsequent improvement

Phosh 2022 in retrospect

• Colors of Noise - Entries tagged planetfsfe
• —
• 15:55, Saturday, 31 December 2022

I wanted to look back at what changed in phosh in 2022 and figured I could share it with you. I'll be focusing on things very close to the mobile shell, for a broader overview see Evangelos upcoming FOSDEM talk.

Some numbers

We're usually aiming for a phosh release at the end of each month. In 2022 We did 10 releases like that, 7 major releases (bumping the middle version number) and three betas. We skipped the April and November releases. We also did one bug fix relesae out of line (bumping the last bit of the version number). I hope we can keep that cadence in 2023 as it allows us to get changes to users in a timely fashion (thus closing usability gaps as early as possible) as well as giving distributions a way to plan ahead. Ideally we'd not skip any release but sometimes real life just interferes.

Those releases contain code contributions from about 20 different people and translations from about 30 translators. These numbers are roughly the same as 2021 which is great. Thanks everyone!

In phosh's git repository we had a bit over 730 non-merge commits (roughly 2 per day), which is about 10% less than in 2021. Looking closer this is easily compensated by commits to phoc (which needed quite some work for the gestures) and phosh-mobile-settings which didn't exist in 2021.

User visible features

Most notable new features are likely the swipe gestures for top and bottom bar, the possibility to use the quick settings on the lock screen as well as the style refresh driven by Sam Hewitt that e.g. touched the modal dialogs (but also sliders, dialpads, etc):

We also added the possibility to have custom widgets via loadable plugins on the lock screen so the user can decide which information should be available. We currently ship plugins to show

• information on upcoming calendar events
• emergency contact information
• PDFs like bus or train tickets
• the current month (as hello world like plugin to get started)

These are maintained within phosh's source tree although out of tree plugins should be doable too.

There's a settings application (the above mentioned phosh-mobile-settings) to enable these. It also allows those plugins to have individual preferences:

Speaking of configurability: Scale-to-fit settings (to work around applications that don't fit the screen) and haptic/led feedback are now configurable without resorting to the command line:

We can also have device specific settings which helps to temporarily accumulate special workaround without affecting other phones.

Other user visible features include the ability to shuffle the digits on the lockscreen's keypad, a VPN quick settings, improved screenshot support and automatic high contrast theme switching when in bright sunlight (based on ambient sensor readings) as shown here.

As mentioned above Evangelos will talk at FOSDEM 2023 about the broader ecosystem improvements including GNOME, GTK, wlroots, phoc, feedbackd, ModemManager, mmsd, NetworkManager and many others without phosh wouldn't be possible.

What else

As I wanted a T-shirt for Debconf 2022 in Prizren so I created a logo heavily inspired by those cute tiger images you often see in Southeast Asia. Based on that I also made a first batch of stickers mostly distributed at FrOSCon 2022:

That's it for 2022. If you want to get involved into phosh testing, development or documentation then just drop by in the matrix room.

Donate to KDE with a 10% power up! (1-week-offer)

• TSDgeos' blog
• —
• 17:24, Friday, 23 December 2022

Hopefully by now, you know that in KDE we are running an End of Year Fundraising campaign.

If you didn't, now you know :)

The campaign has already raised around 16 thousand euros, but there's still a bit to go to the minimum goal of 20 thousand.

So let's spice things up a little, I will donate 10% of every donation you make, you donate 1€, I will donate 0.1€, you donate 100€ I will donate 10€, etc.

I'm placing my maximum total donation amount at (20000-16211.98)/11 = 344.37€, that is if you all donate 3443.7€ (or more), I will donate 344.37€ and we'll reach the 20K goal.

How is this going to work? 

I will make my donation on the 31st of December (just one donation, to save up on fees).

For your donation to be included in my matching donation you need to send me an email to [email protected] with your name/email and put in copy (CC) the KDE e.V. board [email protected] so they can confirm your donation.

Only donations between now (23rd of December around 18:00 CET) and 31st of December will be considered.

sRGB↔L*a*b*↔LCh_ab conversions

• In English — mina86.com
• —
• 01:21, Wednesday, 21 December 2022

This entry includes Maths formulæ which may be rendered incorrectly by your feed reader. You may prefer reading it on the web.

First, rather obvious observation is that shades of grey have the same red, green and blue components. Let’s say that \(C\) is value of those components in linear space.

First observation is that multiplying conversion matrix by column of ones produces coordinates for the reference white point. That is, \(\begin{bmatrix}X_w & Y_w & Z_w\end{bmatrix}^T = M \begin{bmatrix}1 & 1 & 1\end{bmatrix}^T\).

First obvious observation is that for shades of grey, red, green and blue components of the colour are all the same. Let’s call that component \(k\) and with that we can say \(r_8 = g_8 = b_8 = k_8\) and \(R = G = B = K\). This simplifies the sRGB→XYZ formulæ: $$ \begin{align} \begin{bmatrix}X \\ Y \\ Z\end{bmatrix} &= M \begin{bmatrix}C \\ C \\ C\end{bmatrix} = M \begin{bmatrix}1 \\ 1 \\ 1\end{bmatrix} C = \begin{bmatrix}X_w \\ Y_w \\ Z_w\end{bmatrix} C \\[.5em] f_x &= f(X / X_w) = f(X_w C / X_w) = f(C) \\ f_y &= f(Y / Y_w) = f(Y_w C / Y_w) = f(C) \\ f_z &= f(Z / Z_w) = f(Z_w C / Z_w) = f(C) \\ &⇒ f_x = f_y = f_z\\[.5em] a^* &= 500 (f_x - f_y) = 0 \\ b^* &= 200 (f_y - f_z) = 0 \\[.5em] u_w' &= 4X_w / D_w \\ v_w' &= 9Y_w / D_w \\ D_w &= X_w + 15Y_w + 3Z_w\\[.5em] D &= X + 15Y + 3Z = X_w C + 15Y_w C + 3Z_w C = D_w C\\ u' &= 4X / D = 4X_w C / (D_w C) = 4X_w / D_w = u_w'\\ v' &= 9Y / D = 4Y_w C / (D_w C) = 9Y_w / D_w = v_w'\\[.5em] u^* &= 13 L^* (u' - u_w') = 0\\ v^* &= 13 L^* (v' - v_w') = 0\\ \end{align} $$

Chronological order of The Witcher

• In English — mina86.com
• —
• 00:54, Sunday, 11 December 2022

Ever since Witcher games took off the franchise skyrocketed in popularity. Books, comics, TV show, games, more comics, another TV show… The story of Geralt and his marry company has been told in so many ways that it’s becoming a wee bit hard to keep track of chronology of all the events; especially across different forms of media.

In this article I’ve collected all official Witcher works ordering them in chronological order. To avoid any confusion, let me state up front that if you’re new to the franchise or haven’t read the books yet this list might not be for you. If you’re looking for the order to read the books in, I’ve prepared a separate article which describes that.

Skip right to the chronology

Canons

This compilation includes the following works:

• Books refers to the main story as written by Andrzej Sapkowski in the short stories and novels.
• CDPR denotes canon of the video games released by CD Projekt Red. It’s largely consistent with the books. This category includes games, comic books published by Dark Horse and cinematics set in game’s continuum.
• Netflix is continuity of the Netflix TV series and all related spin-offs.
• P&P indicates Polish comic books by BogusÅ‚aw PoÅ‚ch and Maciej Parowski. Except for a single issue (‘Zdrada’), the comics are faithful adaptations of respective short stories.
• Hexer refers to the Polish TV series Wiedźmin released in 2002.

Netflix

Regarding Netflix show. The first season presents split timelines. Each episode can have up to three arcs for each of the main characters: Geralt, Ciri and Yennefer. Stories for each person are presented in chronological order throughout the season, however events between the timelines don’t line up. For example, the second episode shows Yennefer in the year 1206, Geralt in 1240 and Ciri in 1263.

The episodes of the main series are given using SnnEmm notation (indicating episode mm of season nn) rather than using their titles. Furthermore, episodes of the first season have name of the character in parenthesise indicating whose arc the entry refers to. For example, the second episode has three separate entries S01E02 (Yen), S01E02 (Geralt) and S01E02 (Ciri).

Dates of the events in Netflix show are taken from its official website.

Disclaimer

It’s important to note that dates and chronology aren’t always consistent even within a single canon. A common example is Ciri’s date of birth which can be interpreted differently based on different parts of the books.

Furthermore, because of episodic nature of some of the stories, it’s not always possible to order them in relation to other events. For example, A Grain of Truth could take place pretty much at any point in Geralt’s life.

In some cases of ambiguity regarding books and CDPR timelines, I’ve resorted to looking at Witcher fandom wiki.

Lastly, dates between canons aren’t consistent. For example, Geralt and Yennefer meet in 1248 in the books but in 1256 in the Netflix show. The compilation orders entries by ‘events’ rather than by dates.

The chronology

To distinguish different forms of media (books, games, shows etc) the following icons are used next to the titles to identify what they refer to:

• 📜 — short story / 📕 — novel / 🖌 — comic book
• 🕹 — video game / 📱 — mobile game / 🎲 — board game or TTRPG
• 📺 — television series / ã‚¢ — animated film / 🎥 — cinematic trailer

Clicking on canon name in the table’s header allows filtering rows by canon.

Acknowledgments

Thanks to acbagel for pointing out Rogue Mage and Witcher's Journal.

Running Cockpit inside ALP

• Losca
• —
• 13:28, Saturday, 10 December 2022

(quoted from my other blog at since a new OS might be interesting for many and this is published in separate planets)

ALP - The Adaptable Linux Platform – is a new operating system from SUSE to run containerized and virtualized workloads. It is in early prototype phase, but the development is done completely openly so it’s easy to jump in to try it.

For this trying out, I used the latest encrypted build – as of the writing, 22.1 – from ALP images. I imported it in virt-manager as a Generic Linux 2022 image, using UEFI instead of BIOS, added a TPM device (which I’m interested in otherwise) and referring to an Ignition JSON file in the XML config in virt-manager.

The Ignition part is pretty much fully thanks to Paolo Stivanin who studied the secrets of it before me. But here it goes - and this is required for password login in Cockpit to work in addition to SSH key based login to the VM from host - first, create config.ign file:

{
  "ignition": { "version": "3.3.0" },
  "passwd": {
    "users": [
      {
        "name": "root",
        "passwordHash": "YOURHASH",
        "sshAuthorizedKeys": [
          "ssh-... YOURKEY"
        ]
      }
    ]
  },
  "systemd": {
    "units": [{
      "name": "sshd.service",
      "enabled": true
    }]
  },
  "storage": {
    "files": [
      {
        "overwrite": true,
        "path": "/etc/ssh/sshd_config.d/20-enable-passwords.conf",
        "contents": {
          "source": "data:,PasswordAuthentication%20yes%0APermitRootLogin%20yes%0A"
        },
        "mode": 420
      }
    ]
  }
}

…where password SHA512 hash can be obtained using openssl passwd -6 and the ssh key is your public ssh key.

That file is put to eg /tmp and referred in the virt-manager’s XML like follows:

  <sysinfo type="fwcfg">
    <entry name="opt/com.coreos/config" file="/tmp/config.ign"/>
  </sysinfo>

Now we can boot up the VM and ssh in - or you could log in directly too but it’s easier to copy-paste commands when using ssh.

Inside the VM, we can follow the ALP documentation to install and start Cockpit:

podman container runlabel install registry.opensuse.org/suse/alp/workloads/tumbleweed_containerfiles/suse/alp/workloads/cockpit-ws:latest
podman container runlabel --name cockpit-ws run registry.opensuse.org/suse/alp/workloads/tumbleweed_containerfiles/suse/alp/workloads/cockpit-ws:latest
systemctl enable --now cockpit.service

Check your host’s IP address with ip -a, and open IP:9090 in your host’s browser:

Login with root / your password and you shall get the front page:

…and many other pages where you can manage your ALP deployment via browser:

All in all, ALP is in early phases but I’m really happy there’s up-to-date documentation provided and people can start experimenting it whenever they want. The images from the linked directory should be fairly good, and test automation with openQA has been started upon as well.

You can try out the other example workloads that are available just as well.

URLs with // at the beginning

• In English — mina86.com
• —
• 02:09, Sunday, 04 December 2022

Just a quick reminder that relative URLs can start with a double slash and that this means something different than a single slash at the beginning. Specifically, such relative addresses are resolved by taking the schema (and only the schema) of the website they are on.

For example, the code for the link to my repositories in the site’s header is <a href="//github.com/mina86">Code</a>. Since this page uses https schema, browsers will navigate to https://github.com/mina86 if the link is activated.

This little trick can save you some typing, but more importantly, if you’re developing a URL parsing code or a crawler, make sure that it handles this case correctly. It may seem like a small detail, but it can have a big impact on the functionality of your code.

BTRFS Snapshot Cron Script

• Evaggelos Balaskas - System Engineer
• —
• 18:49, Sunday, 20 November 2022

I’ve been using btrfs for a decade now (yes, than means 10y) on my setup (btw I use ArchLinux). I am using subvolumes and read-only snapshots with btrfs, but I have never created a script to automate my backups.

I KNOW, WHAT WAS I DOING ALL THESE YEARS!!

A few days ago, a dear friend asked me something about btrfs snapshots, and that question gave me the nudge to think about my btrfs subvolume snapshots and more specific how to automate them. A day later, I wrote a simple (I think so) script to do automate my backups.

The script as a gist

The script is online as a gist here: BTRFS: Automatic Snapshots Script . In this blog post, I’ll try to describe the requirements and what is my thinking. I waited a couple weeks so the cron (or systemd timer) script run itself and verify that everything works fine. Seems that it does (at least for now) and the behaviour is as expected. I will keep a static copy of my script in this blog post but any future changes should be done in the above gist.

Improvements

The script can be improved by many,many ways (check available space before run, measure the time of running, remove sudo, check if root is running the script, verify the partitions are on btrfs, better debugging, better reporting, etc etc). These are some of the ways of improving the script, I am sure you can think a million more - feel free to sent me your proposals. If I see something I like, I will incorporate them and attribute of-course. But be reminded that I am not driven by smart code, I prefer to have clear and simple code, something that everybody can easily read and understand.

Mount Points

To be completely transparent, I encrypt all my disks (usually with a random keyfile). I use btrfs raid1 on the disks and create many subvolumes on them. Everything exists outside of my primary ssd rootfs disk. So I use a small but fast ssd for my operating system and btrfs-raid1 for my “spinning rust” disks.

BTRFS subvolumes can be mounted as normal partitions and that is exactly what I’ve done with my home and opt. I keep everything that I’ve install outside of my distribution under opt.

This setup is very flexible as I can easy replace the disks when the storage is full by removing one by one of the disks from btrfs-raid1, remove-add the new larger disk, repair-restore raid, then remove the other disk, install the second and (re)balance the entire raid1 on them!

Although this is out of scope, I use a stub archlinux UEFI kernel so I do not have grub and my entire rootfs is also encrypted and btrfs!

mount -o subvolid=10701 LABEL="ST1000DX002" /home
mount -o subvolid=10657 LABEL="ST1000DX002" /opt

Declare variables

# paths MUST end with '/'
btrfs_paths=("/" "/home/" "/opt/")
timestamp=$(date +%Y%m%d_%H%M%S)
keep_snapshots=3
yymmdd="$(date +%Y/%m/%d)"
logfile="/var/log/btrfsSnapshot/${yymmdd}/btrfsSnapshot.log"

The first variable in the script is actually a bash array

btrfs_paths=("/" "/home/" "/opt/")

and all three (3) paths (rootfs, home & opt) are different mount points on different encrypted disks.

MUST end with / (forward slash), either-wise something catastrophic will occur to your system. Be very careful. Please, be very careful!

Next variable is the timestamp we will use, that will create something like

partition_YYYYMMDD_HHMMSS

After that is how many snapshots we would like to keep to our system. You can increase it to whatever you like. But be careful of the storage.

keep_snapshots=3

I like using shortcuts in shell scripts to reduce the long one-liners that some people think that it is alright. I dont, so

yymmdd="$(date +%Y/%m/%d)"

is one of these shortcuts !

Last, I like to have a logfile to review at a later time and see what happened.

logfile="/var/log/btrfsSnapshot/${yymmdd}/btrfsSnapshot.log"

Log Directory

for older dudes -like me- you know that you can not have all your logs under one directory but you need to structure them. The above yymmdd shortcut can help here. As I am too lazy to check if the directory already exist, I just (re)create the log directory that the script will use.

sudo mkdir -p "/var/log/btrfsSnapshot/${yymmdd}/"

For - Loop

We enter to the crucial part of the script. We are going to iterate our btrfs commands in a bash for-loop structure so we can run the same commands for all our partitions (variable: btrfs_paths)

for btrfs_path in "${btrfs_paths[@]}"; do
    <some commands>
done

Snapshot Directory

We need to have our snapshots in a specific location. So I chose .Snapshot/ under each partition. And I am silently (re)creating this directory -again I am lazy, someone should check if the directory/path already exist- just to be sure that the directory exist.

sudo mkdir -p "${btrfs_path}".Snapshot/

I am also using very frequently mlocate (updatedb) so to avoid having multiple (duplicates) in your index, do not forget to update updatedb.conf to exclude the snapshot directories.

PRUNENAMES = ".Snapshot"

How many snapshots are there?

Yes, how many ?

In order to learn this, we need to count them. I will try to skip every other subvolume that exist under the path and count only the read-only, snapshots under each partition.

sudo btrfs subvolume list -o -r -s "${btrfs_path}" | grep -c ".Snapshot/"

Delete Previous snapshots

At this point in the script, we are ready to delete all previous snapshots and only keep the latest or to be exact whatever the keep_snapshots variables says we should keep.

To do that, we are going to iterate via a while-loop (this is a nested loop inside the above for-loop)

while [ "${keep_snapshots}" -le "${list_btrfs_snap}" ]
do
  <some commands>
done

considering that the keep_snapshots is an integer, we iterate the delete command less or equal from the list of already btrfs existing snapshots.

Delete Command

To avoid mistakes, we delete by subvolume id and not by the name of the snapshot, under the btrfs path we listed above.

btrfs subvolume delete --subvolid "${prev_btrfs_snap}" "${btrfs_path}"

and we log the output of the command into our log

Delete subvolume (no-commit): '//.Snapshot/20221107_091028'

Create a new subvolume snapshot

And now we are going to create a new read-only snapshot under our btrfs subvolume.

btrfs subvolume snapshot -r "${btrfs_path}" "${btrfs_path}.Snapshot/${timestamp}"

the log entry will have something like:

Create a readonly snapshot of '/' in '/.Snapshot/20221111_000001'

That’s it !

Output

Log Directory Structure and output

sudo tree /var/log/btrfsSnapshot/2022/11/

/var/log/btrfsSnapshot/2022/11/
├── 07
│   └── btrfsSnapshot.log
├── 10
│   └── btrfsSnapshot.log
├── 11
│   └── btrfsSnapshot.log
└── 18
    └── btrfsSnapshot.log

4 directories, 4 files

sudo cat /var/log/btrfsSnapshot/2022/11/18/btrfsSnapshot.log

######## Fri, 18 Nov 2022 00:00:01 +0200 ########

Delete subvolume (no-commit): '//.Snapshot/20221107_091040'
Create a readonly snapshot of '/' in '/.Snapshot/20221118_000001'

Delete subvolume (no-commit): '/home//home/.Snapshot/20221107_091040'
Create a readonly snapshot of '/home/' in '/home/.Snapshot/20221118_000001'

Delete subvolume (no-commit): '/opt//opt/.Snapshot/20221107_091040'
Create a readonly snapshot of '/opt/' in '/opt/.Snapshot/20221118_000001'

Mount a read-only subvolume

As something extra for this article, I will mount a read-only subvolume, so you can see how it is done.

$ sudo btrfs subvolume list -o -r -s /

ID 462 gen 5809766 cgen 5809765 top level 5 otime 2022-11-10 18:11:20 path .Snapshot/20221110_181120
ID 463 gen 5810106 cgen 5810105 top level 5 otime 2022-11-11 00:00:01 path .Snapshot/20221111_000001
ID 464 gen 5819886 cgen 5819885 top level 5 otime 2022-11-18 00:00:01 path .Snapshot/20221118_000001

$ sudo mount -o subvolid=462 /media/
mount: /media/: can't find in /etc/fstab.

$ sudo mount -o subvolid=462 LABEL=rootfs /media/

$ df -HP /media/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/ssd  112G  9.1G  102G   9% /media

$ sudo touch /media/etc/ebal
touch: cannot touch '/media/etc/ebal': Read-only file system

$ sudo diff /etc/pacman.d/mirrorlist /media/etc/pacman.d/mirrorlist

294c294
< Server = http://ftp.ntua.gr/pub/linux/archlinux/$repo/os/$arch
---
> #Server = http://ftp.ntua.gr/pub/linux/archlinux/$repo/os/$arch

$ sudo umount /media

The Script

Last, but not least, the full script as was the date of this article.

#!/bin/bash
set -e

# ebal, Mon, 07 Nov 2022 08:49:37 +0200

## 0 0 * * Fri /usr/local/bin/btrfsSnapshot.sh

# paths MUST end with '/'
btrfs_paths=("/" "/home/" "/opt/")
timestamp=$(date +%Y%m%d_%H%M%S)
keep_snapshots=3
yymmdd="$(date +%Y/%m/%d)"
logfile="/var/log/btrfsSnapshot/${yymmdd}/btrfsSnapshot.log"

sudo mkdir -p "/var/log/btrfsSnapshot/${yymmdd}/"

echo "######## $(date -R) ########" | sudo tee -a "${logfile}"
echo "" | sudo tee -a "${logfile}"

for btrfs_path in "${btrfs_paths[@]}"; do

    ## Create Snapshot directory
    sudo mkdir -p "${btrfs_path}".Snapshot/

    ## How many Snapshots are there ?
    list_btrfs_snap=$(sudo btrfs subvolume list -o -r -s "${btrfs_path}" | grep -c ".Snapshot/")

    ## Get oldest rootfs btrfs snapshot
    while [ "${keep_snapshots}" -le "${list_btrfs_snap}" ]
    do
        prev_btrfs_snap=$(sudo btrfs subvolume list -o -r -s  "${btrfs_path}" | grep ".Snapshot/" | sort | head -1 | awk '{print $2}')

        ## Delete a btrfs snapshot by their subvolume id
        sudo btrfs subvolume delete --subvolid "${prev_btrfs_snap}" "${btrfs_path}" | sudo tee -a "${logfile}"

        list_btrfs_snap=$(sudo btrfs subvolume list -o -r -s "${btrfs_path}" | grep -c ".Snapshot/")
    done

    ## Create a new read-only btrfs snapshot
    sudo btrfs subvolume snapshot -r "${btrfs_path}" "${btrfs_path}.Snapshot/${timestamp}" | sudo tee -a "${logfile}"

    echo "" | sudo tee -a "${logfile}"

done

Secret command Google doesn’t want you to know

• In English — mina86.com
• —
• 17:55, Sunday, 20 November 2022

Or how to change language of Google website.

If you’ve travelled abroad, you may have noticed that Google tries to be helpful and uses the language of the region you’re in on its websites. It doesn’t matter if your operating system is set to Spanish, for example; Google Search will still use Portuguese if you happen to be in Brazil.

Fortunately, there’s a simple way to force Google to use a specific language. All you need to do is append ?hl=lang to the website’s address, replacing lang with a two-letter code for the desired language. For instance, ?hl=es for Spanish, ?hl=ht for Haitian, or ?hl=uk for Ukrainian.

If the URL already contains a question mark, you need to append &hl=lang instead. Additionally, if it contains a hash symbol, you need to insert the string just before the hash symbol. For example:

• https://www.google.com/?hl=es
• https://www.google.com/search?q=bread+sandwich&hl=es
• https://analytics.google.com/analytics/web/?hl=es#/report-home/

By the way, as a legacy of Facebook having hired many ex-Google employees, the parameter also work on some of the Facebook properties.

This trick doesn’t work on all Google properties. However, it seems to be effective on pages that try to guess your language preference without giving you the option to override it. For example, while Gmail ignores the parameter, you can change its display language in the settings (accessible via the gear icon near the top right of the page). Similarly, YouTube strips the parameter, but it respects preferences configured in the web browser.

Anyone familiar with HTTP may wonder why Google doesn’t just look at the Accept-Language header. The issue is that many users have their browser configured with defaults that send English as the only option, even though they would prefer another language. In those cases, it’s more user-friendly to ignore that header. As it turns out, localisation is really hard.

Baking Qemu KVM Snapshot to Base Image

• Evaggelos Balaskas - System Engineer
• —
• 22:17, Saturday, 19 November 2022

When creating a new Cloud Virtual Machine the cloud provider is copying a virtual disk as the base image (we called it mí̱tra or matrix) and starts your virtual machine from another virtual disk (or volume cloud disk) that in fact is a snapshot of the base image.

Just for the sake of this example, let us say that the base cloud image is the

jammy-server-cloudimg-amd64.img

When creating a new Libvirt (qemu/kvm) virtual machine, you can use this base image to start your VM instead of using an iso to install ubuntu 22.04 LTS. When choosing this image, then all changes will occur to that image and if you want to spawn another virtual machine, you need to (re)download it.

So instead of doing that, the best practice is to copy this image as base and start from a snapshot aka a baking file from that image. It is best because you can always quickly revert all your changes and (re)spawn the VM from the fresh/clean base image. Or you can always create another snapshot and revert if needed.

inspect images

To see how that works here is a local example from my linux machine.

qemu-img info /var/lib/libvirt/images/lEvXLA_tf-base.qcow2

image: /var/lib/libvirt/images/lEvXLA_tf-base.qcow2
file format: qcow2
virtual size: 2.2 GiB (2361393152 bytes)
disk size: 636 MiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

the most important attributes to inspect are

virtual size: 2.2 GiB
disk size: 636 MiB

and the volume disk of my virtual machine

qemu-img info /var/lib/libvirt/images/lEvXLA_tf-vol.qcow2

image: /var/lib/libvirt/images/lEvXLA_tf-vol.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.6 GiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/lEvXLA_tf-base.qcow2
backing file format: qcow2
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

We see here

virtual size: 10 GiB
disk size: 1.6 GiB

cause I have extended the volume disk size to 10G from 2.2G , doing some updates and install some packages.

Now here is a problem.

I would like to use my own cloud image as base for some projects. It will help me speed things up and also do some common things I am usually doing in every setup.

If I copy the volume disk, then I will copy 1.6G of the snapshot disk. I can not use this as a base image. The volume disk contains only the delta from the base image!

baking file

Let’s first understand a bit better what is happening here

qemu-img info –backing-chain /var/lib/libvirt/images/lEvXLA_tf-vol.qcow2

image: /var/lib/libvirt/images/lEvXLA_tf-vol.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.6 GiB
cluster_size: 65536
backing file: /var/lib/libvirt/images/lEvXLA_tf-base.qcow2
backing file format: qcow2
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

image: /var/lib/libvirt/images/lEvXLA_tf-base.qcow2
file format: qcow2
virtual size: 2.2 GiB (2361393152 bytes)
disk size: 636 MiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

By inspecting the volume disk, we see that this image is chained to our base image.

disk size: 1.6 GiB
disk size: 636 MiB

Commit Volume

If we want to commit our volume changes to our base images, we need to commit them.

sudo qemu-img commit /var/lib/libvirt/images/lEvXLA_tf-vol.qcow2

Image committed.

Be aware, we commit our changes the volume disk => so our base will get the updates !!

Base Image

We need to see our base image grow we our changes

  disk size: 1.6 GiB
+ disk size: 636 MiB
=
  disk size: 2.11 GiB

and we can verify that by getting the image info (details)

qemu-img info /var/lib/libvirt/images/lEvXLA_tf-base.qcow2

image: /var/lib/libvirt/images/lEvXLA_tf-base.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 2.11 GiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

That’s it !

GitLab as a Terraform state backend

• Evaggelos Balaskas - System Engineer
• —
• 14:30, Friday, 11 November 2022

Using Terraform for personal projects, is a good way to create your lab in a reproducible manner. Wherever your lab is, either in the “cloud” aka other’s people computers or in a self-hosted environment, you can run your Infrastructure as code (IaC) instead of performing manual tasks each time.

My preferable way is to use QEMU/KVM (Kernel Virtual Machine) on my libvirt (self-hosted) lab. You can quickly build a k8s cluster or test a few virtual machines with different software, without paying extra money to cloud providers.

Terraform uses a state file to store your entire infra in json format. This file will be the source of truth for your infrastructure. Any changes you make in the code, terraform will figure out what needs to add/destroy and run only what have changed.

Working in a single repository, terraform will create a local state file on your working directory. This is fast and reliable when working alone. When working with a team (either in an opensource project/service or it is something work related) you need to share the state with others. Eitherwise the result will be catastrophic as each person will have no idea of the infrastructure state of the service.

In this blog post, I will try to explain how to use GitLab to store the terraform state into a remote repository by using the tf backend: http which is REST.

Greate a new private GitLab Project

We need the Project ID which is under the project name in the top.

Create a new api token

Verify that your Project has the ability to store terraform state files

You are ready to clone the git repository to your system.

Backend

Reading the documentation in the below links

• GitLab-managed Terraform state
• Backend Type: http

seems that the only thing we need to do, is to expand our terraform project with this:

terraform {
  backend "http" {
  }
}

Doing that, we inform our IaC that our terraform backend should be a remote address.

Took me a while to figure this out, but after re-reading all the necessary documentation materials the idea is to declare your backend on gitlab and to do this, we need to initialize the http backend.

The only Required configuration setting is the remote address and should be something like this:

terraform {
  backend "http" {
    address = "https://gitlab.com/api/v4/projects/<PROJECT_ID>/terraform/state/<STATE_NAME>"
  }
}

Where PROJECT_ID and STATE_NAME are relative to your project.

In this article, we go with

GITLAB_PROJECT_ID="40961586"
GITLAB_TF_STATE_NAME="tf_state"

Terraform does not allow to use variables in the backend http, so the preferable way is to export them to our session.

and we -of course- need the address:

TF_HTTP_ADDRESS="https://gitlab.com/api/v4/projects/${GITLAB_PROJECT_ID}/terraform/state/${GITLAB_TF_STATE_NAME}"

For convience reasons, I will create a file named: terraform.config outside of this git repo

cat > ../terraform.config <<EOF
export -p GITLAB_PROJECT_ID="40961586"
export -p GITLAB_TF_STATE_NAME="tf_state"
export -p GITLAB_URL="https://gitlab.com/api/v4/projects"

# Address
export -p TF_HTTP_ADDRESS="${GITLAB_URL}/${GITLAB_PROJECT_ID}/terraform/state/${GITLAB_TF_STATE_NAME}"

EOF

source ../terraform.config

this should do the trick.

Authentication

In order to authenticate via tf against GitLab to store the tf remote state, we need to also set two additional variables:

# Authentication
TF_HTTP_USERNAME="api"
TF_HTTP_PASSWORD="<TOKEN>"

put them in the above terraform.config file.

Pretty much we are done!

Initialize Terraform

source ../terraform.config 

terraform init

Initializing the backend...

Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding latest version of hashicorp/http...
- Finding latest version of hashicorp/random...
- Finding latest version of hashicorp/template...
- Finding dmacvicar/libvirt versions matching ">= 0.7.0"...
- Installing hashicorp/random v3.4.3...
- Installed hashicorp/random v3.4.3 (signed by HashiCorp)
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)
- Installing dmacvicar/libvirt v0.7.0...
- Installed dmacvicar/libvirt v0.7.0 (unauthenticated)
- Installing hashicorp/http v3.2.1...
- Installed hashicorp/http v3.2.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Remote state

by running

terraform plan

we can now see the remote terraform state in the gitlab

Opening Actions –> Copy terraform init command we can see the below configuration:

export GITLAB_ACCESS_TOKEN=<YOUR-ACCESS-TOKEN>

terraform init
    -backend-config="address=https://gitlab.com/api/v4/projects/40961586/terraform/state/tf_state"
    -backend-config="lock_address=https://gitlab.com/api/v4/projects/40961586/terraform/state/tf_state/lock"
    -backend-config="unlock_address=https://gitlab.com/api/v4/projects/40961586/terraform/state/tf_state/lock"
    -backend-config="username=api"
    -backend-config="password=$GITLAB_ACCESS_TOKEN"
    -backend-config="lock_method=POST"
    -backend-config="unlock_method=DELETE"
    -backend-config="retry_wait_min=5"

Update terraform backend configuration

I dislike running a “long” terraform init command, so we will put these settings to our tf code.

Separating the static changes from the dynamic, our Backend http config can become something like this:

terraform {
  backend "http" {
    lock_method    = "POST"
    unlock_method  = "DELETE"
    retry_wait_min = 5
  }
}

but we need to update our terraform.config once more, to include all the variables of the http backend configuration for locking and unlocking the state.

# Lock
export -p TF_HTTP_LOCK_ADDRESS="${TF_HTTP_ADDRESS}/lock"

# Unlock
export -p TF_HTTP_UNLOCK_ADDRESS="${TF_HTTP_ADDRESS}/lock"

Terraform Config

So here is our entire terraform config file

# GitLab

export -p GITLAB_URL="https://gitlab.com/api/v4/projects"
export -p GITLAB_PROJECT_ID="<>"
export -p GITLAB_TF_STATE_NAME="tf_state"

# Terraform

# Address
export -p TF_HTTP_ADDRESS="${GITLAB_URL}/${GITLAB_PROJECT_ID}/terraform/state/${GITLAB_TF_STATE_NAME}"

# Lock
export -p TF_HTTP_LOCK_ADDRESS="${TF_HTTP_ADDRESS}/lock"

# Unlock
export -p TF_HTTP_UNLOCK_ADDRESS="${TF_HTTP_ADDRESS}/lock"

# Authentication
export -p TF_HTTP_USERNAME="api"
export -p TF_HTTP_PASSWORD="<>"

And pretty much that’s it!

Other Colleagues

So in order our team mates/colleagues want to make changes to this specific gitlab repo (or even extended to include a pipeline) they need

1. Git clone the repo
2. Edit the terraform.config
3. Initialize terraform (terraform init)

And terraform will use the remote state file.

KDE Gear 22.12 branches created

• TSDgeos' blog
• —
• 20:38, Saturday, 05 November 2022

Make sure you commit anything you want to end up in the KDE Gear 22.12 releases to them

We're already past the dependency freeze.

The Feature Freeze and Beta is next week Thursday 10 of November.

More interesting dates
 November 24, 2022: 22.12 RC (22.11.90) Tagging and Release
 December 1, 2022: 22.12 Tagging
 December 8, 2022: 22.12 Release

https://community.kde.org/Schedules/KDE_Gear_22.12_Schedule

The KDE Qt5 Patch Collection has been rebased on top of Qt 5.15.7

• TSDgeos' blog
• —
• 21:39, Friday, 28 October 2022

 Commit: https://invent.kde.org/qt/qt/qt5/-/commit/17246d90b97ef6ecc05cb225bc576b4fb30ca77a

Commercial release announcement: https://www.qt.io/blog/commercial-lts-qt-5.15.7-released 

As usual I want to personally extend my gratitude to the Commercial users of Qt for beta testing Qt 5.15.7 for the rest of us.

Implementing Packet Sequence Validation using Pushdown Automata

• foss – vanitasvitae's blog
• —
• 16:14, Wednesday, 26 October 2022

This is part 2 of a small series on verifying the validity of packet sequences using tools from theoretical computer science. Read part 1 here.

In the previous blog post I discussed how a formal grammar can be transformed into a pushdown automaton in order to check if a sequence of packets or tokens is part of the language described by the grammar. In this post I will discuss how I implemented said automaton in Java in order to validate OpenPGP messages in PGPainless.

In the meantime, I made some slight changes to the automaton and removed some superfluous states. My current design of the automaton looks as follows:

If you compare this diagram to the previous iteration, you can see that I got rid of the states “Signed Message”, “One-Pass-Signed Message” and “Corresponding Signature”. Those were states which had ε-transitions to another state, so they were not really useful.

For example, the state “One-Pass-Signed Message” would only be entered when the input “OPS” was read and ‘m’ could be popped from the stack. After that, there would only be a single applicable rule which would read no input, pop nothing from the stack and instead push back ‘m’. Therefore, these two rule could be combined into a single rule which reads input “OPS”, pops ‘m’ from the stack and immediately pushes it back onto it. This rule would leave the automaton in state “OpenPGP Message”. Both automata are equivalent.

One more minor detail: Since I am using Bouncy Castle, I have to deal with some of its quirks. One of those being that BC bundles together encrypted session keys (PKESKs/SKESKs) with the actual encrypted data packets (SEIPD/SED). Therefore when implementing, we can further simplify the diagram by removing the SKESK|PKESK parts:

Now, in order to implement this automaton in Java, I decided to define enums for the input and stack alphabets, as well as the states:

public enum InputAlphabet {
    LiteralData,
    Signature,            // Sig
    OnePassSignature,     // OPS
    CompressedData,
    EncryptedData,        // SEIPD|SED
    EndOfSequence         // End of message/nested data
}

public enum StackAlphabet {
    msg,                 // m
    ops,                 // o
    terminus             // #
}

public enum State {
    OpenPgpMessage,
    LiteralMessage,
    CompressedMessage,
    EncryptedMessage,
    Valid
}

Note, that there is no “Start” state, since we will simply initialize the automaton in state OpenPgpMessage, with ‘m#’ already put on the stack.

We also need an exception class that we can throw when OpenPGP packet is read when its not allowed. Therefore I created a MalformedOpenPgpMessageException class.

Now the first design of our automaton itself is pretty straight forward:

public class PDA {
    private State state;
    private final Stack<StackAlphabet> stack = new Stack<>();
    
    public PDA() {
        state = State.OpenPgpMessage;    // initial state
        stack.push(terminus);            // push '#'
        stack.push(msg);                 // push 'm'
    }

    public void next(InputAlphabet input)
            throws MalformedOpenPgpMessageException {
        // TODO: handle the next input packet
    }

    StackAlphabet popStack() {
        if (stack.isEmpty()) {
            return null;
        }
        return stack.pop();
    }

    void pushStack(StackAlphabet item) {
        stack.push(item);
    }

    boolean isEmptyStack() {
        return stack.isEmpty();
    }

    public boolean isValid() {
        return state == State.Valid && isEmptyStack();
    }
}

As you can see, we initialize the automaton with a pre-populated stack and an initial state. The automatons isValid() method only returns true, if the automaton ended up in state “Valid” and the stack is empty.

Whats missing is an implementation of the transition rules. I found it most straight forward to implement those inside the State enum itself by defining a transition() method:

public enum State {

    OpenPgpMessage {
        @Overrides
        public State transition(InputAlphabet input, PDA automaton)
                throws MalformedOpenPgpMessageException {
            StackAlphabet stackItem = automaton.popStack();
            if (stackItem != OpenPgpMessage) {
                throw new MalformedOpenPgpMessageException();
            }
            swith(input) {
                case LiteralData:
                    // Literal Packet,m/ε
                    return LiteralMessage;
                case Signature:
                    // Sig,m/m
                    automaton.pushStack(msg);
                    return OpenPgpMessage;
                case OnePassSignature:
                    // OPS,m/mo
                    automaton.push(ops);
                    automaton.push(msg);
                    return OpenPgpMessage;
                case CompressedData:
                    // Compressed Data,m/ε
                    return CompressedMessage;
                case EncryptedData:
                    // SEIPD|SED,m/ε
                    return EncryptedMessage;
                case EndOfSequence:
                default:
                    // No transition
                    throw new MalformedOpenPgpMessageException();
            }
        }
    },

    LiteralMessage {
        @Overrides
        public State transition(InputAlphabet input, PDA automaton)
                throws MalformedOpenPgpMessageException {
            StackAlphabet stackItem = automaton.popStack();
            switch(input) {
                case Signature:
                    if (stackItem == ops) {
                        // Sig,o/ε
                        return LiteralMessage;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                case EndOfSequence:
                    if (stackItem == terminus && automaton.isEmptyStack()) {
                        // ε,#/ε
                        return valid;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                default:
                    throw new MalformedOpenPgpMessageException();
            }
        }
    },

    CompressedMessage {
        @Overrides
        public State transition(InputAlphabet input, PDA automaton)
                throws MalformedOpenPgpMessageException {
            StackAlphabet stackItem = automaton.popStack();
            switch(input) {
                case Signature:
                    if (stackItem == ops) {
                        // Sig,o/ε
                        return CompressedMessage;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                case EndOfSequence:
                    if (stackItem == terminus && automaton.isEmptyStack()) {
                        // ε,#/ε
                        return valid;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                default:
                    throw new MalformedOpenPgpMessageException();
            }
        }
    },

    EncryptedMessage {
        @Overrides
        public State transition(InputAlphabet input, PDA automaton)
                throws MalformedOpenPgpMessageException {
            StackAlphabet stackItem = automaton.popStack();
            switch(input) {
                case Signature:
                    if (stackItem == ops) {
                        // Sig,o/ε
                        return EncryptedMessage;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                case EndOfSequence:
                    if (stackItem == terminus && automaton.isEmptyStack()) {
                        // ε,#/ε
                        return valid;
                    } else {
                        throw new MalformedOpenPgpMessageException();
                    }
                default:
                    throw new MalformedOpenPgpMessageException();
            }
        }
    },

    Valid {
        @Overrides
        public State transition(InputAlphabet input, PDA automaton)
                throws MalformedOpenPgpMessageException {
            // Cannot transition out of Valid state
            throw new MalformedOpenPgpMessageException();
        }
    }
    ;

    abstract State transition(InputAlphabet input, PDA automaton)
            throws MalformedOpenPgpMessageException;
}

It might make sense to define the transitions in an external class to allow for different grammars and to remove the dependency on the PDA class, but I do not care about this for now, so I’m fine with it.

Now every State has a transition() method, which takes an input symbol and the automaton itself (for access to the stack) and either returns the new state, or throws an exception in case of an illegal token.

Next, we need to modify our PDA class, so that the new state is saved:

public class PDA {
    [...]

    public void next(InputAlphabet input)
            throws MalformedOpenPgpMessageException {
        state = state.transition(input, this);
    }
}

Now we are able to verify simple packet sequences by feeding them one-by-one to the automaton:

// LIT EOS
PDA pda = new PDA();
pda.next(LiteralData);
pda.next(EndOfSequence);
assertTrue(pda.isValid());

// OPS LIT SIG EOS
pda = new PDA();
pda.next(OnePassSignature);
pda.next(LiteralData);
pda.next(Signature);
pda.next(EndOfSequence);
assertTrue(pda.isValid());

// COMP EOS
PDA pda = new PDA();
pda.next(CompressedData);
pda.next(EndOfSequence);
assertTrue(pda.isValid());

You might say “Hold up! The last example is a clear violation of the syntax! A compressed data packet alone does not make a valid OpenPGP message!”.

And you are right. A compressed data packet is only a valid OpenPGP message, if its decompressed contents also represent a valid OpenPGP message. Therefore, when using our PDA class, we need to take care of packets with nested streams separately. In my implementation, I created an OpenPgpMessageInputStream, which among consuming the packet stream, handling the actual decryption, decompression etc. also takes care for handling nested PDAs. I will not go into too much details, but the following code should give a good idea of the architecture:

public class OpenPgpMessageInputStream {
    private final PDA pda = new PDA();
    private BCPGInputStream pgpIn = ...; // stream of OpenPGP packets
    private OpenPgpMessageInputStream nestedStream;

    public OpenPgpMessageInputStream(BCPGInputStream pgpIn) {
        this.pgpIn = pgpIn;
        switch(pgpIn.nextPacketTag()) {
            case LIT:
                pda.next(LiteralData);
                ...
                break;
            case COMP:
                pda.next(CompressedData);
                nestedStream = new OpenPgpMessageInputStream(decompress());
                ...
                break;
            case OPS:
                pda.next(OnePassSignature);
                ...
                break;
            case SIG:
                pda.next(Signature);
                ...
                break;
            case SEIPD:
            case SED:
                pda.next(EncryptedData);
                nestedStream = new OpenPgpMessageInputStream(decrypt());
                ...
                break;
            default:
                // Unknown / irrelevant packet
                throw new MalformedOpenPgpMessageException();
    }

    boolean isValid() {
        return pda.isValid() &&
               (nestedStream == null || nestedStream.isValid());

    @Override
    close() {
        if (!isValid()) {
            throw new MalformedOpenPgpMessageException();
        }
        ...
    }
}

The key thing to take away here is, that when we encounter a nesting packet (EncryptedData, CompressedData), we create a nested OpenPgpMessageInputStream on the decrypted / decompressed contents of this packet. Once we are ready to close the stream (because we reached the end), we not only check if our own PDA is in a valid state, but also whether the nestedStream (if there is one) is valid too.

This code is of course only a rough sketch and the actual implementation is far more complex to cover many possible edge cases. Yet, it still should give a good idea of how to use pushdown automata to verify packet sequences Feel free to check out my real-world implementation here and here.

Happy Hacking!

KDE Gear 22.12 release schedule finalized

• TSDgeos' blog
• —
• 18:07, Friday, 21 October 2022

This is the release schedule the release team agreed on

https://community.kde.org/Schedules/KDE_Gear_22.12_Schedule

Dependency freeze is in TWO weeks (November 3) and feature freeze one after that. 

Get your stuff ready!

Automatically delete files in object storage

• Evaggelos Balaskas - System Engineer
• —
• 12:19, Friday, 21 October 2022

In the last few months of this year, a business question exists in all our minds:

-Can we reduce Cost ?
-Are there any legacy cloud resources that we can remove ?

The answer is YES, it is always Yes. It is part of your Technical Debt (remember that?).

In our case we had to check a few cloud resources, but the most impressive were our Object Storage Service that in the past we were using Buckets and Objects as backup volumes … for databases … database clusters!!

So let’s find out what is our Techinical Debt in our OBS … ~ 1.8PB . One petabyte holds 1000 terabytes (TB), One terabyte holds 1000 gigabytes (GB).

We have confirmed with our colleagues and scheduled the decomissions of these legacy buckets/objects. We’ve noticed that a few of them are in TB sizes with million of objects and in some cases with not a hierarchy structure (paths) so there is an issue with command line tools or web UI tools.

The problem is called LISTING and/or PAGING.

That means we can not list in a POSIX way (nerdsssss) all our objects so we can try delete them. We need to PAGE them in 50 or 100 objects and that means we need to batch our scripts. This could be a very long/time based job.

Then after a couple days of reading manuals (remember these ?), I found that we can create a Lifecycle Policy to our Buckets !!!

But before going on to setup the Lifecycle policy, just a quick reminder how the Object Lifecycle works

The objects can be in Warm/Cold or in Expired State as buckets support versioning. This has to do with the retention policy of logs/data.

So in order to automatically delete ALL objects from a bucket, we need to setup up the Expire Time to 1 day.

Then you have to wait for 24h and next day

yehhhhhhhhhhhh !

PS. Caveat Remember BEFORE all that, do disable Logging as the default setting is to log every action to a local log path, inside the Bucket.

Come to Barcelona for Akademy-es 2022!

• TSDgeos' blog
• —
• 09:12, Saturday, 17 September 2022

As previously announced, Akademy 2022 will be happening in Barcelona at the beginning of October.

On top of that, Akademy-es [the Spain spin-off of Akademy] is also happening in Barcelona the days before (29 and 30 of September). So if you're interested in KDE and understand Spanish a bit, please drop by https://www.kde-espana.org/akademy-es-2022 and register yourself at https://www.kde-espana.org/akademy-es2022/registro.php

There's also a quite pretty t-shirt you can order but if you want it, you have to register BEFORE the end of the weekend if you want it!

5 Years of Freedom with OpenPOWER

• tobias_platen's blog
• —
• 14:23, Thursday, 15 September 2022

5 years ago I preorded my Talos II from Raptor Computing Systems. The Talos II is a POWERful system built from the ground up with freedom in mind. In one of its PCIe 4.0 slots, I plugged an AMD Radeon RX 5700 (Navi 10) which I mainly use for playing VR games, but also for multi monitor setups, faster video decoding and many more. Unfortunately all modern graphics cards require non-free firmware, but currently the libre-soc project is developing an OpenPOWER hybrid CPU/VPU/GPU that comes with its own Vulkan drivers.

Currently the next candidate for Respects Your Freedom certification is the Arctic Tern, a BMC development kit for OpenPOWER systems. A prototype libre GPU can be implemented using two FPGAs, each one for one screen, with a resolution of up to 1920×1200. Currently I use an OrangeCrab for my work on libre-soc, I have no need for an Arctic Tern. I also have a BeagleWire, an FPGA development cape for the BeagleBone Black, using an ICE40 FPGA which is also found on the Valve Index and Talos II.

Unlike a modern x86-64, such as the Steam Deck, the Talos II can’t run Steam, so the is no way to play VR games such as Beat Saber, Blade & Sorcery or VRChat. Currenly I can only play the godot4_openxr_demo using Monado and Libsurvice, but I have begun doing a VR port of Minetest, a libre clone of Minecraft and I am also trying to get Godot Beep Saber VR working with my Valve Index using Monado. Currently Beep Saber only works with SteamVR and the Oculus Quest, both non-free platforms incompatible with OpenPOWER systems.

Since I want a mobile VR headset that works without any non-free software, I propose building one using libre-soc and the already existing Monado OpenXR stack. For both projects there is still much work todo. Hopefully the number of libre VR games will grow in the next few years, if more and more people switch to OpenPOWER and ethical distros. Since I avoid both Android and SteamOS, so I won’t buy the Oclulus Quest nor the Steam Deck. Once a libre VR headset exists, it could get Respects Your Freedom certification. In guess that that will be another 5 years.

Sniffing Android apps network traffic

• Andrea Scarpino's blog
• —
• 00:00, Thursday, 15 September 2022

Back in the days, it was really easy to sniff the network traffic made by the Apps in Android. You could do it in a few minutes by adding mitmproxy’s certificate and setting the HTTP proxy on your wifi network settings. That was it. But things have changed (for good) and that’s no longer the case. However, I still want to sniff the network traffic made by the Apps in Android.

How? Well, I can no longer use my smartphone to do it, but I can set up the Android emulator, install the application via the Google Play Store and sniff the network traffic it generates on my PC \o/

Let’s get started. First, install the Android SDK and create an Android virtual device using Android API 30 and x86 architecture (any API and any architecture is fine). However, we need an image without Google Play Store preinstalled as we need a writable /system folder to inject mitmproxy’s certificate later. That’s okay, because we’ll install the Play Store manually.

echo no | ./Android/Sdk/tools/bin/avdmanager create avd -n Pixel_5_API_30 --abi google_apis/x86 --package 'system-images;android-30;google_apis;x86'

Start the virtual device with the additional -writable-system flag which permits us to make /system writable. I also have to unset QT_QPA_PLATFORM= because I’m on wayland and the emulator doesn’t support it.

QT_QPA_PLATFORM= ./Android/Sdk/emulator/emulator @Pixel_5_API_30 -writable-system

Now let’s download the OpenGAPPs that match our API and architecture. Select the pico variant because we don’t need anything else, just the Play Store.

curl -OL 'https://master.dl.sourceforge.net/project/opengapps/x86/20220503/open_gapps-x86-11.0-pico-20220503.zip'

We’ve to decompress it in order to get and push Phonesky.apk to the virtual device. We also need to whitelist its permissions (thank you to the MinMicroG guys).

unzip open_gapps-x86-11.0-pico-20220503.zip
lzip -d Core/vending-x86.tar.lz
tar xf vending-x86.tar
adb root
adb shell avbctl disable-verification # adb disable-verity makes the emulator crash
adb reboot
adb wait-for-device
adb root
adb remount
adb push vending-x86/nodpi/priv-app/Phonesky/Phonesky.apk /system/priv-app/
curl -O https://raw.githubusercontent.com/FriendlyNeighborhoodShane/MinMicroG/master/res/system/etc/permissions/com.android.vending.xml
adb push com.android.vending.xml /system/etc/permissions/

Now, create a dedicated user to run mitmproxy as it’s written in the documentation:

sudo useradd --create-home mitmproxyuser
sudo iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 80 -j REDIRECT --to-port 8080
sudo iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 443 -j REDIRECT --to-port 8080
sudo -u mitmproxyuser -H bash -c 'mitmproxy --mode transparent --showhost --set block_global=false'

Mandatory copy’n’paste from the mitmproxy documentation page: > Note, as soon as you add the iptables rules, you won’t be able to perform successful network calls until you start mitmproxy.

At this point we are almost there, we just need another step to add the mitmproxy certificate as it’s written in the documentation page:

hashed_name=`sudo openssl x509 -inform PEM -subject_hash_old -in ~mitmproxyuser/.mitmproxy/mitmproxy-ca-cert.cer | head -1`
sudo adb push ~mitmproxyuser/.mitmproxy/mitmproxy-ca-cert.cer /system/etc/security/cacerts/$hashed_name.0
adb shell chmod 664 /system/etc/security/cacerts/$hashed_name.0
adb reboot

You should now have the Play Store, login with your Google account and install the App you need.

That’s it! Happy sniffing!

Using Pushdown Automata to verify Packet Sequences

• foss – vanitasvitae's blog
• —
• 21:46, Wednesday, 14 September 2022

This is part 1 of a 2-part series on using pushdown automata for packet sequence validation. Read part two here.

As a software developer, most of my work day is spent working practically by coding and hacking away. Recently though I stumbled across an interesting problem which required another, more theoretical approach;

An OpenPGP message contains of a sequence of packets. There are signatures, encrypted data packets and their accompanying encrypted session keys, compressed data and literal data, the latter being the packet that in the end contains the plaintext body of the message.

Those packets can be sequential, e.g. a one-pass-signature followed by a literal data packet and then a signature, or nested, where for example an encrypted data packet contains a compressed data packet, in turn containing a literal data packet. A typical OpenPGP message can be visualized as follows:

This particular message consists of a sequence of Public-Key Encrypted Session Keys (PKESKs), followed by a Symmetrically Encrypted Integrity-Protected Data packet (SEIPD), and Modification Detection Code packet (MDC). Decrypting the SEIPD using the session key obtained from any of the PKESKs by providing an OpenPGP secret key yields a new data stream consisting of a OnePassSignature (OPS) followed by a Compressed Data packet and a Signature. Decompressing the Compressed Data packet yields a Literal Data packet which in turn contains the plaintext of the message.

I am pretty confident that PGPainless can currently handle all possible combinations of packets just fine. Basically it simply reads the next packet, processes it in however way the packet needs to be processed and then reads the next packet. That makes it very powerful, but there is a catch! Not possible combinations are valid!

The RFC contains a section describing the syntax of OpenPGP messages using a set of expressions which form a context-free grammar:

11.3.  OpenPGP Messages

An OpenPGP message is a packet or sequence of packets that corresponds to the following grammatical rules (comma  represents sequential composition, and vertical bar separates alternatives):

   OpenPGP Message :- Encrypted Message | Signed Message |
                      Compressed Message | Literal Message.

   Compressed Message :- Compressed Data Packet.

   Literal Message :- Literal Data Packet.

   ESK :- Public-Key Encrypted Session Key Packet |
          Symmetric-Key Encrypted Session Key Packet.

   ESK Sequence :- ESK | ESK Sequence, ESK.

   Encrypted Data :- Symmetrically Encrypted Data Packet |
         Symmetrically Encrypted Integrity Protected Data Packet

   Encrypted Message :- Encrypted Data | ESK Sequence, Encrypted Data.

   One-Pass Signed Message :- One-Pass Signature Packet,
               OpenPGP Message, Corresponding Signature Packet.

   Signed Message :- Signature Packet, OpenPGP Message |
               One-Pass Signed Message.

In addition, decrypting a Symmetrically Encrypted Data packet or a Symmetrically Encrypted Integrity Protected Data packet as well as decompressing a Compressed Data packet must yield a valid OpenPGP Message.

Using this grammar, we can construct OpenPGP messages by starting with the term “OpenPGP Message” and iteratively replacing parts of it according to the rules until only final “Packet” terms are left.

Let’s create the message from the diagram above to illustrate the process:

OpenPGP Message
-> Encrypted Message
-> ESK Sequence, Encrypted Data
-> ESK, ESK Sequence, Encrypted Data
-> ESK, ESK, Encrypted Data
-> SKESK, ESK, Encrypted Data
-> SKESK, SKESK, Encrypted Data
-> SKESK, SKESK, SEIPD(OpenPGP Message)
-> SKESK, SKESK, SEIPD(Signed Message)
-> SKESK, SKESK, SEIPD(One-Pass Signed Message)
-> SKESK, SKESK, SEIPD(OPS, OpenPGP Message, Sig)
-> SKESK, SKESK, SEIPD(OPS, Compressed Message, Sig)
-> SKESK, SKESK, SEIPD(OPS, Compressed Packet(OpenPGP Message), Sig)
-> SKESK, SKESK, SEIPD(OPS, Compressed Packet(Literal Message), Sig)
-> SKESK, SKESK, SEIPD(OPS, Compressed Packet(Literal Packet("Hello, World!")), Sig)

Here, cursive text marks the term that gets replaced in the next step. Bold text marks final OpenPGP packets which are not being replaced any further. Text inside of braces symbolizes nested data, which is the content of the packet before the braces. Some packet terms were abbreviated to make the text fit into individual lines.

Now, applying the rules in the “forwards direction” as we just did is rather simple and if no non-final terms are left, we end up with a valid OpenPGP message. There are infinitely many solutions for “valid” OpenPGP messages. A brief selection:

Literal Packet("Hello, World")
OPS, Literal Packet("Hey!"), Sig
OPS, OPS, Literal Packet("Hoh!"), Sig, Sig
SEIPD(Literal Packet("Wow"))
Sig, Compressed Packet(Literal Packet("Yay"))

On the other hand, some examples for invalid OpenPGP packet streams:

<empty>
Literal Packet(_), Literal Packet(_)
OPS, Compressed Packet(<empty>), Sig
Literal Packet(_), Sig
OPS, Literal Packet(_)
SKESK, SEIPD(Literal Packet(_)), Literal Packet(_)

Give it a try, I can guarantee you, you cannot create these messages using the OpenPGP grammar when starting with the term OpenPGP Message.

So now the problem becomes: How can we check, whether a given OpenPGP packet stream forms a valid OpenPGP message according to the grammar? Surely just trying to reverse engineer the message structure manually by brute force would be a daunting task… Luckily, theoretical computer science has a solution for us: Pushdown Automata!

Note: The following description of a PDA is kept brief and simplistic. I may well have made imprecise simplifications on the formal definition. If you want to learn more, but care about correctness, or if you are reading this post in preparation for an exam, you really should check out the Wikipedia page linked above instead.

Me, perhaps not totally accurate on the internet

A Pushdown Automaton (PDA) consists of a set of states with transition rules between those states, as well as a stack. It further has an initial state, as well as a set of accepting states. Each time the automaton reads an input symbol from a word (in our case a packet from a packet stream), it pops the top most item from the stack and then checks, if there is a transition from its current state using the given input and stack item to another state. If there is, it transitions into that state, possibly pushing a new item onto the stack, according to the transition rule. If all input symbols have been read, the word (packet stream) is valid if and only if current state is an accepting state, the stack of the automaton is empty and there are no more input symbols left. If the current state is not accepting or if the stack of the automaton is not empty, the word is invalid.

Formally defined, a transition rule is a tuple (state, input-symbol, stack-symbol, state, stack-symbol), where the first state is the origin, the first stack-symbol is what needs to be popped from the stack, the second state is the destination and the second stack-symbol what we push back onto the stack.

There is a special symbol ‘ε’ which means “nothing”. If the input symbol is ε, it means we can apply the rule without reading any input. If a stack symbol is nothing it means we can apply the rule without popping or pushing the stack.

I translated the OpenPGP grammar into a PDA. There is an initial state “start”, a single accepting state “Valid” and a set of transition rules which I annotated with arrows in the following diagram. Let’s take a closer look.

Let’s say we want to validate an OpenPGP message of the format OPS, Literal Packet(_), Sig.

We start with the “start” state to the left. As you can see, the only rule we can apply now is labelled ε,ε/m#, which means we do not read any input and do not pop the stack, but we push ‘#’ and ‘m’ onto the stack. After we applied the rule, we are in the state labelled “OpenPGP Message” and the top of our stack contains the symbol ‘m’.

To advance from here, we can peek at our stack and at the input to check our options. Since our message begins with a One-Pass-Signature (OPS), the only rule we can now apply is OPS,m/o, which requires that the top stack symbol is ‘m’. We read “OPS”, pop ‘m’ from the stack, push ‘o’ onto it and transition into the state “One-Pass-Signed Message”. From here there is only one rule ε,ε/m, which means we simply push ‘m’ onto the stack without popping an item and without reading input. That leaves us back in the state “OpenPGP Message”. Our stack now contains (top to bottom) ‘mo#’.

Now we read the next input symbol “Literal Packet” and apply the rule Literal Packet,m/ε, which means we pop ‘m’ from the stack and transition into state “Literal Message”. The stack now contains ‘o#’.

Next, we read “Sig” from the input, pop ‘o’ from the stack without pushing back an item, transitioning to state “Corresponding Signature”.

Last but not least, we apply rule ε,#/ε, do not read from the input, but pop the top symbol ‘#’ from the stack, transitioning to state “Valid”.

In the end, our stack is empty, we read all of the input data and ended up in an accepting state; The packet sequence “OPS, Literal Packet, Sig” forms a valid OpenPGP message.

Would we start over, but with an invalid input like “Literal Packet, Sig”, the play would go like this:

First, we transition from “start” to “OpenPGP Message” by pushing ‘#’ and ‘m’ to the stack. Then we apply rule Literal Packet,m/ε, read “Literal Packet” from the input, pop ‘m’ from the stack without pushing anything back onto it. This brings us into state “Literal Message” with ‘#’ being our new stack symbol. From here, we only have two rules that we could apply: Sig,o/ε would require us to read “Sig” from the input and have ‘o’ on the top of our stack. Both of these requirements we cannot fulfill. The other option ε,#/ε requires us to pop ‘#’ from the stack without reading input. It even brings us into a state called “Valid”! Okay, let’s do that then!

So far we have read “Literal Packet” from the packet stream. Would the data stream end here, we would have a valid OpenPGP message. Unfortunately, there is still some input left. However, there are no valid rules which allow us to transition any further with input “Sig”. Therefore, the input “Literal Packet, Sig” is not a valid OpenPGP message.

You can try any of the invalid messages listed above and you will see that you will always end up in a situation where you either have not fully read all the input symbols, your stack is not empty, or you end up in a non-accepting state.

You might notice some transitions are drawn using dashed lines. Those illustrate transitions for validating the content of nested packets. For example the Compressed Packet MUST contain another valid OpenPGP message. Depending on how you implement the PDA in software, you can either replace the input stream and use the nested transitions which require you to jump back from a nested “Valid” to the state where the nested transition originated from, or use child PDAs as described below.

The packet sequence Compressed Packet(Literal Packet(_)) for example would be resolved like follows:

From “start” we transition to “OpenPGP Message” by pushing ‘#’ and ‘m’ on the stack. The we read “Compressed Packet from input, pop ‘m’ from the stack and transition to state “Compressed Message”. Since the “Literal Packet” is part of the Compressed Packet”s contents, we now create a new child PDA with input stream “Literal Packet”. After initializing this PDA by pushing ‘#’ and ‘m’ to the stack, we then transition from “OpenPGP Message” to “Literal Message” by reading “Literal Packet” and popping ‘m’, after which we transition to “Valid” by popping ‘#’. Now this PDA is ended up in a valid state, so our parent PDA can transition from “Compressed Message” by reading nothing from the input (remember, the “Compressed Packet” was the only packet in this PDAs stream), popping ‘#’, leaving us with an empty stack and empty input in the valid state.

In PGPainless’ code I am planning to implement OpenPGP message validation by using InputStreams with individual PDAs. If a packet contains nested data (such as the Compressed or Encrypted Packet), a new InputStream will be opened on the decompressed/decrypted data. This new InputStream will in turn have a its own PDA to ensure that the content of the packet forms a valid OpenPGP message on its own. The parent stream on the other hand must check, whether the PDA of it’s child stream ended up in a valid state before accepting its own packet stream.

Initial tests already show promising results, so stay tuned for another blog post where I might go into more details on the implementation

I really enjoyed this journey into the realm of theoretical computer science. It was a welcome change to my normal day-to-day programming.

Happy Hacking!

Ada & Zangemann ready to pre-order in English

• Matthias Kirschner's Web log - fsfe
• —
• 14:58, Thursday, 08 September 2022

My English book Ada & Zangemann - A Tale of Software, Skateboards, and Raspberry Ice Cream can be pre-ordered at No Starch Press with the coupon code "Hacking4Freedom".

This still feels a bit like a wonderful dream to me and I am so thankful it all worked out. I hope you will enjoy reading the book yourself and to others, that you share it with children around you to inspire their interest in tinkering with software and hardware and encourage them to shape technology themselves. I also hope, that it serves as a useful gift for your family, friends, and co-workers because it explores topics you care about, and that you also appreciate that it is published under Creative Commons By Share Alike.

“A rousing tale of self-reliance, community, and standing up to bullies…software freedom is human freedom!” —Cory Doctorow, Sci-Fi Author

“Introduces readers young and old to the power and peril of software. Behind it all is a backdrop of ethics of knowledge sharing upon which the arc of human history rides.” —Vint Cerf, Computer Scientist and One of the Inventors of the Internet

“In this hopeful story Ada and her friends join a movement that started back in 1983. Their courageous adventure of software freedom and learning how technology works is a wonderful way to introduce young people everywhere to the joys of tinkering!” —Zoë Kooyman, Executive Director, Free Software Foundation

“Even as a non-child, I was captivated by the story from the first page to the last. Kudos to the author for packaging difficult topics such as monopolies, lobbyism, digital divide, software freedom, digital autonomy, IoT, consumer control, e-waste and much more in a child-friendly form in an easily understandable and exciting storyline.” —Jörg Luther, chief editor of the German Linux-Magazin, LinuxUser, and Raspberry Pi Geek

If you are from the US you can pre-order the hardcover from No Starch Press, get 25% off with the coupon code, receive the DRM free ebook now, and get the book sent from the US in starting in December.

If you live outside the US there are other options. According to the current plan, by the end of 2022, you should be able to pre-order the book from local bookstores worldwide, so the book does not have to be shipped from the US. Those bookstores should then be able to ship the book from ~May 2023 onwards.

Such a book would not be possible without the help of many people and without space restrictions, I can be a bit more verbose in thanking them (and I am pretty sure I unfortunately still forgot some: if so, please let me know):

Many thanks to Reinhard Wiesemann from the Linuxhotel for the financial support and motivation to no longer plan but to make. Thanks to Sandra Brandstätter, who has brought me joy with every new design for the illustrations. Thanks to my editor Wiebke Helmchen, who made developing and sharpening the story fun, and to No Starch Press, and my German publisher d.punkt / O'Reilly guiding me through the whole process of publishing a book and for agreeing to publish the book under a free culture license.

Thanks to Bea and Benni, Christine and Marc, Bernhard Reiter, Isabel Drost-Fromm and Amelia, Katta, Kristina, Martin, Mona and Arne, Nina, Oliver Diedrich, Reinhard Müller, Sabine, and Torsten Grote for great ideas, inspiration, and practical tips.

Thanks to my collegues and volunteers from the FSFE, and the team at d.punkt / O'Reilly who helped a lot to promote the book to a larger audiences.

For the English version a special thanks goes to Cory Doctorow, Brian Osborn, and Vint Cerf for helping me find the right publisher, to John Sullivan for his valuable feedback for language improvements, to Luca Bonessi for his great help with maintaining the git repository for the book, Catharina Maracke and Till Jaeger for their help with Creative Commons licensing with the German version and the many hours in which Pamela Chestek helped No Starch Press and myself to optimise the contract for CC-BY-SA.

Thanks to Bill Pollock and the team at No Starch Press for working on all those many details included in book publishing, from editing, production, marketing, sales, and all the other nitty-gritty details that are often not recognized by readers, but that make a huge difference.

Thanks to the many people of the Free Software movement in general and all the volunteers and staffers of the FSFE, from whom I was allowed to learn and whose engagement motivated me so that this book can inspire children’s interest in tinkering and encourage them to shape technology.

Finally, thanks to my family, who made me write early in the morning, late in the evening, and on holiday, and who always supports my work for software freedom. Especially, I would like to thank my children, who inspired me with new ideas at each reading. Without them, this book would not exist.

For additional information about the book, please visit the FSFE's website about it. All royalties of the book will go to the charitable FSFE and supports the FSFE's work for software freedom.

FSFE information desk on Veganmania Danube Island 2022

• English – FSFE supporters Vienna
• —
• 10:52, Tuesday, 06 September 2022

It was the usual information stall like described several times before in this blog. Unfortunately I didn’t have time yet to write more about it. I created an updated information leaflet and really should get a tent because this time we had heavy rain twice and it was very hard to protect the paper materials with only an umbrella as cover.

I will write more as soon as I find time to do so.

Creating a Web-of-Trust Implementation: Accessing Certificate Stores

• foss – vanitasvitae's blog
• —
• 11:19, Thursday, 01 September 2022

Currently, I am working on a Web-of-Trust implementation for the OpenPGP library PGPainless. This work is being funded by the awesome NLnet foundation through NGI Assure. Check them out! NGI Assure is made possible with financial support from the European Commission’s Next Generation Internet programme.

In this post, I will outline some progress I made towards a full WoT implementation. The current milestone entails integrating certificate stores more closely with the core API.

On most systems, OpenPGP certificates (public keys) are typically stored and managed by GnuPGs internal key store. The downside of this approach is, that applications that want to make use of OpenPGP either need to depend on GnuPG, or are required to manage their very own exclusive certificate store. The latter approach, which e.g. Thunderbird is taking, leads to a situation where there are multiple certificate stores with different contents. Your GnuPG certificate store might contain Bobs certificate, while the Thunderbird store does not. This is confusing for users, as they now have to manage two places with OpenPGP certificates.

There is a proposal for a Shared PGP Certificate Directory nicknamed “pgp.cert.d” which aims to solve this issue by specifying a shared, maildir-like directory for OpenPGP certificates. This directory serves as a single source for all OpenPGP certificates a user might have to deal with. Being well-defined through the standards draft means different applications can access the certificate store without being locked into a single OpenPGP backend.

Since the Web-of-Trust also requires a certificate store of some kind to work on, I thought that pgp.cert.d might be the ideal candidate to implement. During the past months I reworked my existing implementation to allow for different storage backends and defined an abstraction layer for generalized certificate stores (not only pgp.cert.d). This abstraction layer was integrated with PGPainless to allow encryption and verification operations to request certificates from a store. Let me introduce the different components in more detail:

The library pgp-cert-d-java contains an implementation of the pgp.cert.d specification. It provides an API for applications to store and fetch certificates to and from the pgp.cert.d directory. The task of parsing the certificate material was delegated to the consumer application, so the library is independent from OpenPGP backends.

The library pgp-certificate-store defines an abstraction layer above pgp-cert-d-java. It contains interfaces for a general OpenPGP certificate store. Implementations of this interface could for example access GnuPGs certificate store, since the interface does not make assumptions about how the certificates are stored. Inside pgp-cert-d-java, there is an adapter class that adapts the PGPCertificateDirectory class to the PGPCertificateStore interface.

The pgpainless-cert-d module provides certificate parsing functionality using pgpainless-core. It further provides a factory class to instantiate PGPainless-backed instances of the PGPCertificateDirectory interface (both file-based, as well as in-memory directories).

Lastly, the pgpainless-cert-d-cli application is a command line tool to demonstrate the pgp.cert.d functionality. It can be used to manage the certificate directory by inserting and fetching certificates:

$ pgpainless-cert-d-cli help
Store and manage public OpenPGP certificates
Usage: certificate-store [-s=DIRECTORY] [COMMAND]

Options:
  -s, --store=DIRECTORY   Overwrite the default certificate directory path

Commands:
  help    Display the help text for a subcommand
  export  Export all certificates in the store to Standard Output
  insert  Insert or update a certificate
  import  Import certificates into the store from Standard Input
  get     Retrieve certificates from the store
  setup   Setup a new certificate directory
  list    List all certificates in the directory
  find    Lookup primary certificate fingerprints by subkey ids or fingerprints
Powered by picocli

Now let’s see how the certificate store can integrate with PGPainless:

Firstly, let’s set up a pgp.cert.d using pgpainless-cert-d-cli:

$ pgpainless-cert-d-cli setup
facf859c9dc1106c4a30f56b1c38b70b755017cf

This command initializes the certificate directory in .local/share/pgp.cert.d/ and creates a trust-root key with the displayed fingerprint. This trust-root currently is not of use, but eventually we will use it as the root of trust in the Web-of-Trust.

Just for fun, let’s import our OpenPGP certificates from GnuPG into the pgp.cert.d:

$ gpg --export --armor | pgpainless-cert-d-cli import

The first part of the command exports all public keys from GnuPG, while the second part imports them into the pgp.cert.d directory.

We can now access those certificates like this:

$ pgpainless-cert-d-cli get -a 7F9116FEA90A5983936C7CFAA027DB2F3E1E118A
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: PGPainless
Comment: 7F91 16FE A90A 5983 936C  7CFA A027 DB2F 3E1E 118A
Comment: Paul Schaub <[email protected]>
Comment: 2 further identities

mQINBFfz1ucBEADXSvUjnOWSzgW5hXki1xUpGv7vacT8XqqGbO9Z32P3eFxa4E9J
vveJmx+voxRWpleZ/L6XCYYmCKnagjF0fMxFD1Zxicp5tzbruC1cm/Els0IJVjFV
RLke3SegTHxHncA8+BYn2k/VnTKwDXzP0ZLyc7mUbDl8CCtWGGUkXpaa7WyZIA/q
[...]
-----END PGP PUBLIC KEY BLOCK-----

Would this certificate change over time, e.g. because someone signs it and sends me an updated copy, I could merge the new signatures into the store by simply inserting the updated certificate again:

pgpainless-cert-d-cli insert < update.asc

Now, I said earlier that the benefit of the pgp.cert.d draft was that applications could access the certificate store without the need to rely on a certain backend. Let me demonstrate this by showing how to access my certificate within a Java application without the need to use pgpainless-cert-d-cli.

First, let’s write a small piece of code which encrypts a message to my certificate:

// Setup the store
SubkeyLookupFactory lookupFactory = new DatabaseSubkeyLookupFactory();
PGPCertificateDirectory pgpCertD = PGPainlessCertD.fileBased(lookupFactory);
PGPCertificateStoreAdapter store = new PGPCertificateStoreAdapter(pgpCertD);

OpenPgpFingerprint myCert = OpenPgpFingerprint.parse("7F9116FEA90A5983936C7CFAA027DB2F3E1E118A");

ByteArrayInputStream plaintext = new ByteArrayInputStream("Hello, World! This message is encrypted using a cert from a store!".getBytes());
ByteArrayOutputStream ciphertextOut = new ByteArrayOutputStream();

// Encrypt
EncryptionStream encryptionStream = PGPainless.encryptAndOrSign()
  .onOutputStream(ciphertextOut)
  .withOptions(ProducerOptions.encrypt(
    EncryptionOptions.encryptCommunications()
      .addRecipient(adapter, myCert)));
Streams.pipeAll(plaintext, encryptionStream);
encryptionStream.close();

System.out.println(ciphertextOut.toString())

In this example, we first set up access to the shared certificate directory. For that we need a method to look up certificates by subkey-ids. In this case this is done through an SQLite database. Next, we instantiate a PGPCertificateDirectory object, which we then wrap in a PGPCertificateStoreAdapter to make it usable within PGPainless.

Next, we only need to know our certificates fingerprint in order to instruct PGPainless to encrypt a message to it. Lastly, we print out the encrypted message.

In the future, once the Web-of-Trust is implemented, it should be possible to pass in the recipients email address instead of the fingerprint. The WoT would then find trustworthy keys with that email address and select those for encryption. Right now though, the user still has to identify trustworthy keys of recipients themselves still.

Similarly, we can use a certificate store when verifying a signed message:

ByteArrayInputStream ciphertextIn = ...; // signed message
ByteArrayOutputStream plaintextOut = new ByteArrayOutputStream();

// Verify
DecryptionStream verificationStream = PGPainless.decryptAndOrVerify()
  .onInputStream(ciphertextIn)
  .withOptions(new ConsumerOptions()
    .addVerificationCerts(adapter));
Streams.pipeAll(verificationStream, plaintextOut);
verificationStream.close();

OpenPgpMetadata result = decryptionStream.getResult();
assertTrue(result.isVerified()); // signature is correct and valid
assertTrue(result.containsVerifiedSignatureFrom(myCert));

Here, PGPainless will process the signed message, identify the key that was used for signing and fetch its certificate from the certificate store.

Note, that if you implement verification like that, it is up to you to verify the trustworthiness of the certificate yourself.

In the future, this task will be done by a WoT library in the PGPainless ecosystem automatically though

The current state of the certificate store integration into PGPainless can be found on the storeIntegration branch.

Creating a kubernetes cluster with kubeadm on Ubuntu 22.04 LTS

• Evaggelos Balaskas - System Engineer
• —
• 15:58, Wednesday, 31 August 2022

In this blog post, I’ll try to share my personal notes on how to setup a kubernetes cluster with kubeadm on ubuntu 22.04 LTS Virtual Machines.

I am going to use three (3) Virtual Machines in my local lab. My home lab is based on libvirt Qemu/KVM (Kernel-based Virtual Machine) and I run Terraform as the infrastructure provision tool.

There is a copy of this blog post to github.

https://github.com/ebal/k8s_cluster

If you notice something wrong you can either contact me via the contact page, or open a PR in the github project.

you can also follow me at twitter: https://twitter.com/ebalaskas

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

• Prerequisites
• Git Terraform Code for the kubernetes cluster
  • Ubuntu 22.04 Image
• Control-Plane Node
  • Ports on the control-plane node
  • Firewall on the control-plane node
  • Hosts file in the control-plane node
  • No Swap on the control-plane node
  • Kernel modules on the control-plane node
  • NeedRestart on the control-plane node
  • Installing a Container Runtime on the control-plane node
  • Installing kubeadm, kubelet and kubectl on the control-plane node
  • Initializing the control-plane node
  • Create user access config to the k8s control-plane node
  • Verify the control-plane node
  • Install an overlay network provider on the control-plane node
  • Verify CoreDNS is running on the control-plane node
• Worker Nodes
  • Ports on the worker nodes
  • Firewall on the worker nodes
  • Hosts file in the worker node
  • No Swap on the worker node
  • Kernel modules on the worker node
  • NeedRestart on the worker node
  • Installing a Container Runtime on the worker node
  • Installing kubeadm, kubelet and kubectl on the worker node
  • Get Token from the control-plane node
  • Get Certificate Hash from the control-plane node
  • Join Workers to the kubernetes cluster
• Is the kubernetes cluster running ?
• Kubernetes Dashboard
  • Install kubernetes dashboard
  • Add a Node Port to kubernetes dashboard
  • Patch kubernetes-dashboard
  • Edit kubernetes-dashboard Service
  • Accessing Kubernetes Dashboard
  • Create An Authentication Token (RBAC)
  • Creating a Service Account
  • Creating a ClusterRoleBinding
  • Getting a Bearer Token
  • Browsing Kubernetes Dashboard
• Nginx App
  • Install nginx-app
  • Get Deployment
  • Expose Nginx-App
  • Verify Service nginx-app
  • Describe Service nginx-app
  • Curl Nginx-App
  • Nginx-App from Browser
• That’s it !

git clone https://github.com/ebal/k8s_cluster.git
cd tf_libvirt

terraform init

./start.sh

...
Apply complete! Resources: 16 added, 0 changed, 0 destroyed.

Outputs:

VMs = [
  "192.168.122.169  k8scpnode",
  "192.168.122.40   k8wrknode1",
  "192.168.122.8    k8wrknode2",
]

ssh  -l ubuntu 192.168.122.169

../jammy-server-cloudimg-amd64.img

cd ../
curl -sLO https://cloud-images.ubuntu.com/jammy/current/focal-server-cloudimg-amd64.img
cd -

sudo ufw allow 6443/tcp
sudo ufw allow 2379:2380/tcp
sudo ufw allow 10250/tcp
sudo ufw allow 10259/tcp
sudo ufw allow 10257/tcp

#sudo ufw disable
sudo ufw status

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
6443/tcp                   ALLOW       Anywhere
2379:2380/tcp              ALLOW       Anywhere
10250/tcp                  ALLOW       Anywhere
10259/tcp                  ALLOW       Anywhere
10257/tcp                  ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
6443/tcp (v6)              ALLOW       Anywhere (v6)
2379:2380/tcp (v6)         ALLOW       Anywhere (v6)
10250/tcp (v6)             ALLOW       Anywhere (v6)
10259/tcp (v6)             ALLOW       Anywhere (v6)
10257/tcp (v6)             ALLOW       Anywhere (v6)

echo $(hostname -I) $(hostname) | sudo tee -a /etc/hosts

192.168.122.169  k8scpnode
192.168.122.40   k8wrknode1
192.168.122.8    k8wrknode2

sudo swapoff -a

sudo grep -i swap /etc/fstab

sudo tee /etc/modules-load.d/kubernetes.conf <<EOF
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

sudo lsmod | grep netfilter

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

echo "\$nrconf{restart} = 'a';" | sudo tee -a /etc/needrestart/needrestart.conf

curl -sL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/docker-keyring.gpg

sudo apt-add-repository -y "deb https://download.docker.com/linux/ubuntu jammy stable"

sleep 5

sudo apt -y install containerd.io

containerd config default                              \
 | sed 's/SystemdCgroup = false/SystemdCgroup = true/' \
 | sudo tee /etc/containerd/config.toml

sudo systemctl restart containerd.service

systemd cgroup driver

sudo curl -sLo /etc/apt/trusted.gpg.d/kubernetes-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

sudo apt-add-repository -y "deb http://apt.kubernetes.io/ kubernetes-xenial main"

sleep 5

sudo apt install -y kubelet kubeadm kubectl

sudo kubeadm config images pull

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

rm -rf $HOME/.kube

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

ls -la $HOME/.kube/config

alias k="kubectl"

kubectl cluster-info
#kubectl cluster-info dump

k get nodes -o wide; k get pods  -A -o wide

k apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

$ k get nodes -o wide

NAME        STATUS   ROLES           AGE   VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
k8scpnode   Ready    control-plane   54s   v1.25.0   192.168.122.169   <none>        Ubuntu 22.04.1 LTS   5.15.0-46-generic   containerd://1.6.8

$ k get pods -A -o wide

NAMESPACE    NAME                              READY STATUS  RESTARTS AGE IP              NODE      NOMINATED NODE READINESS GATES
kube-flannel kube-flannel-ds-zqv2b             1/1   Running 0        36s 192.168.122.169 k8scpnode <none>         <none>
kube-system  coredns-565d847f94-lg54q          1/1   Running 0        38s 10.244.0.2      k8scpnode <none>         <none>
kube-system  coredns-565d847f94-ms8zk          1/1   Running 0        38s 10.244.0.3      k8scpnode <none>         <none>
kube-system  etcd-k8scpnode                    1/1   Running 0        50s 192.168.122.169 k8scpnode <none>         <none>
kube-system  kube-apiserver-k8scpnode          1/1   Running 0        50s 192.168.122.169 k8scpnode <none>         <none>
kube-system  kube-controller-manager-k8scpnode 1/1   Running 0        50s 192.168.122.169 k8scpnode <none>         <none>
kube-system  kube-proxy-pv7tj                  1/1   Running 0        39s 192.168.122.169 k8scpnode <none>         <none>
kube-system  kube-scheduler-k8scpnode          1/1   Running 0        50s 192.168.122.169 k8scpnode <none>         <none>

sudo ufw allow 10250/tcp
sudo ufw allow 30000:32767/tcp

sudo ufw status

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
10250/tcp                  ALLOW       Anywhere
30000:32767/tcp            ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
10250/tcp (v6)             ALLOW       Anywhere (v6)
30000:32767/tcp (v6)       ALLOW       Anywhere (v6)

192.168.122.169  k8scpnode
192.168.122.40   k8wrknode1
192.168.122.8    k8wrknode2

sudo swapoff -a

sudo tee /etc/modules-load.d/kubernetes.conf <<EOF
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

sudo lsmod | grep netfilter

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

echo "\$nrconf{restart} = 'a';" | sudo tee -a /etc/needrestart/needrestart.conf

curl -sL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/docker-keyring.gpg

sudo apt-add-repository -y "deb https://download.docker.com/linux/ubuntu jammy stable"

sleep 5

sudo apt -y install containerd.io

containerd config default                              \
 | sed 's/SystemdCgroup = false/SystemdCgroup = true/' \
 | sudo tee /etc/containerd/config.toml

sudo systemctl restart containerd.service

sudo curl -sLo /etc/apt/trusted.gpg.d/kubernetes-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

sudo apt-add-repository -y "deb http://apt.kubernetes.io/ kubernetes-xenial main"

sleep 5

sudo apt install -y kubelet kubeadm kubectl

sudo kubeadm config images pull

sudo kubeadm  token list

TOKEN                     TTL         EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
zt36bp.uht4cziweef1jo1h   23h         2022-08-31T18:38:16Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

zt36bp.uht4cziweef1jo1h

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

a4833f8c82953370610efaa5ed93b791337232c3a948b710b2435d747889c085

sudo kubeadm join 192.168.122.169:6443 \
       --token zt36bp.uht4cziweef1jo1h \
       --discovery-token-ca-cert-hash sha256:a4833f8c82953370610efaa5ed93b791337232c3a948b710b2435d747889c085

kubectl get nodes   -o wide
kubectl get pods -A -o wide

NAME         STATUS   ROLES           AGE     VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
k8scpnode    Ready    control-plane   64m     v1.25.0   192.168.122.169   <none>        Ubuntu 22.04.1 LTS   5.15.0-46-generic   containerd://1.6.8
k8wrknode1   Ready    <none>          2m32s   v1.25.0   192.168.122.40    <none>        Ubuntu 22.04.1 LTS   5.15.0-46-generic   containerd://1.6.8
k8wrknode2   Ready    <none>          2m28s   v1.25.0   192.168.122.8     <none>        Ubuntu 22.04.1 LTS   5.15.0-46-generic   containerd://1.6.8

NAMESPACE      NAME                                READY   STATUS    RESTARTS      AGE     IP                NODE         NOMINATED NODE   READINESS GATES
kube-flannel   kube-flannel-ds-52g92               1/1     Running   0             2m32s   192.168.122.40    k8wrknode1   <none>           <none>
kube-flannel   kube-flannel-ds-7qlm7               1/1     Running   0             2m28s   192.168.122.8     k8wrknode2   <none>           <none>
kube-flannel   kube-flannel-ds-zqv2b               1/1     Running   0             64m     192.168.122.169   k8scpnode    <none>           <none>
kube-system    coredns-565d847f94-lg54q            1/1     Running   0             64m     10.244.0.2        k8scpnode    <none>           <none>
kube-system    coredns-565d847f94-ms8zk            1/1     Running   0             64m     10.244.0.3        k8scpnode    <none>           <none>
kube-system    etcd-k8scpnode                      1/1     Running   0             64m     192.168.122.169   k8scpnode    <none>           <none>
kube-system    kube-apiserver-k8scpnode            1/1     Running   0             64m     192.168.122.169   k8scpnode    <none>           <none>
kube-system    kube-controller-manager-k8scpnode   1/1     Running   1 (12m ago)   64m     192.168.122.169   k8scpnode    <none>           <none>
kube-system    kube-proxy-4khw6                    1/1     Running   0             2m32s   192.168.122.40    k8wrknode1   <none>           <none>
kube-system    kube-proxy-gm27l                    1/1     Running   0             2m28s   192.168.122.8     k8wrknode2   <none>           <none>
kube-system    kube-proxy-pv7tj                    1/1     Running   0             64m     192.168.122.169   k8scpnode    <none>           <none>
kube-system    kube-scheduler-k8scpnode            1/1     Running   1 (12m ago)   64m     192.168.122.169   k8scpnode    <none>           <none>

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml

namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created

kubectl get all -n kubernetes-dashboard

NAME                                             READY   STATUS    RESTARTS   AGE
pod/dashboard-metrics-scraper-64bcc67c9c-kvll7   1/1     Running   0          2m16s
pod/kubernetes-dashboard-66c887f759-rr4gn        1/1     Running   0          2m16s

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/dashboard-metrics-scraper   ClusterIP   10.110.25.61    <none>        8000/TCP   2m16s
service/kubernetes-dashboard        ClusterIP   10.100.65.122   <none>        443/TCP    2m16s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/dashboard-metrics-scraper   1/1     1            1           2m16s
deployment.apps/kubernetes-dashboard        1/1     1            1           2m16s

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/dashboard-metrics-scraper-64bcc67c9c   1         1         1       2m16s
replicaset.apps/kubernetes-dashboard-66c887f759        1         1         1       2m16s

kubectl --namespace kubernetes-dashboard patch svc kubernetes-dashboard -p '{"spec": {"type": "NodePort"}}'

service/kubernetes-dashboard patched

kubectl get svc -n kubernetes-dashboard

NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
dashboard-metrics-scraper   ClusterIP   10.110.25.61    <none>        8000/TCP        11m
kubernetes-dashboard        NodePort    10.100.65.122   <none>        443:32709/TCP   11m

kubectl edit svc -n kubernetes-dashboard kubernetes-dashboard

type: ClusterIP

type: NodePort

kubectl get pods -n kubernetes-dashboard -o wide

NAME                                         READY   STATUS    RESTARTS   AGE     IP           NODE         NOMINATED NODE   READINESS GATES
dashboard-metrics-scraper-64bcc67c9c-fs7pt   1/1     Running   0          2m43s   10.244.1.9   k8wrknode1   <none>           <none>
kubernetes-dashboard-66c887f759-pzt4z        1/1     Running   0          2m44s   10.244.2.9   k8wrknode2   <none>           <none>

k get svc -n kubernetes-dashboard -o wide

https://192.168.122.8:32709

cat > kubernetes-dashboard.ServiceAccount.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

EOF

kubectl apply -f kubernetes-dashboard.ServiceAccount.yaml

serviceaccount/admin-user created

cat > kubernetes-dashboard.ClusterRoleBinding.yaml <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

EOF

kubectl apply -f kubernetes-dashboard.ClusterRoleBinding.yaml

clusterrolebinding.rbac.authorization.k8s.io/admin-user created

kubectl -n kubernetes-dashboard create token admin-user

eyJhbGciOiJSUzI1NiIsImtpZCI6Im04M2JOY2k1Yk1hbFBhLVN2cjA4X1pkdktXNldqWkR4bjB6MGpTdFgtVHcifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNjYxOTU2NDQ1LCJpYXQiOjE2NjE5NTI4NDUsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiN2M4OWIyZDktMGIwYS00ZDg4LTk2Y2EtZDU3NjhjOWU2ZGYxIn19LCJuYmYiOjE2NjE5NTI4NDUsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.RMRQkZZhcoC5vCvck6hKfqXJ4dfN4JoQyAaClHZvOMI6JgQZEfB2-_Qsh5MfFApJUEit-0TX9r3CzW3JqvB7dmpTPxUQvHK68r82WGveBVp1wF37UyXu_IzxiCQzpCWYr3GcVGAGZVBbhhqNYm765FV02ZA_khHrW3WpB80ikhm_TNLkOS6Llq2UiLFZyHHmjl5pwvGzT7YXZe8s-llZSgc0UenEwPG-82eE279oOy6r4_NltoV1HB3uu0YjUJPlkqAPnHuSfAA7-8A3XAAVHhRQvFPea1qZLc4-oD24AcU0FjWqDMILEyE8zaD2ci8zEQBMoxcf2qmj0wn9cfbZwQ

kubectl create deployment nginx-app --image=nginx --replicas=2

deployment.apps/nginx-app created

kubectl get deployment nginx-app -o wide

NAME        READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx-app   2/2     2            2           64s   nginx        nginx    app=nginx-app

kubectl expose deployment nginx-app --type=NodePort --port=80

service/nginx-app exposed

kubectl get svc nginx-app -o wide

NAME        TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE   SELECTOR
nginx-app   NodePort   10.98.170.185   <none>        80:31761/TCP   27s   app=nginx-app

kubectl describe svc nginx-app

Name:                     nginx-app
Namespace:                default
Labels:                   app=nginx-app
Annotations:              <none>
Selector:                 app=nginx-app
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.98.170.185
IPs:                      10.98.170.185
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  31761/TCP
Endpoints:                10.244.1.10:80,10.244.2.10:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

curl http://192.168.122.8:31761

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

./destroy.sh

...

libvirt_domain.domain-ubuntu["k8wrknode1"]: Destroying... [id=446cae2a-ce14-488f-b8e9-f44839091bce]
libvirt_domain.domain-ubuntu["k8scpnode"]: Destroying... [id=51e12abb-b14b-4ab8-b098-c1ce0b4073e3]
time_sleep.wait_for_cloud_init: Destroying... [id=2022-08-30T18:02:06Z]
libvirt_domain.domain-ubuntu["k8wrknode2"]: Destroying... [id=0767fb62-4600-4bc8-a94a-8e10c222b92e]
time_sleep.wait_for_cloud_init: Destruction complete after 0s
libvirt_domain.domain-ubuntu["k8wrknode1"]: Destruction complete after 1s
libvirt_domain.domain-ubuntu["k8scpnode"]: Destruction complete after 1s
libvirt_domain.domain-ubuntu["k8wrknode2"]: Destruction complete after 1s
libvirt_cloudinit_disk.cloud-init["k8wrknode1"]: Destroying... [id=/var/lib/libvirt/images/Jpw2Sg_cloud-init.iso;b8ddfa73-a770-46de-ad16-b0a5a08c8550]
libvirt_cloudinit_disk.cloud-init["k8wrknode2"]: Destroying... [id=/var/lib/libvirt/images/VdUklQ_cloud-init.iso;5511ed7f-a864-4d3f-985a-c4ac07eac233]
libvirt_volume.ubuntu-base["k8scpnode"]: Destroying... [id=/var/lib/libvirt/images/l5Rr1w_ubuntu-base]
libvirt_volume.ubuntu-base["k8wrknode2"]: Destroying... [id=/var/lib/libvirt/images/VdUklQ_ubuntu-base]
libvirt_cloudinit_disk.cloud-init["k8scpnode"]: Destroying... [id=/var/lib/libvirt/images/l5Rr1w_cloud-init.iso;11ef6bb7-a688-4c15-ae33-10690500705f]
libvirt_volume.ubuntu-base["k8wrknode1"]: Destroying... [id=/var/lib/libvirt/images/Jpw2Sg_ubuntu-base]
libvirt_cloudinit_disk.cloud-init["k8wrknode1"]: Destruction complete after 1s
libvirt_volume.ubuntu-base["k8wrknode2"]: Destruction complete after 1s
libvirt_cloudinit_disk.cloud-init["k8scpnode"]: Destruction complete after 1s
libvirt_cloudinit_disk.cloud-init["k8wrknode2"]: Destruction complete after 1s
libvirt_volume.ubuntu-base["k8wrknode1"]: Destruction complete after 1s
libvirt_volume.ubuntu-base["k8scpnode"]: Destruction complete after 2s
libvirt_volume.ubuntu-vol["k8wrknode1"]: Destroying... [id=/var/lib/libvirt/images/Jpw2Sg_ubuntu-vol]
libvirt_volume.ubuntu-vol["k8scpnode"]: Destroying... [id=/var/lib/libvirt/images/l5Rr1w_ubuntu-vol]
libvirt_volume.ubuntu-vol["k8wrknode2"]: Destroying... [id=/var/lib/libvirt/images/VdUklQ_ubuntu-vol]
libvirt_volume.ubuntu-vol["k8scpnode"]: Destruction complete after 0s
libvirt_volume.ubuntu-vol["k8wrknode2"]: Destruction complete after 0s
libvirt_volume.ubuntu-vol["k8wrknode1"]: Destruction complete after 0s
random_id.id["k8scpnode"]: Destroying... [id=l5Rr1w]
random_id.id["k8wrknode2"]: Destroying... [id=VdUklQ]
random_id.id["k8wrknode1"]: Destroying... [id=Jpw2Sg]
random_id.id["k8wrknode2"]: Destruction complete after 0s
random_id.id["k8scpnode"]: Destruction complete after 0s
random_id.id["k8wrknode1"]: Destruction complete after 0s

Destroy complete! Resources: 16 destroyed.

Remove Previous GitLab Pipelines from a project

• Evaggelos Balaskas - System Engineer
• —
• 21:44, Wednesday, 24 August 2022

So you build a GitLab project, you created a pipeline and then a scheduler to run every week your pipeline.

And then you realize that you are polluting the internet with deprecated (garbage) things, at some point you have a debug option on, bla bla bla… etc etc.

It is time to clean up your mess!

Create a GitLab API Token

aka Personal Access Tokens

Select scopes: api.

Verify API token

run something like this

export GITLAB_API="glpat-HldkXzyurwBmroAdQCMo"

curl -sL --header "PRIVATE-TOKEN: ${GITLAB_API}" "https://gitlab.com/api/v4/projects?owned=true" | jq .[].path_with_namespace

you should see your projects.

Get your Project ID

create a new bash variable:

export PROJECT="terraform-provider/terraform-provider-hcloud-ci"

and then use the get rest api call

curl -sL --header "PRIVATE-TOKEN: ${GITLAB_API}" "https://gitlab.com/api/v4/projects?owned=true&search=${PROJECT}&search_namespaces=true" | jq -r .[].id

or you can also put the id into a new bash variable:

export ID=$(curl -sL --header "PRIVATE-TOKEN: ${GITLAB_API}" "https://gitlab.com/api/v4/projects?owned=true&search=${PROJECT}&search_namespaces=true" | jq -r .[].id)

View the previous pipelines

curl -sL \
  --header "PRIVATE-TOKEN: ${GITLAB_API}" \
  https://gitlab.com/api/v4/projects/${ID}/pipelines | jq .

Remove deprecated pipelines

just delete them via the API

curl -sL --header "PRIVATE-TOKEN: ${GITLAB_API}"   "https://gitlab.com/api/v4/projects/${ID}/pipelines?per_page=150"  \
 | jq  -r .[].id    \
 | awk '{print "curl -sL --header \"PRIVATE-TOKEN: ${GITLAB_API}\" --request DELETE https://gitlab.com/api/v4/projects/${ID}/pipelines/"$1}'   \
 | sh -x

that’s it !

Pessimistic perspectives on technological sustainability

• English – Paul Boddie's Free Software-related blog
• —
• 22:24, Monday, 15 August 2022

I was recently perusing the Retro Computing Forum when I stumbled across a mention of Collapse OS. If your anxiety levels have not already been maxed out during the last few years of climate breakdown, psychological warfare, pandemic, and actual warmongering, accompanied by supply chain breakdowns, initially in technology and exacerbated by overconsumption and spivcoin, now also in commodities and exacerbated by many of those other factors (particularly the warmongering), then perhaps focusing on societal and civilisational collapse isn’t going to improve your mood or your outlook. Unusually, then, after my last, rather negative post on such topics, may I be the one to introduce some constructive input and perhaps even some slight optimism?

If I understand the motivations behind Collapse OS correctly, it is meant to provide a modest computing environment that can work on well-understood, commonplace, easily repaired and readily sourced hardware, with the software providing the environment itself being maintainable on the target hardware, as opposed to being cross-built on more powerful hardware and then deployed to simpler, less capable hardware. The envisaged scenario for its adoption is a world where powerful new hardware is no longer produced or readily available and where people must scavenge and “make do” with the hardware already produced. Although civilisation may have brought about its own collapse, the consolation is that so much hardware will have been strewn across the planet for a variety of purposes that even after semiconductor fabrication and sophisticated manufacturing have ceased, there will remain a bounty of hardware usable for people’s computational needs (whatever they may be).

I am not one to try and predict the future, and I don’t really want to imagine it as being along the same lines as the plot for one of Kevin Costner’s less successful movies, either, but I feel that Collapse OS and its peers, in considering various dystopian scenarios and strategies to mitigate their impacts, may actually offer more than just a hopefully sufficient kind of preparedness for a depressing future. In that future, without super-fast Internet, dopamine-fired social media, lifelike gaming, and streaming video services with huge catalogues of content available on demand, everyone has to accept that far less technology will be available to them: they get no choice in the matter. Investigating how they might manage is at the very least an interesting thought experiment. But we would be foolish to consider such matters as purely part of a possible future and not instructive in other ways.

An Overlap of Interests

As readers of my previous articles will be aware, I have something of an interest in older computers, open source hardware, and sustainable computing. Older computers lend themselves to analysis and enhancement even by individuals with modest capabilities and tools because they employ technologies that may have been regarded as “miniaturised” when they were new, but they were still amenable to manual assembly and repair. Similarly, open source hardware has grown to a broad phenomenon because the means to make computing systems and accessories has now become more accessible to individuals, as opposed to being the preserve of large and well-resourced businesses. Where these activities experience challenges, it is typically in the areas that have not yet become quite as democratised, such as semiconductor fabrication at the large-scale integration level, along with the development and manufacture of more advanced technology, such as components and devices that would be competitive with off-the-shelf commercial products.

Some of the angst around open source hardware concerns the lack of investment it receives from those who would benefit from it, but much of that investment would largely be concerned with establishing an ability to maintain some kind of parity with modern, proprietary hardware. Ignoring such performance-led requirements and focusing on simpler hardware projects, as many people already do, brings us a lot closer to retrocomputing and a lot closer to the constrained hardware scenario envisaged by Collapse OS. My own experiments with PIC32-based microcontrollers are not too far removed from this, and it would not be inconceivable to run a simple environment in the 64K of RAM and 256K of flash memory of the PIC32MX270, this being much more generous than many microcomputers and games consoles of the 1980s.

Although I relied on cross-compilation to build the programs that would run on the minimal hardware of the PIC32 microcontroller, Collapse OS emphasises self-hosting: that it is possible to build the software within the running software itself. After all, how sustainable would a frugal computing environment be if it needed a much more powerful development system to fix and improve it? For Collapse OS, such self-hosting is enabled by the use of the Forth programming language, as explained by the rationale for switching to Forth from a system implemented in assembly language. Such use of Forth is not particularly unusual: its frugal demands were prized in the microcomputer era and earlier, with its creator Charles Moore describing the characteristics of a computer designed to run Forth as needing around 8K of RAM and 8K of ROM, this providing a complete interactive system.

(If you are interested in self-hosting and bootstrapping, one place to start might be the bootstrapping wiki.)

For a short while, Forth was perhaps even thought to be the hot new thing in some circles within computing. One fairly famous example was the Jupiter Ace microcomputer, developed by former Sinclair Research designers, offering a machine that followed on fairly closely from Sinclair’s rudimentary ZX81. But in a high-minded way one might have expected from the Sinclair stable and the Cambridge scene, it offered Forth as its built-in language in response to all the other microcomputers offering “unstructured” BASIC dialects. Worthy as such goals might have been, the introduction of a machine with outdated hardware specifications condemned it in its target market as a home computer, with it offering primitive black-and-white display output against competitors offering multi-colour graphics, and offering limited amounts of memory as competitors launched with far more fitted as standard. Interestingly, the Z80 processor at the heart of the Ace was the primary target of Collapse OS, and one might wonder if the latter might actually be portable to the former, which would be an interesting project if any hardware collector wants to give it a try!

Other Forth-based computers were delivered such as the Canon Cat: an unusual “information appliance” that might have formed the basis of Apple’s Macintosh had that project not been diverted towards following up on the Apple Lisa. Dedicated Forth processors were even delivered, as anticipated already by Moore back in 1980, reminiscent of the Lisp machine era. However, one hardware-related legacy of Forth is that of the Open Firmware standard where a Forth environment provides an interactive command-line interface to a system’s bootloader. Collapse OS fits in pretty well with that kind of application of Forth. Curiously, someone did contact me when I first wrote about my PIC32 experiments, this person maintaining their own microcontroller Forth implementation, and in the context of this article I have re-established contact because I never managed to properly follow up on the matter.

Changing the Context

According to a broad interpretation of the Collapse OS hardware criteria, the PIC32MX270 would actually not be a bad choice. Like the AVR microcontrollers and the microprocessors of the 1980s, PIC32MX microcontrollers are available in convenient dual in-line packages, but unlike those older microprocessors they also offer the 32-bit MIPS architecture that is nicer to program than the awkward instruction sets of the likes of the Z80 and 6502, no matter how much nostalgia colours people’s preferences. However, instead of focusing on hardware suitability in a resource-constrained future, I want to consider the messages of simplicity and sustainability that underpin the Collapse OS initiative and might be relevant to the way we practise computing today.

When getting a PIC32 microcontroller to produce a video signal, part of the motivation was just to see how straightforward it might be to make a simple “single chip” microcomputer. Like many microcomputers back in the 1980s, it became tempting to consider how it might be used to deliver graphical demonstrations and games, but I also wondered what kind of role such a system might have in today’s world. Similar projects, including the first versions of the Maximite have emphasised such things as well, along with interfacing and educational applications (such as learning BASIC). Indeed, many low-end microcontroller-based computers attempt to recreate and to emphasise the sparse interfaces of 1980s microcomputers as a distraction-free experience for learning and teaching.

Eliminating distractions is a worthy goal, whether those distractions are things that we can conveniently seek out when our attention wanders, such as all our favourite, readily accessible Internet content, or whether they come in the form of the notifications that plague “modern” user interfaces. Another is simply reducing the level of consumption involved in our computational activities: civilisational collapse would certainly impose severe limits on that kind of consumption, but it would seem foolish to acknowledge that and then continue on the same path of ever-increasing consumption that also increasingly fails to deliver significant improvements in the user experience. When desktop applications, mobile “apps”, and Web sites frequently offer sluggish and yet overly-simplistic interfaces that are more infuriating than anything else, it might be wise to audit our progress and reconsider how we do certain things.

Human nature has us constantly exploring the boundaries of what is possible with technology, but some things which captivate people at any given point on the journey of technological progress may turn out to be distracting diversions from the route ultimately taken. In my trawl of microcomputing history over the last couple of years, I was reminded of an absurd but illustrative example of how certain technological exercises seem to become the all-consuming focus of several developers, almost being developed for the sake of it, before the fad in question flames out and everybody moves on. That example concerned “morphing” software, inspired by visual effects from movies such as Terminator 2, but operating on a simpler, less convincing level.

Suddenly, such effects were all over the television and for a few months in late 1993, everyone was supposedly interested in making the likeness of one famous person slowly change into the likeness of another, never mind that it really required a good library of images, this being somewhat before widespread digital imaging and widespread image availability. Fast-forward a few years, and it all seemed like a crazy mass delusion best never spoken of again. We might want to review our own time’s obsessions with performative animations and effects, along with the peculiarities of touch-based interfaces, the assumption of pervasive and fast connectivity, and how these drive hardware consumption and obsolescence.

Once again, some of this comes back to asking how people managed to do things in earlier times and why things sometimes seem so complicated now. Thinking back to the 1980s era of microcomputing, my favourite 8-bit computer of those times was the Acorn Electron, this being the one I had back then, and it was certainly possible to equip it to do word processing to a certain level. Acorn even pitched an expanded version as a messaging terminal for British Telecom, although I personally think that they could have made more of such opportunities, especially given the machine’s 80-column text capabilities being made available at such a low price. The user experience would not exactly be appealing by today’s standards, but then nor would that of Collapse OS, either.

When I got my PIC32 experiment working reasonably, I asked myself if it would be sufficient for tasks like simple messaging and writing articles like this. The answer, assuming that I would enhance that effort to use a USB keyboard and external storage, is probably the same as whether anyone might use a Maximite for such applications: it might not be as comfortable as on a modern system but it would be possible in some way. Given the tricks I used, certain things would actually be regressions from the Electron, such as the display resolution. Conversely, the performance of a 48MHz MIPS-based processor is obviously going to be superior to a 2MHz 6502, even when having to generate the video signal, thus allowing for some potential in other areas.

Reversing Technological Escalation

Using low-specification hardware for various applications today, considering even the PIC32 as low-spec and ignoring the microcomputers of the past, would also need us to pare back the demands that such applications have managed to accumulate over the years. As more powerful, higher-performance hardware has become available, software, specifications and standards have opportunistically grown to take advantage of that extra power, leaving many people bewildered by the result: their new computer being just as slow as their old one, for example.

Standards can be particularly vulnerable where entrenched interests drive hardware consumption whilst seeking to minimise the level of adaptation their own organisations will need to undertake in order to deliver solutions based on such standards. A severely constrained computing device may not have the capacity or performance to handle all the quirks of a “full fat” standard, but it might handle an essential core of that standard, ignoring all the edge cases and special treatment for certain companies’ products. Just as important, the developers of an implementation handling a standard also may not have the capacity or tenacity for a “full fat” standard, but they may do a reasonable job handling one that cuts out all the corporate cruft.

And beyond the technology needed to perform some kind of transaction as part of an activity, we might reconsider what is necessary to actually perform that activity. Here, we may consider the more blatant case of the average “modern” Web site or endpoint, where an activity may end up escalating and involving the performance of a number of transactions, many of which superfluous and, in the case of the pervasive cult of analytics, exploitative. What once may have been a simple Web form is often now an “experience” where the browser connects to dozens of sites, where all the scripts poll the client computer into oblivion, and where the functionality somehow doesn’t manage to work, anyway (as I recently experienced on one airline’s Web site).

Technologists and their employers may drive consumption, but so do their customers. Public institutions, utilities and other companies may lazily rely on easily procured products and services, these insisting (for “security” or “the best experience”) that only the latest devices or devices from named vendors may be used to gain access. Here, the opposite of standardisation occurs, where adherence to brand names dictates the provision of service, compounded by the upgrade treadmill familiar from desktop computing, bringing back memories of Microsoft and Intel ostensibly colluding to get people to replace their computer as often as possible.

A Broader Brush

We don’t need to go back to retrocomputing levels of technology to benefit from re-evaluating the prevalent technological habits of our era. I have previously discussed single-board computers like the MIPS Creator CI20 which, in comparison to contemporary boards from the Raspberry Pi series, was fairly competitive in terms of specification and performance (having twice the RAM of the Raspberry Pi Models A+, B and B+). Although hardly offering conventional desktop performance upon its introduction, the CI20 would have made a reasonable workstation in certain respects in earlier times: its 1GHz CPU and 1GB of RAM should certainly be plenty for many applications even now.

Sadly, starting up and using the main two desktop environments on the CI20 is an exercise in patience, and I recommend trying something like the MATE desktop environment just for something responsive. Using a Web browser like Firefox is a trial, and extensive site blocking is needed just to prevent the browser wanting to download things from all over the place, as it tries to do its bit in shoring up Google’s business model. My father was asking me the other day why a ten-year-old computer might be slow on a “modern” Web site but still perfectly adequate for watching video. I would love to hear the Firefox and Chrome developers, along with the “architects of the modern Web”, give any explanation for this that doesn’t sound like they are members of some kind of self-realisation cult.

If we can envisage a microcomputer, either a vintage one or a modern microcontroller-based one, performing useful computing activities, then we can most certainly envisage machines of ten or so years ago, even ones behind the performance curve, doing so as well. And by realising that, we might understand that we might even have the power to slow down the engineered obsolescence of computing hardware, bring usable hardware back into use, and since not everyone on the planet can afford the latest and greatest, we might even put usable hardware into the hands of more people who might benefit from it.

Naturally, this perspective is rather broader than one that only considers a future of hardship and scarcity, but hardship and scarcity are part of the present, just as they have always been part of the past. Applying many of the same concerns and countermeasures to today’s situation, albeit in less extreme forms, means that we have the power to mitigate today’s situation and, if we are optimistic, perhaps steer it away from becoming the extreme situation that the Collapse OS initiative seeks to prepare for.

Concrete Steps

I have, in the past, been accused of complaining about injustices too generally for my complaints to be taken seriously, never mind such injustices being blatant and increasingly obvious in our modern societies and expressed through the many crises of our times. So how might we seek to mitigate widespread hardware obsolescence and technology-driven overconsumption? Some suggestions in a concise list for those looking for actionable things:

• Develop, popularise and mandate lightweight formats, protocols and standards
• Encourage interoperability and tolerance for multiple user interfaces, clients and devices
• Insist on an unlimited “right to repair” for computing devices including the software
• Encourage long-term thinking in software and systems development

And now for some elucidation…

Mandatory Accessible Standards

This suggestion has already been described above, but where it would gain its power is in the idea of mandating that public institutions and businesses would be obliged to support lightweight formats, protocols and standards, and not simply as an implementation detail for their chosen “app”, like a REST endpoint might be, but actually as a formal mechanism providing service to those who would interact with those institutions. This would make the use of a broad range of different devices viable, and in the case of special-purpose devices for certain kinds of users, particularly those who would otherwise be handed a smartphone and told to “get with it”, it would offer a humane way of accessing services that is currently denied to them.

For simple dialogue-based interactions, existing formats such as JSON might even be sufficient as they are. I am reminded of a paper I read when putting together my degree thesis back in the 1990s, where the idea was that people would be able to run programs safely in their mail reader, with one example being that of submitting forms.

T-shirt ordering dialogues shown by Safe-Tcl running in a mail program, offering the recipient the chance to order some merchandise that might not be as popular now.

In that paper, most of the emphasis was on the safety of the execution environment as opposed to the way in which the transaction was to be encoded, but it is not implausible that one might have encoded the details of the transaction – the T-shirt size (with the recipient’s physical address presumably already being known to the sender) – in a serialised form of the programming language concerned (Safe-Tcl) as opposed to just dumping some unstructured text in the body of a mail. I would need to dig out my own thesis to see what ideas I had for serialised information. Certainly, such transactions even embellished with other details and choices and with explanatory information, prompts and questions do not require megabytes of HTML, CSS, JavaScript, images, videos and so on.

Interoperability and Device Choice

One thing that the Web was supposed to liberate us from was the insistence that to perform a particular task, we needed a particular application, and that particular application was only available on a particular platform. In the early days, HTML was deliberately simplistic in its display capabilities, and people had to put up with Web pages that looked very plain until things like font tags allowed people to go wild. With different factions stretching HTML in all sorts of directions, CSS was introduced to let people apply presentation attributes to documents, supposedly without polluting or corrupting the original HTML that would remain semantically pure. We all know how this turned out, particularly once the Web 2.0 crowd got going.

Back in the 1990s, I worked on an in-house application at my employer that used a document model inspired by SGML (as HTML had been), and the graphical user interface to the documents being exchanged initially offered a particular user interface paradigm when dealing with collections of data items, this being the one embraced by the Macintosh’s Finder when showing directory hierarchies in what we would now call a tree view. Unfortunately, users seemed to find expanding and hiding things by clicking on small triangles to be annoying, and so alternative presentation approaches were explored. Interestingly, the original paradigm would be familiar even now to those using generic XML editor software, but many people would accept that while such low-level editing capabilities are nice to have, higher-level representations of the data are usually much more preferable.

Such user preferences could quite easily be catered to through the availability of client software that works in the way they expect, rather than the providers of functionality or the operators of services trying to gauge what the latest fashions in user interfaces might be, as we have all seen when familiar Web sites change to mimic something one would expect to see on a smartphone, even with a large monitor on a desk with plenty of pixels to spare. With well-defined standards, if a client device or program were to see that it needed to allow a user to peruse a large collection of items or to choose a calendar date, it would defer to the conventions of that device or platform, giving the user the familiarity they expect.

This would also allow clients and devices with a wide range of capabilities to be used. The Web tried to deliver a reasonable text-only experience for a while, but most sites can hardly be considered usable in a textual browser these days. And although there is an “accessibility story” for the Web, it largely appears to involve retrofitting sites with semantic annotations to help users muddle through the verbose morass encoded in each page. Certainly, the Web of today does do one thing reasonably by mixing up structure and presentation: it can provide a means of specifying and navigating new kinds of data that might be unknown to the client, showing them something more than a text box. A decent way of extending the range of supported data types would be needed in any alternative, but it would need to spare everyone suddenly having scripts running all over the place.

Rights to Repair

The right to repair movement has traditionally been focused on physical repairs to commercial products, making sure that even if the manufacturer has abandoned a product and really wants you to buy something new from them, you can still choose to have the product repaired so that it can keep serving you well for some time to come. But if hardware remains capable enough to keep doing its job, and if we are able to slow down or stop the forces of enforced obsolescence, we also need to make sure that the software running on the hardware may also be repaired, maintained and updated. A right to repair very much applies to software.

Devotees of the cult of the smartphone, those who think that there is an “app” for everything, should really fall silent with shame. Not just for shoehorning every activity they can think of onto a device that is far from suitable for everyone, and not just for mandating commercial relationships with large multinational corporations for everyone, but also for the way that happily functioning smartphones have to be discarded because they run software that is too old and cannot be fixed or upgraded. Demanding the right to exercise the four freedoms of Free Software for our devices means that we get to decide when those devices are “too old” for what we want to use them for. If a device happens to be no longer usable for its original activity even after some Free Software repairs, we can repurpose it for something else, instead of having the vendor use those familiar security scare stories and pretending that they are acting in our best interests.

Long-Term Perspectives

If we are looking to preserve the viability of our computing devices by demanding interoperability to give them a chance to participate in the modern world and by demanding that they may be repaired, we also need to think about how the software we develop may itself remain viable, both in terms of the ability to run the software on available devices as well as the ability to keep maintaining, improving and repairing it. That potentially entails embracing unfashionable practices because “modern” practices do not exactly seem conducive to the kind of sustainable activities we have in mind.

I recently had the opportunity to contemplate the deployment of software in “virtual environments” containing entire software stacks, each of which running their own Web server program, that would receive their traffic from another Web server program running in the same virtual machine, all of this running in some cloud infrastructure. It was either that or using containers containing whole software distributions, these being deployed inside virtual machines containing their own software distributions. All because people like to use the latest and greatest stuff for everything, this stuff being constantly churned by fashionable development methodologies and downloaded needlessly over and over again from centralised Internet services run by monopolists.

Naturally, managing gigabytes of largely duplicated software is worlds, if not galaxies, away from the modest computing demands of things like Collapse OS, but it would be distasteful to anyone even a decade ago and shocking to anyone even a couple of decades ago. Unfashionable as it may seem now, software engineering courses once emphasised things like modularity and the need for formal interfaces between modules in systems. And a crucial benefit of separating out functionality into modules is to allow those modules to mature, do the job they were designed for, and to recede into the background and become something that can be relied upon and not need continual, intensive maintenance. There is almost nothing better than writing a library that one may use constantly but never need to touch again.

Thus, the idea that a precarious stack of precisely versioned software is required to deliver a solution is absurd, but it drives the attitude that established software distributions only deliver “old” software, and it drives the demand for wasteful container or virtual environment solutions whose advocates readily criticise traditional distributions whilst pilfering packages from them. Or as Docker users might all too easily say, “FROM debian:sid”. Part of the problem is that it is easy to rely on methods of mass consumption to solve problems with software – if something is broken, just update and see if it fixes it – but such attitudes permeate the entire development process, leading to continual instability and a software stack constantly in flux.

Dealing with a multitude of software requirements is certainly a challenging problem that established operating systems struggle to resolve convincingly, despite all the shoehorning of features into the Linux technology stack. Nevertheless, the topic of operating system design is rather outside the scope of this article. Closer to relevance is the matter of how people seem reluctant to pick a technology and stick with it, particularly in the realm of programming languages. Then again, I covered much of this before and fairly recently, too. Ultimately, we want to be establishing software stacks that people can readily acquaint themselves with decades down the line, without the modern-day caveats that “feature X changed in version Y” and that if you were not there at the time, you have quite the job to do to catch up with that and everything else that went on, including migrations to a variety of source management tools and venues, maybe even completely new programming languages.

A Different Mindset

If anything, Collapse OS makes us consider a future beyond tomorrow, next week, next year, or a few years’ time. Even if the wheels do not start falling off the vehicle of human civilisation, there are still plenty of other things that can go away without much notice. Corporations like Apple and Google might stick around, whether that is good news or not, but it does not stop them from pulling the plug on products and services. Projects and organisations do not always keep going forever, not least because they are often led by people who cannot keep going forever, either.

There are ways we can mitigate these threats to sustainability and longevity, however. We can share our software through accessible channels without insisting that others use those monopolist-run centralised hosting services. We can document our software so that others have a chance of understanding what we were thinking when we wrote it. We can try and keep the requirements for our software modest and give people a chance to deploy it on modest hardware. And we might think about what kind of world we are leaving behind and whether it is better than the world we were born into.

Come to Barcelona for Akademy 2022!

• TSDgeos' blog
• —
• 21:16, Monday, 15 August 2022

Akademy is KDE's yearly community event, this year it happens between October 1st and October 7th in my city comarca [one of the reasons of why it's happening is that our Mr. President tricked me into helping organize it [*]]

You don't need to be a "KDE expert" to join, if you're remotely involved or interested in KDE you should really attend if you can (in person if possible, for me online really doesn't work for conferences), and not only the weekend of talks, but the whole week!

I still remember 2007 when our back-then-newbie Mr. President asked me "should I really go to Akademy? All the week? is it really worth it?" and i said "as many days as you can", and I guess we made a good enough impression to convince him to stay around and even want to do all the paper work that involves being in the KDE eV board :D

Anyhow, what i said, come to Akademy 2022! It's free, you'll learn a lot, meet nice people, will be both fun and productive.

And you should too! Register today!

[*] I should really work on my "no" skills, I'm still working on Okular because decades ago i said "how hard can it be" when someone asked to update KPDF to use a newer version of a dependency.

On a road to Prizren with a Free Software Phone

• Colors of Noise - Entries tagged planetfsfe
• —
• 14:06, Friday, 12 August 2022

Since people are sometimes slightly surprised that you can go onto a multi week trip with a smartphone running free sofware so only I wanted to share some impressions from my recent trip to Prizren/Kosovo to attend Debconf 22 using a Librem 5. It's a mix of things that happend and bits that got improved to hopefully make things more fun to use. And, yes, there won't be any big surprises like being stranded without the ability to do phone calls in this read because there weren't and there shouldn't be.

After two online versions Debconf 22 (the annual Debian Conference) took place in Prizren / Kosovo this year and I sure wanted to go. Looking for options I settled for a train trip to Vienna, to meet there with friends and continue the trip via bus to Zagreb, then switching to a final 11h direct bus to Prizren.

When preparing for the trip and making sure my Librem 5 phone has all the needed documents I noticed that there will be quite some PDFs to show until I arrive in Kosovo: train ticket, bus ticket, hotel reservation, and so on. While that works by tapping unlocking the phone, opening the file browser, navigating to the folder with the PDFs and showing it via evince this looked like a lot of steps to repeat. Can't we have that information on the Phone Shell's lockscreen?

This was a good opportunity to see if the upcoming plugin infrastructure for the lock screen (initially meant to allow for a plugin to show upcoming events) was flexible enough, so I used some leisure time on the train to poke at this and just before I reached Vienna I was able to use it for the first time. It was the very last check of that ticket, it also was a bit of cheating since I didn't present the ticket on the phone itself but from phosh (the phones graphical shell) running on my laptop but still.

This was possible since phosh is written in GTK and so I could just leverage evince's EvView. Unfortunately the hotel check in didn't want to see any documents ☹.

For the next day I moved the code over to the Librem 5 and (being a bit nervous as the queue to get on the bus was quite long) could happily check into the Flixbus by presenting the barcode to the barcode reader via the Librem 5's lockscreen.

When switching to the bus to Prizren I didn't get to use that feature again as we bought the tickets at a counter but we got a nice krem banana after entering the bus - they're not filled with jelly, but krem - a real Kosovo must eat!).

Although it was a rather long trip we had frequent breaks and I'd certainly take the same route again. Here's a photo of Prizren taken on the Librem 5 without any additional postprocessing:

What about seeing the conference schedule on the phone? Confy(a conferences schedule viewer using GTK and libhandy) to the rescue:

Since Debian's confy maintainer was around too, confy saw a bunch of improvements over the conference.

For getting around Puremaps(an application to display maps and show routing instructions) was very helpful, here geolocating me in Prizren via GPS:

Puremaps currently isn't packaged in Debian but there's work onging to fix that (I used the flatpak for the moment).

We got ourselves sim cards for the local phone network. For some reason mine wouldn't work (other sim cards from the same operator worked in my phone but this one just wouldn't). So we went to the sim card shop and the guy there was perfectly able to operate the Librem 5 without further explanation (including making calls, sending USSD codes to query balance, …). The sim card problem turned out to be a problem on the operator side and after a couple of days they got it working.

We had nice, sunny weather about all the time. That made me switch between high contrast mode (to read things in bright sunlight) and normal mode (e.g. in conference rooms) on the phone quite often. Thankfully we have a ambient light sensor in the phone so we can make that automatic.

See here for a video.

Jathan kicked off a DebianOnMobile sprint during the conference where we were able to improve several aspects of mobile support in Debian and on Friday I had the chance to give a talk about the state of Debian on smartphones. pdf-presenter-console is a great tool for this as it can display the current slide together with additional notes. I needed some hacks to make it fit the phone screen but hopefully we figure out a way to have this by default.

I had two great weeks in Prizren. Many thanks to the organizers of Debconf 22 - I really enjoyed the conference.