February 25, 2023

25 February, 2023 10:36PM · ...

---
 name: pyupgrade
 command: 'pyupgrade --exit-zero-even-if-changed $(find -name "test_*.py")'
 mode: propose
 merge-request:
   commit-message: Upgrade Python code to a modern version

candidates.yaml
 ---
 - url: https://github.com/jelmer/dulwich
 - url: https://github.com/jelmer/xandikos

 $ svp batch generate --recipe=pyupgrade.yaml --candidates=candidate.syml pyupgrade

pyupgrade
$ ls pyupgrade
batch.yaml  dulwich  xandikos

$ cd pyupgrade/dulwich
$ git log
commit 931f9ffb26e9143c56f20e0b85e6ddb0a8eee2eb (HEAD -> master)
Author: Jelmer Vernooĳ <[email protected]>
Date:   Sat Feb 25 22:28:12 2023 +0000

Run pyupgrade

diff --git a/dulwich/tests/compat/test_client.py b/dulwich/tests/compat/test_client.py
index 02ab6c0a..9b0661ed 100644
--- a/dulwich/tests/compat/test_client.py
+++ b/dulwich/tests/compat/test_client.py
@@ -628,7 +628,7 @@ class HTTPGitServer(http.server.HTTPServer):
         self.server_name = "localhost"

     def get_url(self):
-        return "http://{}:{}/".format(self.server_name, self.server_port)
+        return f"http://{self.server_name}:{self.server_port}/"


 class DulwichHttpClientTest(CompatTestCase, DulwichClientTestBase):
...

batch.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
name: pyupgrade
work:
- url: https://github.com/dulwich/dulwich
  name: dulwich
  description: Upgrade to modern Python statements
  commit-message: Run pyupgrade
  mode: propose
- url: https://github.com/jelmer/xandikos
  name: xandikos
  description: Upgrade to modern Python statements
  commit-message: Run pyupgrade
  mode: propose
recipe: ../pyupgrade.yaml

batch.yaml
$ svp batch publish pyupgrade

batch.yaml
batch.yaml
svp batch status
$ svp batch status pyupgrade

svp batch publish pyupgrade

(setq linkifier-patterns '(
          ("\\\<XXX-[0-9]+\\\>" "https://jira.example.com/browse/%s")
          ("\\\<BUG-[0-9]+\\\>" "https://bugzilla.example.com/show?id=%s")))

[[jira:XXX-1234][XXX-1234]]

(add-to-list 'org-link-abbrev-alist
    '("jira" . "http://jira.example.com/browse/%s"))

 // Parse a frequency using hz.tools/rf.ParseHz, and print it to stdout.
 freq := rf.MustParseHz("-10kHz")
fmt.Printf("Frequency: %s\n", freq+rf.MHz)
// Prints: 'Frequency: 990kHz'

// Return the Intersection between two RF ranges, and print
 // it to stdout.
 r1 := rf.Range{rf.KHz, rf.MHz}
r2 := rf.Range{rf.Hz(10), rf.KHz * 100}
fmt.Printf("Range: %s\n", r1.Intersection(r2))
// Prints: Range: 1000Hz->100kHz

 // Basic sketch for playing FM radio using a reader stream from
 // an SDR or other IQ stream.

bandwidth := 150*rf.KHz
reader, err = stream.ConvertReader(reader, sdr.SampleFormatC64)
if err != nil {
...
}
demod, err := fm.Demodulate(reader, fm.DemodulatorConfig{
Deviation: bandwidth / 2,
Downsample: 8, // some value here depending on sample rate
 Planner: fftw.Plan,
})
if err != nil {
...
}
speaker, err := pulseaudio.NewWriter(pulseaudio.Config{
Format: pulseaudio.SampleFormatFloat32NE,
Rate: demod.SampleRate(),
AppName: "rf",
StreamName: "fm",
Channels: 1,
SinkName: "",
})
if err != nil {
...
}
buf := make([]float32, 1024*64)
for {
i, err := demod.Read(buf)
if err != nil {
...
}
if i == 0 {
panic("...")
}
if err := speaker.Write(buf[:i]); err != nil {
...
}
}

hcitool lescan

# lspci
0000:00:00.0 PCI bridge: PLDA XpressRich-AXI Ref Design (rev 02)
0000:01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 Controller (rev 01)
0001:00:00.0 PCI bridge: PLDA XpressRich-AXI Ref Design (rev 02)
0001:01:00.0 Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]

# mkfs -t ext4 /dev/nvme0n1p1
# mount /dev/nvme0n1p1 /mnt
# debootstrap --keyring=/etc/apt/trusted.gpg.d/debian-ports-archive-2023.gpg \
	unstable /mnt https://deb.debian.org/debian-ports

$ time make -j 4 bindeb-pkg
… [linux-image-5.15.94-00787-g1fbe8ac32aa8]
real	37m0.134s
user	117m27.392s
sys	6m49.804s

[155933.434038] nvme nvme0: I/O 436 QID 4 timeout, completion polled
[156173.351166] nvme nvme0: I/O 48 QID 3 timeout, completion polled
[156346.228993] nvme nvme0: I/O 108 QID 3 timeout, completion polled

[ FC Stegerman ]
* Update code to match latest version of Black. (Closes: #1031433)

[ Chris Lamb ]
* Require at least Black version 23.1.0 to run the internal Black tests.
* Update copyright years.

. /etc/default/grub
echo "root=/dev/mapper/root $GRUB_CMDLINE_LINUX $GRUB_CMDLINE_LINUX_DEFAULT" > /etc/kernel/cmdline

apt install systemd-boot

bootctl install
for kernel in /boot/vmlinuz-*; do
  kernel-install add "${kernel#*/vmlinuz-}" "$kernel"
done

bootctl set-timeout 4

cat <<END > /boot/efi/loader/entries/grub.conf
title Grub
linux /EFI/debian/grubx64.efi
END

sbctl create-keys
sbctl enroll-keys
for file in /boot/efi/*/*/linux /boot/efi/EFI/*/*.efi; do
  ./sbctl sign -s $file
done

mount -o loop /boot/grml/grml64-full_2022.11.iso /mnt/
mkdir /boot/efi/grml/
cp /mnt/boot/grml64full/* /boot/efi/grml/
umount /mnt/

cat <<END > /boot/efi/loader/entries/grml.conf
title GRML
linux /grml/vmlinuz
initrd /grml/initrd.img
options boot=live bootid=grml64full202211 findiso=/grml/grml64-full_2022.11.iso live-media-path=/live/grml64-full net.ifnames=0 
END

bootctl
System:
      Firmware: UEFI 2.70 (Lenovo 0.4496)
 Firmware Arch: x64
   Secure Boot: enabled (user)
  TPM2 Support: yes
  Boot into FW: supported

Current Boot Loader:
      Product: systemd-boot 252.5-2
     Features: ✓ Boot counting
               ✓ Menu timeout control
               ✓ One-shot menu timeout control
               ✓ Default entry control
               ✓ One-shot entry control
               ✓ Support for XBOOTLDR partition
               ✓ Support for passing random seed to OS
               ✓ Load drop-in drivers
               ✓ Support Type #1 sort-key field
               ✓ Support @saved pseudo-entry
               ✓ Support Type #1 devicetree field
               ✓ Boot loader sets ESP information
          ESP: /dev/disk/by-partuuid/1b767f8e-70fa-5a48-b444-cfe5c272d66e
         File: └─/EFI/systemd/systemd-bootx64.efi

...

ConditionIPExpr "condition on IP" ←
  column:("ExporterAddress"i { return "ExporterAddress", nil }
        / "SrcAddr"i { return "SrcAddr", nil }
        / "DstAddr"i { return "DstAddr", nil }) _ 
  operator:("=" / "!=") _ 
  ip:IP {
    return fmt.Sprintf("%s %s IPv6StringToNum(%s)",
      toString(column), toString(operator), quote(ip)), nil
  }

IP "IP address" ← [0-9A-Fa-f:.]+ {
  ip, err := netip.ParseAddr(string(c.text))
  if err != nil {
    return "", errors.New("expecting an IP address")
  }
  return ip.String(), nil
}

WHERE InIfBoundary = 'external' 
AND ExporterRegion = 'france' 
AND InIfConnectivity = 'transit' 
AND SrcAS = 15169 
AND DstAddr BETWEEN toIPv6('2a01:e0f:ffff::') 
                AND toIPv6('2a01:e0f:ffff:ffff:ffff:ffff:ffff:ffff')

@top Filter {
  expression
}

expression {
 Not expression |
 "(" expression ")" |
 "(" expression ")" And expression |
 "(" expression ")" Or expression |
 comparisonExpression And expression |
 comparisonExpression Or expression |
 comparisonExpression
}
comparisonExpression {
 Column Operator Value
}

Value {
  String | Literal | ValueLParen ListOfValues ValueRParen
}
ListOfValues {
  ListOfValues ValueComma (String | Literal) |
  String | Literal
}

// […]
@tokens {
  // […]
  Column { std.asciiLetter (std.asciiLetter|std.digit)* }
  Operator { $[a-zA-Z!=><]+ }

  String {
    '"' (![\\\n"] | "\\" _)* '"'? |
    "'" (![\\\n'] | "\\" _)* "'"?
  }
  Literal { (std.digit | std.asciiLetter | $[.:/])+ }
  // […]
}

Filter(Column, Operator, Value(Literal),
  And, Column, Operator, Value(Literal),
  Or, Column, Operator, Value(Literal))

export const FilterLanguage = LRLanguage.define({
  parser: parser.configure({
    props: [
      styleTags({
        Column: t.propertyName,
        String: t.string,
        Literal: t.literal,
        LineComment: t.lineComment,
        BlockComment: t.blockComment,
        Or: t.logicOperator,
        And: t.logicOperator,
        Not: t.logicOperator,
        Operator: t.compareOperator,
        "( )": t.paren,
      }),
    ],
  }),
});

{
  "message": "at line 1, position 12: string literal not terminated",
  "errors": [{
    "line":    1,
    "column":  12,
    "offset":  11,
    "message": "string literal not terminated",
  }]
}

results := []struct {
  Label  string `ch:"label"`
  Detail string `ch:"detail"`
}{}
columnName := "DstAS"
sqlQuery := fmt.Sprintf(`
 SELECT concat('AS', toString(%s)) AS label, dictGet('asns', 'name', %s) AS detail
 FROM flows
 WHERE TimeReceived > date_sub(minute, 1, now())
 AND detail != ''
 AND positionCaseInsensitive(detail, $1) >= 1
 GROUP BY label, detail
 ORDER BY COUNT(*) DESC
 LIMIT 20
`, columnName, columnName)
if err := conn.Select(ctx, &results, sqlQuery, input.Prefix); err != nil {
  c.r.Err(err).Msg("unable to query database")
  break
}
for _, result := range results {
  completions = append(completions, filterCompletion{
    Label:  result.Label,
    Detail: result.Detail,
    Quoted: false,
  })
}

import/jon/amex/%.journal: import/jon/amex/%.csv rules/amex.csv.rules
    rm -f $(@D)/.latest.$*.csv $@
    hledger import --rules-file rules/amex.csv.rules -f $@ $<

import/opening/2023.csv: 2022.journal
    mkdir -p import/opening
    hledger bal -f $< \
                $(list_of_accounts_I_want_to_carry_over) \
        -O csv -N > $@

import/opening/2023.journal: import/opening/2023.csv rules/opening.rules
    rm -f $(@D)/.latest.2023.csv $@
    hledger import --rules-file rules/opening.rules \
        -f $@ $<

2022-01-02 ZTL*RELISH
    family:liabilities:creditcard      £ -3.00
    family:dues:jon                     £ 3.00
    (jon:expenses:snacks)               £ 3.00

2022-01-02 ZTL*RELISH
    family:liabilities:creditcard      £ -3.00
    family:liabilities:jon:snacks

[ Akihiro Suda ]
* Update .gitlab-ci.yml to push versioned tags to the container registry.
  (Closes: reproducible-builds/diffoscope!119)

[ Chris Lamb ]
* Fix compatibility with PyPDF2. (Closes: reproducible-builds/diffoscope#331)
* Fix compatibility with ImageMagick 7.1.
  (Closes: reproducible-builds/diffoscope#330)

[ Daniel Kahn Gillmor ]
* Update from PyPDF2 to pypdf. (Closes: #1029741, #1029742)

[ FC Stegerman ]
* Add support for Android resources.arsc files.
  (Closes: reproducible-builds/diffoscope!116)
* Add support for dexdump. (Closes: reproducible-builds/diffoscope#134)
* Improve DexFile's FILE_TYPE_RE and add FILE_TYPE_HEADER_PREFIX, and remove
  "Dalvik dex file" from ApkFile's FILE_TYPE_RE as well.

[ Efraim Flashner ]
* Update external tool for isoinfo on guix.
  (Closes: reproducible-builds/diffoscope!124)

#!/bin/bash

# Either rb3011 (arm) or rb5009 (arm64)
#HOSTNAME="rb3011"
HOSTNAME="rb5009"

if [ "x${HOSTNAME}" == "xrb3011" ]; then
	ARCH=armhf
elif [ "x${HOSTNAME}" == "xrb5009" ]; then
	ARCH=arm64
else
	echo "Unknown host: ${HOSTNAME}"
	exit 1
fi

BASE_DIR=$(dirname $0)
IMAGE_FILE=$(mktemp --tmpdir router.${ARCH}.XXXXXXXXXX.img)
MOUNT_POINT=$(mktemp -p /mnt -d router.${ARCH}.XXXXXXXXXX)

# Build and mount an ext4 image file to put the root file system in
dd if=/dev/zero bs=1 count=0 seek=1G of=${IMAGE_FILE}
mkfs -t ext4 ${IMAGE_FILE}
mount -o loop ${IMAGE_FILE} ${MOUNT_POINT}

# Add dpkg excludes
mkdir -p ${MOUNT_POINT}/etc/dpkg/dpkg.cfg.d/
cat <<EOF > ${MOUNT_POINT}/etc/dpkg/dpkg.cfg.d/path-excludes
# Exclude docs
path-exclude=/usr/share/doc/*

# Only locale we want is English
path-exclude=/usr/share/locale/*
path-include=/usr/share/locale/en*/*
path-include=/usr/share/locale/locale.alias

# No man pages
path-exclude=/usr/share/man/*
EOF

# Setup fstab + mtab
echo "# Empty fstab as root is pre-mounted" > ${MOUNT_POINT}/etc/fstab
ln -s ../proc/self/mounts ${MOUNT_POINT}/etc/mtab

# Setup hostname
echo ${HOSTNAME} > ${MOUNT_POINT}/etc/hostname

# Add the root SSH keys
mkdir -p ${MOUNT_POINT}/root/.ssh/
cat <<EOF > ${MOUNT_POINT}/root/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8NkUeVdsVdegS+JT9qwFwiHEgcC9sBwnv6RjpH6I4d3im4LOaPOatzneMTZlH8Gird+H4nzluciBr63hxmcFjZVW7dl6mxlNX2t/wKvV0loxtEmHMoI7VMCnrWD0PyvwJ8qqNu9cANoYriZRhRCsBi27qPNvI741zEpXN8QQs7D3sfe4GSft9yQplfJkSldN+2qJHvd0AHKxRdD+XTxv1Ot26+ZoF3MJ9MqtK+FS+fD9/ESLxMlOpHD7ltvCRol3u7YoaUo2HJ+u31l0uwPZTqkPNS9fkmeCYEE0oXlwvUTLIbMnLbc7NKiLgniG8XaT0RYHtOnoc2l2UnTvH5qsQ== [email protected]
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQb9+qFemcwKhey3+eTh5lxp+3sgZXW2HQQEZMt9hPvVXk+MiiNMx9WUzxPJnwXqlmmVdKsq+AvjA0i505Pp8fIj5DdUBpSqpLghmzpnGuob7SSwXYj+352hjD52UC4S0KMKbIaUpklADgsCbtzhYYc4WoO8F7kK63tS5qa1XSZwwRwPbYOWBcNocfr9oXCVWD9ismO8Y0l75G6EyW8UmwYAohDaV83pvJxQerYyYXBGZGY8FNjqVoOGMRBTUcLj/QTo0CDQvMtsEoWeCd0xKLZ3gjiH3UrknkaPra557/TWymQ8Oh15aPFTr5FvKgAlmZaaM0tP71SOGmx7GpCsP4jZD1Xj/7QMTAkLXb+Ou6yUOVM9J4qebdnmF2RGbf1bwo7xSIX6gAYaYgdnppuxqZX1wyAy+A2Hie4tUjMHKJ6OoFwBsV1sl+3FobrPn6IuulRCzsq2aLqLey+PHxuNAYdSKo7nIDB3qCCPwHlDK52WooSuuMidX4ujTUw7LDTia9FxAawudblxbrvfTbg3DsiDBAOAIdBV37HOAKu3VmvYSPyqT80DEy8KFmUpCEau59DID9VERkG6PWPVMiQnqgW2Agn1miOBZeIQV8PFjenAySxjzrNfb4VY/i/kK9nIhXn92CAu4nl6D+VUlw+IpQ8PZlWlvVxAtLonpjxr9OTw== noodles@yubikey
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0I8UHj4IpfqUcGE4cTvLB0d2xmATSUzqtxW6ZhGbZxvQDKJesVW6HunrJ4NFTQuQJYgOXY/o82qBpkEKqaJMEFHTCjcaj3M6DIaxpiRfQfs0nhtzDB6zPiZn9Suxb0s5Qr4sTWd6iI9da72z3hp9QHNAu4vpa4MSNE+al3UfUisUf4l8TaBYKwQcduCE0z2n2FTi3QzmlkOgH4MgyqBBEaqx1tq7Zcln0P0TYZXFtrxVyoqBBIoIEqYxmFIQP887W50wQka95dBGqjtV+d8IbrQ4pB55qTxMd91L+F8n8A6nhQe7DckjS0Xdla52b9RXNXoobhtvx9K2prisagsHT noodles@cup
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK6iGog3WbNhrmrkglNjVO8/B6m7mN6q1tMm1sXjLxQa+F86ETTLiXNeFQVKCHYrk8f7hK0d2uxwgj6Ixy9k0Cw= noodles@sevai
EOF

# Bootstrap our install
debootstrap \
	--arch=${ARCH} \
	--include=collectd-core,conntrack,dnsmasq,ethtool,iperf3,kexec-tools,mosquitto,mtd-utils,mtr-tiny,ppp,tcpdump,rng-tools5,ssh,watchdog,wget \
	--exclude=dmidecode,isc-dhcp-client,isc-dhcp-common,makedev,nano \
	bullseye ${MOUNT_POINT} https://deb.debian.org/debian/

# Install mqtt-arp
cp ${BASE_DIR}/debs/mqtt-arp_1_${ARCH}.deb ${MOUNT_POINT}/tmp
chroot ${MOUNT_POINT} dpkg -i /tmp/mqtt-arp_1_${ARCH}.deb
rm ${MOUNT_POINT}/tmp/mqtt-arp_1_${ARCH}.deb

# Frob the mqtt-arp config so it starts after mosquitto
sed -i -e 's/After=.*/After=mosquitto.service/' ${MOUNT_POINT}/lib/systemd/system/mqtt-arp.service

# Frob watchdog so it starts earlier than multi-user
sed -i -e 's/After=.*/After=basic.target/' ${MOUNT_POINT}/lib/systemd/system/watchdog.service

# Make sure the watchdog is poking the device file
sed -i -e 's/^#watchdog-device/watchdog-device/' ${MOUNT_POINT}/etc/watchdog.conf

# Clean up docs + locales
rm -r ${MOUNT_POINT}/usr/share/doc/*
rm -r ${MOUNT_POINT}/usr/share/man/*
for dir in ${MOUNT_POINT}/usr/share/locale/*/; do
	if [ "${dir}" != "${MOUNT_POINT}/usr/share/locale/en/" ]; then
		rm -r ${dir}
	fi
done

# Set root password to root
echo "root:root" | chroot ${MOUNT_POINT} chpasswd

# Add security to sources.list + update
echo "deb https://security.debian.org/debian-security bullseye-security main" >> ${MOUNT_POINT}/etc/apt/sources.list
chroot ${MOUNT_POINT} apt update
chroot ${MOUNT_POINT} apt -y full-upgrade
chroot ${MOUNT_POINT} apt clean

# Cleanup the APT lists
rm ${MOUNT_POINT}/var/lib/apt/lists/www.*
rm ${MOUNT_POINT}/var/lib/apt/lists/security.*

# Disable the daily APT timer
rm ${MOUNT_POINT}/etc/systemd/system/timers.target.wants/apt-daily.timer

# Disable daily dpkg backup
cat <<EOF > ${MOUNT_POINT}/etc/cron.daily/dpkg
#!/bin/sh

# Don't do the daily dpkg backup
exit 0
EOF

# We don't want a persistent systemd journal
rmdir ${MOUNT_POINT}/var/log/journal

# Enable nftables
ln -s /lib/systemd/system/nftables.service \
	${MOUNT_POINT}/etc/systemd/system/sysinit.target.wants/nftables.service

# Add systemd-coredump + systemd-timesync user / group
echo "systemd-timesync:x:998:" >> ${MOUNT_POINT}/etc/group
echo "systemd-coredump:x:999:" >> ${MOUNT_POINT}/etc/group
echo "systemd-timesync:!*::" >> ${MOUNT_POINT}/etc/gshadow
echo "systemd-coredump:!*::" >> ${MOUNT_POINT}/etc/gshadow
echo "systemd-timesync:x:998:998:systemd Time Synchronization:/:/usr/sbin/nologin" >> ${MOUNT_POINT}/etc/passwd
echo "systemd-coredump:x:999:999:systemd Core Dumper:/:/usr/sbin/nologin" >> ${MOUNT_POINT}/etc/passwd
echo "systemd-timesync:!*:47358::::::" >> ${MOUNT_POINT}/etc/shadow
echo "systemd-coredump:!*:47358::::::" >> ${MOUNT_POINT}/etc/shadow

# Create /etc/.pwd.lock, otherwise it'll end up in the overlay
touch ${MOUNT_POINT}/etc/.pwd.lock
chmod 600 ${MOUNT_POINT}/etc/.pwd.lock

# Copy config files
cp --recursive --preserve=mode,timestamps ${BASE_DIR}/etc/* ${MOUNT_POINT}/etc/
cp --recursive --preserve=mode,timestamps ${BASE_DIR}/etc-${ARCH}/* ${MOUNT_POINT}/etc/
chroot ${MOUNT_POINT} chown mosquitto /etc/mosquitto/mosquitto.users
chroot ${MOUNT_POINT} chown mosquitto /etc/ssl/mqtt.home.key

# Build symlinks into flash for boot / modules
ln -s /mnt/flash/lib/modules ${MOUNT_POINT}/lib/modules
rmdir ${MOUNT_POINT}/boot
ln -s /mnt/flash/boot ${MOUNT_POINT}/boot

# Put our git revision into os-release
echo -n "GIT_VERSION=" >> ${MOUNT_POINT}/etc/os-release
(cd ${BASE_DIR} ; git describe --tags) >> ${MOUNT_POINT}/etc/os-release

# Add some stuff to root's .bashrc
cat << EOF >> ${MOUNT_POINT}/root/.bashrc
alias ls='ls -F --color=auto'
eval "\$(dircolors)"

case "\$TERM" in
xterm*|rxvt*)
	PS1="\\[\\e]0;\\u@\\h: \\w\a\\]\$PS1"
	;;
*)
	;;
esac
EOF

# Build the squashfs
mksquashfs ${MOUNT_POINT} /tmp/router.${ARCH}.squashfs \
	-comp xz

# Save the installed package list off
chroot ${MOUNT_POINT} dpkg --get-selections > /tmp/wip-installed-packages

class layout(CarthageLayout):
    add_provider(ConfigLayout)
    add_provider(carthage.ansible.ansible_log, str(_dir/"ansible.log"))
    class CarthageImage(PodmanImageModel, carthage_base.CarthageServerRole):
        base_image = injector_access('from_scratch_debian')
        oci_image_tag = 'localhost/carthage:latest'
        oci_image_command = ['/bin/systemd']
        base_image = ‘debian:bookworm’

        class customize_for_oci(FilesystemCustomization):

            @setup_task("Remove Software")
            async def remove_software(self):
                await self.run_command("apt", "-y", "purge",
                                       "exim4-base",
                                       )

            @setup_task("Install service")
            async def install_service(self):
               # installs and activates a systemd unit
sudo PYTHONPATH=$(pwd) python3 ./bin/carthage-runner ./oci_images build
sudo apt install apt-cacher-ng

Acquire::http::proxy "http://127.0.0.1:3142";
Acquire::https::proxy "DIRECT";

sudo apt install mmdebstrap

#!/bin/sh
mmdebstrap \
  --variant=buildd \
  --aptopt='Acquire::http::proxy "http://127.0.0.1:3142";' \
  --arch=amd64 \
  --components=main,contrib,non-free \
  unstable \
  ~/.cache/sbuild/unstable-amd64.tar.xz \
  http://deb.debian.org/debian

chmod +x ~/.local/bin/mmupdate
mkdir -p ~/.cache/sbuild
~/.local/bin/mmupdate

mkdir -p ~/.config/systemd/user

[Unit]
Description=Run mmupdate
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=%h/.local/bin/mmupdate

systemctl --user daemon-reload
systemctl --user start mmupdate
systemctl --user status mmupdate

[Unit]
Description=Run mmupdate daily

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

systemctl --user enable mmupdate.timer

sudo apt install --no-install-recommends sbuild autopkgtest

# backend for using mmdebstrap chroots
$chroot_mode = 'unshare';

# build in tmpfs
$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXX';

# upgrade before starting build
$apt_update = 1;
$apt_upgrade = 1;

# build everything including source for source-only uploads
$build_arch_all = 1;
$build_arch_any = 1;
$build_source = 1;
$source_only_changes = 1;

# go to shell on failure instead of exiting
$external_commands = { "build-failed-commands" => [ [ '%SBUILD_SHELL' ] ] };

# always clean build dir, even on failure
$purge_build_directory = "always";

# run lintian
$run_lintian = 1;
$lintian_opts = [ '-i', '-I', '-E', '--pedantic' ];

# do not run piuparts
$run_piuparts = 0;

# run autopkgtest
$run_autopkgtest = 1;
$autopkgtest_root_args = '';
$autopkgtest_opts = [ '--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a', '--prefix=/dev/shm/tmp.autopkgtest.' ];

# set uploader for correct signing
$uploader_name = 'Stephan Lachnit <[email protected]>';

export DEBFULLNAME="<your_name>"
export DEBEMAIL="<your_email>"
export DEB_BUILD_OPTIONS="parallel=<threads>"

sudo apt install devscripts dput-ng

DEBSIGN_KEYID=<your_gpg_fingerpring>
USCAN_SYMLINK=rename

debsign ../<pkgname_version_arch>.changes

debsign -S ../<pkgname_version_arch>_source.changes

dput ../<filename>.changes

rsync -v -n --files-from=<(ssh colette.anarc.at find Maildir -name '*colette*' ) colette.anarc.at: colette/
rsync -v -n --files-from=<(ssh colette.anarc.at find Maildir -name '*colette*' ) colette/ marcos.anarc.at:

Type       │ Partition      │ Suggested size
───────────┼────────────────┼───────────────
EFI        │ /dev/nvme0n1p1 │         512MiB
Linux swap │ /dev/nvme0n1p2 │           1GiB
Linux fs   │ /dev/nvme0n1p3 │      remainder

sudo apt install f2fs-tools

mkfs.f2fs -O extra_attr,inode_checksum,sb_checksum,compression,encrypt /dev/nvme0n1p3

mkdir boostrap
cd boostrap
mkdir root
export DFS=$(pwd)/root

sudo mount -o compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime /dev/nvme0n1p3 $DFS

sudo apt install debootstrap

debootstrap --arch=amd64 --components=main,contrib,non-free,non-free-firmware unstable $DFS http://deb.debian.org/debian

sudo mkdir -p $DFS/dev $DFS/dev/pts $DFS/proc $DFS/sys $DFS/run $DFS/sys/firmware/efi/efivars $DFS/boot/efi

sudo mount -v -B /dev $DFS/dev
sudo mount -v -B /dev/pts $DFS/dev/pts
sudo mount -v -B /proc $DFS/proc
sudo mount -v -B /sys $DFS/sys
sudo mount -v -B /run $DFS/run
sudo mount -v -B /sys/firmware/efi/efivars $DFS/sys/firmware/efi/efivars

sudo mount -v -B /dev/nvme0n1p1 $DFS/boot/efi

sudo chroot $DFS /bin/bash

export LC_ALL=C.UTF-8 LANG=C.UTF-8
apt install locales console-setup

dpkg-reconfigure locales

dpkg-reconfigure keyboard-configuration

dpkg-reconfigure tzdata

# file system                               mount point   type   options                                                            dump   pass

# NVME efi partition
UUID=XXXX-XXXX                              /boot/efi     vfat   umask=0077                                                         0      0

# NVME swap
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX   none          swap   sw                                                                 0      0

# NVME main partition
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX   /             f2fs   compress_algorithm=zstd:6,compress_chksum,atgc,gc_merge,lazytime   0      1

ls -lAph /dev/disk/by-uuid/

apt install systemd-boot efibootmgr

bootctl install --path=/boot/efi

efibootmgr -v

apt search linux-image-*

apt install linux-image-amd64

title      Debian GNU/Linux bookworm/sid
version    6.1.0-3-amd64
machine-id 2967cafb6420ce7a2b99030163e2ee6a
sort-key   debian
options    root=PARTUUID=f81d4fae-7dec-11d0-a765-00a0c91e6bf6 ro systemd.machine_id=2967cafb6420ce7a2b99030163e2ee6a
linux      /2967cafb6420ce7a2b99030163e2ee6a/6.1.0-3-amd64/linux
initrd     /2967cafb6420ce7a2b99030163e2ee6a/6.1.0-3-amd64/initrd.img-6.1.0-3-amd64

ls -lAph /dev/disk/by-partuuid/

efibootmgr --bootorder

useradd -m -G sudo -s /usr/bin/bash -c 'Full Name' username

tasksel

reboot

$ head flow-4.proto
syntax = "proto3";
package decoder;
option go_package = "akvorado/inlet/flow/decoder";

message FlowMessagev4 {

  uint64 TimeReceived = 2;
  uint32 SequenceNum = 3;
  uint64 SamplingRate = 4;
  uint32 FlowDirection = 5;
$ protoc -I=. --plugin=protoc-gen-go --go_out=module=akvorado:. flow-4.proto
$ head inlet/flow/decoder/flow-4.pb.go
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
//      protoc-gen-go v1.28.0
//      protoc        v3.21.12
// source: inlet/flow/data/schemas/flow-4.proto

package decoder

import (
        protoreflect "google.golang.org/protobuf/reflect/protoreflect"

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ initial.txt  │              final.txt              │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      12.963µ ± 2%   7.836µ ± 1%  -39.55% (p=0.000 n=10)
Sflow/with_encoding-12         19.37µ ± 1%   10.15µ ± 2%  -47.63% (p=0.000 n=10)

func BenchmarkDecodeEncodeSflow(b *testing.B) {
    r := reporter.NewMock(b)
    sdecoder := sflow.New(r)
    data := helpers.ReadPcapPayload(b,
        filepath.Join("decoder", "sflow", "testdata", "data-1140.pcap"))

    for _, withEncoding := range []bool{true, false} {
        title := map[bool]string{
            true:  "with encoding",
            false: "without encoding",
        }[withEncoding]
        var got []*decoder.FlowMessage
        b.Run(title, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                got = sdecoder.Decode(decoder.RawFlow{
                    Payload: data,
                    Source: net.ParseIP("127.0.0.1"),
                })
                if withEncoding {
                    for _, flow := range got {
                        buf := []byte{}
                        buf = protowire.AppendVarint(buf, uint64(proto.Size(flow)))
                        proto.MarshalOptions{}.MarshalAppend(buf, flow)
                    }
                }
            }
        })
    }
}

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ initial.txt │             bench-2.txt             │
                            │   sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      12.96µ ± 2%   10.28µ ± 2%  -20.67% (p=0.000 n=10)
Netflow/without_encoding-12   8.935µ ± 2%   8.975µ ± 2%        ~ (p=0.143 n=10)
Sflow/with_encoding-12        19.37µ ± 1%   16.67µ ± 2%  -13.93% (p=0.000 n=10)
Sflow/without_encoding-12     14.62µ ± 3%   14.87µ ± 1%   +1.66% (p=0.007 n=10)

type Column struct {
    Key       ColumnKey
    Name      string
    Disabled  bool

    // […]
    // For protobuf.
    ProtobufIndex    protowire.Number
    ProtobufType     protoreflect.Kind // Uint64Kind, Uint32Kind, …
    ProtobufEnum     map[int]string
    ProtobufEnumName string
    ProtobufRepeated bool
}

sch.ProtobufAppendVarint(bf, schema.ColumnBytes, uint64(recordData.Base.Length))
sch.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(recordData.Base.Protocol))
sch.ProtobufAppendVarint(bf, schema.ColumnSrcPort, uint64(recordData.Base.SrcPort))
sch.ProtobufAppendVarint(bf, schema.ColumnDstPort, uint64(recordData.Base.DstPort))
sch.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv4)

type FlowMessage struct {
    TimeReceived uint64
    SamplingRate uint32

    // For exporter classifier
    ExporterAddress netip.Addr

    // For interface classifier
    InIf  uint32
    OutIf uint32

    // For geolocation or BMP
    SrcAddr netip.Addr
    DstAddr netip.Addr
    NextHop netip.Addr

    // Core component may override them
    SrcAS     uint32
    DstAS     uint32
    GotASPath bool

    // protobuf is the protobuf representation for the information not contained above.
    protobuf      []byte
    protobufSet   bitset.BitSet
}

func (schema *Schema) ProtobufMarshal(bf *FlowMessage) []byte {
    schema.ProtobufAppendVarint(bf, ColumnTimeReceived, bf.TimeReceived)
    schema.ProtobufAppendVarint(bf, ColumnSamplingRate, uint64(bf.SamplingRate))
    schema.ProtobufAppendIP(bf, ColumnExporterAddress, bf.ExporterAddress)
    schema.ProtobufAppendVarint(bf, ColumnSrcAS, uint64(bf.SrcAS))
    schema.ProtobufAppendVarint(bf, ColumnDstAS, uint64(bf.DstAS))
    schema.ProtobufAppendIP(bf, ColumnSrcAddr, bf.SrcAddr)
    schema.ProtobufAppendIP(bf, ColumnDstAddr, bf.DstAddr)

    // Add length and move it as a prefix
    end := len(bf.protobuf)
    payloadLen := end - maxSizeVarint
    bf.protobuf = protowire.AppendVarint(bf.protobuf, uint64(payloadLen))
    sizeLen := len(bf.protobuf) - end
    result := bf.protobuf[maxSizeVarint-sizeLen : end]
    copy(result, bf.protobuf[end:end+sizeLen])

    return result
}

$ go test -run=__nothing__ -bench=Netflow/with_encoding \
>         -benchmem -cpuprofile profile.out \
>         akvorado/inlet/flow
goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
Netflow/with_encoding-12             143953              7955 ns/op            8256 B/op        134 allocs/op
PASS
ok      akvorado/inlet/flow     1.418s
$ go tool pprof profile.out
File: flow.test
Type: cpu
Time: Feb 4, 2023 at 8:12pm (CET)
Duration: 1.41s, Total samples = 2.08s (147.96%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ bench-2.txt  │             bench-3.txt             │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      10.284µ ± 2%   7.758µ ± 3%  -24.56% (p=0.000 n=10)
Netflow/without_encoding-12    8.975µ ± 2%   7.304µ ± 2%  -18.61% (p=0.000 n=10)
Sflow/with_encoding-12         16.67µ ± 2%   14.26µ ± 1%  -14.50% (p=0.000 n=10)
Sflow/without_encoding-12      14.87µ ± 1%   13.56µ ± 2%   -8.80% (p=0.000 n=10)

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ bench-3.txt  │             bench-4.txt             │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12       7.758µ ± 3%   7.365µ ± 2%   -5.07% (p=0.000 n=10)
Netflow/without_encoding-12    7.304µ ± 2%   6.931µ ± 3%   -5.11% (p=0.000 n=10)
Sflow/with_encoding-12        14.256µ ± 1%   9.834µ ± 2%  -31.02% (p=0.000 n=10)
Sflow/without_encoding-12     13.559µ ± 2%   9.353µ ± 2%  -31.02% (p=0.000 n=10)

if m.OutIfSpeed != 0 {
    i = encodeVarint(dAtA, i, uint64(m.OutIfSpeed))
    i--
    dAtA[i] = 0x6
    i--
    dAtA[i] = 0xd8
}

LAN_IFACE="ens18,ens21,ens22"
HEADER_FILE=/etc/radvd.conf.header

#!/bin/bash
#
# Copyright 2023 Collier Technologies LLC
# https://wp.c9h.org/cj/?p=1844
#

# # These variables are for the use of the scripts run by run-parts
# PPP_IFACE="$1"
# PPP_TTY="$2"
# PPP_SPEED="$3"
# PPP_LOCAL="$4"
# PPP_REMOTE="$5"
# PPP_IPPARAM="$6"

${PPP_IFACE:="ppp0"}
if [[ -z ${PPP_LOCAL} ]]; then
    PPP_LOCAL=$(curl -s https://icanhazip.com/)
fi

6rd_prefix="2602::/24"

#printf "%x%x:%x%x\n" $(echo $PPP_LOCAL | tr . ' ')

# This configuration option came from CenturyLink:
# https://www.centurylink.com/home/help/internet/modems-and-routers/advanced-setup/enable-ipv6.html
border_router=205.171.2.64

TUNIFACE=6rdif

first_ipv6_network=$(printf "2602:%02x:%02x%02x:%02x00::" $(echo ${PPP_LOCAL} | tr . ' '))

ip tunnel del ${TUNIFACE} 2>/dev/null
ip -l 5 -6 addr flush scope global dev ${IFACE}

ip tunnel add ${TUNIFACE} mode sit local ${PPP_LOCAL} remote ${border_router}
ip tunnel 6rd dev ${TUNIFACE} 6rd-prefix "${first_ipv6_network}0/64" 6rd-relay_prefix ${border_router}/32
ip link set up dev ${TUNIFACE}
ip -6 route add default dev ${TUNIFACE}

ip addr add ${first_ipv6_network}1/64 dev ${TUNIFACE}

rm /etc/radvd.conf
i=1
DEFAULT_FILE="/etc/default/centurylink-6rd"
if [[ -f ${DEFAULT_FILE} ]]; then
    source ${DEFAULT_FILE}
    if [[ -f ${HEADER_FILE} ]]; then
	cp ${HEADER_FILE} /etc/radvd.conf
    fi
else
    LAN_IFACE="setThis"
fi
   
for IFACE in $( echo ${LAN_IFACE} | tr , ' ' ) ; do

    ipv6_network=$(printf "2602:%02x:%02x%02x:%02x0${i}::" $(echo ${PPP_LOCAL} | tr . ' '))

    ip addr add ${ipv6_network}1/64 dev ${TUNIFACE}
    ip -6 route replace "${ipv6_network}/64" dev ${IFACE} metric 1

    echo "
interface ${IFACE} {
   AdvSendAdvert on;
   MinRtrAdvInterval 3;
   MaxRtrAdvInterval 10;
   AdvLinkMTU 1280;
   prefix ${ipv6_network}/64 {
     AdvOnLink on;
     AdvAutonomous on;
     AdvRouterAddr on;
     AdvValidLifetime 86400;
     AdvPreferredLifetime 86400;
   };
};
" >> /etc/radvd.conf

    let "i=i+1"
done

systemctl restart radvd

FROM jgoerzen/debian-base-security:bullseye
RUN echo "deb http://deb.debian.org/debian bullseye-backports main" >> /etc/apt/sources.list && \
    apt-get --allow-releaseinfo-change update && \
    apt-get -y --no-install-recommends -t bullseye-backports install yggdrasil
...
COPY yggdrasil.conf /etc/yggdrasil/
RUN set -x; \
    chown root:yggdrasil /etc/yggdrasil/yggdrasil.conf && \
    chmod 0750 /etc/yggdrasil/yggdrasil.conf && \
    systemctl enable yggdrasil

--cap-add=NET_ADMIN --sysctl net.ipv6.conf.all.disable_ipv6=0 --device=/dev/net/tun:/dev/net/tun

gpg --no-default-keyring \
    --keyring /usr/share/keyrings/debian-archive-bullseye-stable.gpg \
    --verify InRelease
awk '/installer-armhf\/current\/images\/SHA256SUMS/ {print $1 "
SHA256SUMS"}' InRelease | tail -n 1 | sha256sum -c 
grep hd-media SHA256SUMS \
| sed 's#hd-media/SD-card-images/##' \
| sha256sum -c \
| grep -v "No such file or directory" \
| grep -v "FAILED open or read" 2> /dev/null
grep netboot SHA256SUMS \
| sed 's#netboot/SD-card-images/##' \
| sha256sum -c \
| grep -v "No such file or directory" \
| grep -v "FAILED open or read" 2> /dev/null
$ update-alternatives --display shimx64.efi.signed
shimx64.efi.signed - auto mode
  link best version is /usr/lib/shim/shimx64.efi.signed.latest
  link currently points to /usr/lib/shim/shimx64.efi.signed.latest
  link shimx64.efi.signed is /usr/lib/shim/shimx64.efi.signed
/usr/lib/shim/shimx64.efi.signed.latest - priority 100
/usr/lib/shim/shimx64.efi.signed.previous - priority 50


IQ Format	hz.tools Name	Underlying Go Type
Interleaved uint8 (rtl-sdr)	`sdr.SamplesU8`	`[][2]uint8`
Interleaved int8 (hackrf, uhd)	`sdr.SamplesI8`	`[][2]int8`
Interleaved int16 (pluto, uhd)	`sdr.SamplesI16`	`[][2]int16`
Interleaved float32 (airspy, uhd)	`sdr.SamplesC64`	`[]complex64`

SDR	Format	RX/TX	State
rtl	u8	RX	Good
HackRF	i8	RX/TX	Good
PlutoSDR	i16	RX/TX	Good
rtl kerberos	u8	RX	Old
uhd	i16/c64/i8	RX/TX	Good
airspyhf	c64	RX	Exp

Import	What is it?
hz.tools/sdr	Core IQ types, supporting types and implementations that interact with the byte boundary
hz.tools/sdr/rtl	`sdr.Receiver` implementation using `librtlsdr`.
hz.tools/sdr/rtl/kerberos	Helpers to enable coherent RX using the Kerberos SDR.
hz.tools/sdr/rtl/e4k	Helpers to interact with the E4000 RTL-SDR dongle.
hz.tools/sdr/fft	Interfaces for performing an FFT, which are implemented by other packages.
hz.tools/sdr/rtltcp	`sdr.Receiver` implementation for rtl_tcp servers.
hz.tools/sdr/pluto	`sdr.Transceiver` implementation for the PlutoSDR using `libiio`.
hz.tools/sdr/uhd	`sdr.Transceiver` implementation for UHD radios, specifically the B210 and B200mini
hz.tools/sdr/hackrf	`sdr.Transceiver` implementation for the HackRF using `libhackrf`.
hz.tools/sdr/mock	Mock SDR for testing purposes.
hz.tools/sdr/airspyhf	`sdr.Receiver` implementation for the AirspyHF+ Discovery with `libairspyhf`.
hz.tools/sdr/internal/simd	SIMD helpers for IQ operations, written in Go ASM. This isn’t the best to learn from, and it contains pure go implemtnations alongside.
hz.tools/sdr/stream	Common Reader/Writer helpers that operate on IQ streams.

February 25, 2023

Background

Example running pyupgrade

The intermediate results

Debian Reunion Hamburg 2023 from May 23 to 30

February 24, 2023

Changes in ttdo version 0.0.9 (2023-02-21)

February 23, 2023

Intention behind hz.tools

Should you use this?

hz.tools/rf - Frequency types

hz.tools/sdr - SDR I/O and IQ Types

hz.tools/fftw - hz.tools/sdr/fft implementation

hz.tools/{fm,am} - analog audio demodulation and modulation

hz.tools/rfcap - byte serialization for IQ data

February 22, 2023

Changes in RcppArmadillo version 0.12.0.1.0 (2023-02-20)

Changes in RcppArmadillo version 0.11.4.4.0 (2023-02-09)

February 21, 2023

Debian LTS contributors

Evolution of the situation

Thanks to our sponsors

February 20, 2023

February 19, 2023

February 17, 2023

Bluetooth LE

Bluetooth LE from Python

Heart rate monitors

How about a proper fitness tracker?

Why script it from a desktop computer?

February 16, 2023

February 15, 2023

Highlights

Changelog

Bug fixes:

February 14, 2023

I love free software and I ❤️ OSUOSL

February 13, 2023

Language parser​

Integration in CodeMirror​

Syntax highlighting​

Linting​

Completion​

February 12, 2023

The Dell T320

The HP Z640

February 11, 2023

February 10, 2023

new year, new journal

one journal per import

tracking dependencies

opening balances

run lots of reports

addressing the virtual posting problem

catching errors after an import

going forward

Background

Current ideas

USB chargers

USB Docks

Your turn!

February 09, 2023

February 08, 2023

Barbie

Blitz

Dune, Part II

Ferrari

How Do You Live

Indiana Jones and the Dial of Destiny

Kafka

Language parser

Integration in CodeMirror

Syntax highlighting

Linting

Completion