Bookworm Upgrades

I upgraded several more systems to Bookworm, including a Raspberry Pi 4 Model B 4GB and a Raspberry Pi Zero W.

All of the upgrades have gone smoothly except for the Pi Zero W, which failed with “Illegal instruction” errors during dist-upgrade and left the system unbootable in two separate upgrade attempts.

I don’t know if this is a pre-release issue with Bookworm or if this particular Pi Zero W is starting to fail. In either case, be sure to back up your SD card before trying to upgrade a Pi Zero Ws to Bookworm.

PNG Compressors

I tried the following PNG compressors on a set of several dozen screenshots:

• pngcrush: Lossless PNG compression command-line tool.
• optipng: Lossless multi-format image compression command-line tool.
• pngquant: Lossy PNG compression command-line tool and library.

pngquant produced the smallest output images. The results were also visually indistinguishable from the source images, at least for my use case. Of the two lossless encoders, optipng produced marginally smaller images than pngcrush.

(The bitmap images on this site are served as lossless WebPs with PNG as a fallback, courtesy of the <picture> element).

Domain Registrar

I’ve been considering switching to another domain registrar for a few years. The recent .zip TLD tizzy seemed like a perfect opportunity to register a new domain through a different registrar. I’d heard good things about Namecheap, so I created an account and registered my shiny new .zip domain: fdsa.zip.

Thoughts on Namecheap so far:

• Good prices.
• Fast and simple interface. Account setup and domain registration only took a few minutes.
• 2FA prominently located and easy to enable.
• They haven’t sold, lost, or spammed my email address (yet).
• DNS propagation only took 2 hours.

My next step will be to migrate several secondary domains from GoDaddy to Namecheap.

Last month our solar panels generated more energy than we consumed for the entire month. Here’s the Enphase dashboard for April:

Here is our latest power bill:

(Note: The date ranges in the two pictures overlap by 21 days but don’t match precisely, because our billing period is not quite monthly).

The highlighted sections of the bill above show the following:

• The total was $7.14
• We have an 18 kWh credit (e.g. net metering)

The kicker? Our car is electric; the bill includes the electricity for the house and the car.

I’ve spent the last couple days fiddling with Debian Bookworm RC2 in a VM. No issues to report. It’s shaping up to be a great release.

I’ve been looking for a suitable Docker replacement for a few years because of their repeated license shenanigans. Last year I tried switching to Podman, but ran into into several incompatibilities and minor annoyances.

Podman 4.3 ships with Bookworm and seems to fix all the issues I had before. Rootless containers, multi-stage builds, and all of my muscle-memory docker commands now work as expected. There is even a decent clone of docker-compose named (surprise!) podman-compose.

The only real differences I noticed are:

1. The command is podman instead of docker.
2. Image names must be registry-prefixed. Example: FROM docker.io/bash instead of FROM bash.
3. Searches must be registry-prefixed. Example: podman search docker.io/pablotron.

A couple of quick tests:

• Sinatra web application (rootless)
• Go web application (rootless, scratch, multi-stage build)

Update (2023-05-05): I put together a simple web application named Bookman to put podman-compose through it’s paces. It uses multiple containers, multi-stage builds, boot dependencies, secrets, and volumes.

Here’s a log of the setup process, and here’s a screenshot of the exposed web interface.

Update (2023-05-13): I upgraded several VMs from Bullseye (and one from Stretch!?!) to Bookworm, without any significant issues.

After upgrading to Bookworm, I migrated two VMs from Docker to Podman and installed Podman on a third VM. Useful tip: Rootless Podman does not agree with an NFS-mounted home directory.

One workaround is to create a local (that is, non-NFS), user-owned directory and then symlink ~/.local/share/containers to it, like so:

# create local containers directory for user pabs,
# then symlink ~pabs/.local/share/containers to it.
sudo mkdir -pm 700 /data/containers/pabs && \
  sudo chown pabs:pabs /data/containers/pabs && \
  ln -s /data/containers/pabs ~pabs/.local/share/containers

Alternatively, the Podman man page and Storage Table section of the storage.conf documentation suggest editing the graphroot in ~/.config/containers/storage.conf and pointing at a local directory.

Enphase dashboard for our first month of solar:

The blue bars represent energy produced by the solar panels. The orange bars represent energy consumed by us. The Y axis is kilowatt-hours (pay attention to the Y axis, because the dashboard rescales it).

According to the numbers above, the solar panels generated 78% (1 - 117.4/538.6 = ~0.78) of our consumed energy for the month. Virginia has net metering, so our power bill should be 21% of what it would be otherwise.

On clear, sunny days the panels generate more energy than we consume:

And in a shocking twist that will surprise no one, on cloudy days the solar panels don’t do very well:

A couple weeks ago I released census-geocoder, a Go wrapper for the Census Geocoding Services API.

Example

Here’s an example application which geocodes the command-line argument and then prints the normalized address from the geocoding result of each address to standard output:

package main

import (
  "fmt"
  "log"
  "os"
  "pablotron.org/census-geocoder/geocoder"
)

func main() {
  for _, arg := range(os.Args[1:]) {
    // get address matches
    matches, err := geocoder.Locations(arg)
    if err != nil {
      log.Fatal(err)
    }

    // print matches
    for _, match := range(matches) {
      fmt.Println(match)
    }
  }
}

Links

• Git repository
• Documentation

Last month I got a reel of Raspberry Pi Pico Ws. Several will eventually be paired with BME280s and used to measure temperature, humidity, and barometric pressure throughout the house.

That project will have to wait a few weeks until I can unpack my soldering iron.

In the mean time I’ve been fiddling MicroPython. My thoughts so far are below…

Ordering and Delivery

Ordering was a bit of a headache. The sites that I normally buy from all had one or more of the following annoyances:

• Ridiculous markup
• Outrageous shipping fees
• Comically low quantity limit

The Pico and Pico W are not supply chain constrained like the Raspberry Pi 4B and Raspberry Pi Zero 2 W, so I’m not sure why retailers are doing this.

I finally checked DigiKey, and they had plenty of Pico Ws in stock and none of the aforementioned shenanigans. I ordered a reel of 20 Pico Ws for $6.00 apiece on October 3rd. The package arrived on October 5th.

As of this writing, DigiKey still has over 15,000 Pico Ws in stock.

Hardware

Thoughts:

• 264 kB of RAM feels quite roomy on a microcontroller.
• Built-in wifi is incredibly useful because you don’t have to sacrifice a bunch of pins for external communication.
• PIO looks promising, although I haven’t really used it yet.
• The onboard temperature sensor is not very accurate.

Flashing MicroPython

Flashing via USB mass storage is great. I didn’t have to install an SDK, fight with finicky wiring, or use an obscure microcontroller flashing tool.

To get up and running with MicroPython, you can follow these instructions. The basic steps in Linux are as follows:

1. Press the BOOTSEL button on the Pico W.
2. Connect the Pico W to your computer via USB. It appears as USB mass storage device.
3. Mount the mass storage device. Many distributions will automatically mount the device for you as /media/RPI-RP2.
4. Copy a UF2 image to the mounted disk.
5. The Pico W will flash itself and reboot.

If you’ve flashed MicroPython, then at this point you can connect to a REPL via USB serial (e.g. screen /dev/ttyACM0).

The Pico and Pico W have SWD pins, so you can still flash via SWD if you want to; instructions are available in the documentation.

MicroPython

MicroPython is fantastic. I was able to blink the onboard LED, read the onboard temperature sensor, connect to wifi, and make web requests less than an hour after unpacking the Pico W.

Example: Blink LED

MicroPython v1.19.1 on 2022-10-06; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> led = machine.Pin('LED', machine.Pin.OUT) # get LED
>>> led.toggle() # enable LED
>>> led.value() # check status
1
>>> led.toggle() # disable LED
>>> led.value() # check status
0
>>>

Example: Connect to Wifi and send an HTTP request

MicroPython v1.19.1 on 2022-10-06; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> import network
>>> wifi = network.WLAN()
>>> wifi.active(True) # enable wifi
>>> wifi.connect('ssid', 'password') # replace 'ssid' and 'password' with your SSID and password
>>> wifi.status() # wait until wifi is connected (e.g., status = 3)
3
>>> import urequests
>>> data = urequests.get('https://pablotron.org/').content # send http request, get response body
>>> len(data) # print number of bytes received
48693

One useful tip: machine.unique_id() returns a unique ID for the device. This is useful because it allows you to flash the same image to several Pico Ws and then use machine.unique_id() to uniquely identify them. Example:

>>> import binascii
>>> binascii.hexlify(machine.unique_id())
b'e6614c311b867937'

Miscellaneous MicroPython notes:

• Many of the libraries in the MicroPython standard library are reasonable subsets of their counterparts from the Python standard library.
• MicroPython does not have Pip or PyPI. Instead you install packages via mip. I used it mip install and use the MQTT library.
• You can build a Unix version of MicroPython by following the instructions in the ports/unix directory of the MicroPython Git repository.
• You can exhaust the RAM fairly easily by trying to fetch web content which is larger than the available RAM. Not that I would make such a silly mistake…

Pictures

Description of the last picture:

• Foreground: Pico W connected via USB to a Raspberry Pi 4B (4GB), sitting in a delivery container. The Pi 4B is used for miscellaneous aarch64 testing like the hash benchmarks from June.
• Background: Raspberry Pi 3B in a proper case with a BME280 soldered to it to the I²C pins. The Pi 3B is the bedroom HTPC (running Kodi) and temperature monitor.

Last week we received permission to operate from Dominion, and this afternoon we got the final go-ahead from Solar Energy World.

Here’s a screenshot of the Enphase energy dashboard for the day:

Update (2022-11-11): New post with array performance for the first month.

Video of our solar panels:

The permit was approved on September 21st and our meter is in the net metering queue:

From: ...@dominionenergy.com
Sent: Friday, September 30, 2022 10:34 AM                                    
Subject: RE: Interconnection Certificate of Completion for Paul Duncan       
                                                                                
Reprogramed: A meter exchange schedule is being set for the week of 10/3
for this project. This exchange will be an over-the-air programmed
conversion and not physical.  If there are no issues with the
exchange, we will forward a PTO letter in the next 1 to 2 days.
Please let us know if you have any questions.

Kindest Regards

Net Metering
600 East Canal Street, 11^th Flr.
Richmond, VA 23219
Email: ...@dominionenergy.com

Enphase provides an optional, publicly-visible URL with solar array details and energy production. I set up a redirect here:

https://pmdn.org/solar

P.S. Here’s the high-tech method we used to create the video above:

Some people dislike jQuery because it isn’t really needed for modern browsers (see You Might Not Need jQuery).

Many older sites embed multiple versions of jQuery, often as a transitive dependency.

I thought “wouldn’t it be funny if there was a page that loaded all versions of jQuery?”.

Behold:

Notes:

• Embeds every version of jQuery from 1.2 to 3.6.1, including the slim builds.
• Each version of jQuery animates it’s own row. See the unminified script.js for details.
• You may have trouble hosting jQueries if you have a restrictive Content-Security-Policy (like mine). In particular, versions of jQuery older than 1.4.0 will fail to load.
• Deliberately serves up old versions of jQuery with known security vulnerabilities. Be careful where you host this.

Source code, build instructions, and additional details are available in the GitHub repository.

Yesterday the Solar Energy World team installed our solar panels. Here are a few pictures…

Update (2022-10-01): Added a new post with the latest status and a video of the array.

Update (2022-11-11): New post with array performance for the first month.