Solar: One Month

November 11, 2022

Enphase dashboard for our first month of solar:

Total energy production and consumption.

Total energy production and consumption.

The blue bars represent energy produced by the solar panels. The orange bars represent energy consumed by us. The Y axis is kilowatt-hours (pay attention to the Y axis, because the dashboard rescales it).

According to the numbers above, the solar panels generated 78% (1 - 117.4/538.6 = ~0.78) of our consumed energy for the month. Virginia has net metering, so our power bill should be 21% of what it would be otherwise.

On clear, sunny days the panels generate more energy than we consume:

Energy production and consumption on a sunny day.

Energy production and consumption on a sunny day.

And in a shocking twist that will surprise no one, on cloudy days the solar panels don’t do very well:

Energy production and consumption on a cloudy day.

Energy production and consumption on a cloudy day.

Census Geocoder Released

November 11, 2022

A couple weeks ago I released census-geocoder, a Go wrapper for the Census Geocoding Services API.

Example

Here’s an example application which geocodes the command-line argument and then prints the normalized address from the geocoding result of each address to standard output:

package main

import (
  "fmt"
  "log"
  "os"
  "pablotron.org/census-geocoder/geocoder"
)

func main() {
  for _, arg := range(os.Args[1:]) {
    // get address matches
    matches, err := geocoder.Locations(arg)
    if err != nil {
      log.Fatal(err)
    }

    // print matches
    for _, match := range(matches) {
      fmt.Println(match)
    }
  }
}

 

Pico Ws

November 11, 2022

Last month I got a reel of Raspberry Pi Pico Ws. Several will eventually be paired with BME280s and used to measure temperature, humidity, and barometric pressure throughout the house.

That project will have to wait a few weeks until I can unpack my soldering iron.

In the mean time I’ve been fiddling MicroPython. My thoughts so far are below…

Ordering and Delivery

Ordering was a bit of a headache. The sites that I normally buy from all had one or more of the following annoyances:

  • Ridiculous markup
  • Outrageous shipping fees
  • Comically low quantity limit

The Pico and Pico W are not supply chain constrained like the Raspberry Pi 4B and Raspberry Pi Zero 2 W, so I’m not sure why retailers are doing this.

I finally checked DigiKey, and they had plenty of Pico Ws in stock and none of the aforementioned shenanigans. I ordered a reel of 20 Pico Ws for $6.00 apiece on October 3rd. The package arrived on October 5th.

As of this writing, DigiKey still has over 15,000 Pico Ws in stock.

Hardware

Thoughts:

  • 264 kB of RAM feels quite roomy on a microcontroller.
  • Built-in wifi is incredibly useful because you don’t have to sacrifice a bunch of pins for external communication.
  • PIO looks promising, although I haven’t really used it yet.
  • The onboard temperature sensor is not very accurate.

Flashing MicroPython

Flashing via USB mass storage is great. I didn’t have to install an SDK, fight with finicky wiring, or use an obscure microcontroller flashing tool.

To get up and running with MicroPython, you can follow these instructions. The basic steps in Linux are as follows:

  1. Press the BOOTSEL button on the Pico W.
  2. Connect the Pico W to your computer via USB. It appears as USB mass storage device.
  3. Mount the mass storage device. Many distributions will automatically mount the device for you as /media/RPI-RP2.
  4. Copy a UF2 image to the mounted disk.
  5. The Pico W will flash itself and reboot.

If you’ve flashed MicroPython, then at this point you can connect to a REPL via USB serial (e.g. screen /dev/ttyACM0).

The Pico and Pico W have SWD pins, so you can still flash via SWD if you want to; instructions are available in the documentation.

MicroPython

MicroPython is fantastic. I was able to blink the onboard LED, read the onboard temperature sensor, connect to wifi, and make web requests less than an hour after unpacking the Pico W.

MicroPython v1.19.1 on 2022-10-06; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> led = machine.Pin('LED', machine.Pin.OUT) # get LED
>>> led.toggle() # enable LED
>>> led.value() # check status
1
>>> led.toggle() # disable LED
>>> led.value() # check status
0
>>>

 

Example: Connect to Wifi and send an HTTP request

MicroPython v1.19.1 on 2022-10-06; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> import network
>>> wifi = network.WLAN()
>>> wifi.active(True) # enable wifi
>>> wifi.connect('ssid', 'password') # replace 'ssid' and 'password' with your SSID and password
>>> wifi.status() # wait until wifi is connected (e.g., status = 3)
3
>>> import urequests
>>> data = urequests.get('https://pablotron.org/').content # send http request, get response body
>>> len(data) # print number of bytes received
48693

 

One useful tip: machine.unique_id() returns a unique ID for the device. This is useful because it allows you to flash the same image to several Pico Ws and then use machine.unique_id() to uniquely identify them. Example:

>>> import binascii
>>> binascii.hexlify(machine.unique_id())
b'e6614c311b867937'

 

Miscellaneous MicroPython notes:

Pictures

Reel of Raspberry Pi Pico Ws.

Reel of Raspberry Pi Pico Ws.

Individual Pico W.

Individual Pico W.

Advanced Pico W testing facility.

Advanced Pico W testing facility.

Description of the last picture:

Solar Panels Are Live

October 10, 2022

Last week we received permission to operate from Dominion, and this afternoon we got the final go-ahead from Solar Energy World.

Here’s a screenshot of the Enphase energy dashboard for the day:

First day of solar.

First day of solar.

Update (2022-11-11): New post with array performance for the first month.

Solar Panel Updates

October 1, 2022

Video of our solar panels:

Solar panels (video).

Solar panels (video).

The permit was approved on September 21st and our meter is in the net metering queue:

From: ...@dominionenergy.com
Sent: Friday, September 30, 2022 10:34 AM                                    
Subject: RE: Interconnection Certificate of Completion for Paul Duncan       
                                                                                
Reprogramed: A meter exchange schedule is being set for the week of 10/3
for this project. This exchange will be an over-the-air programmed
conversion and not physical.  If there are no issues with the
exchange, we will forward a PTO letter in the next 1 to 2 days.
Please let us know if you have any questions.

Kindest Regards

Net Metering
600 East Canal Street, 11^th Flr.
Richmond, VA 23219
Email: ...@dominionenergy.com

Enphase provides an optional, publicly-visible URL with solar array details and energy production. I set up a redirect here:

https://pmdn.org/solar

P.S. Here’s the high-tech method we used to create the video above:

Advanced Phone-Taped-on-Stick™ technology (patent pending).

Advanced Phone-Taped-on-Stick™ technology (patent pending).

jQueries

September 18, 2022

Some people dislike jQuery because it isn’t really needed for modern browsers (see You Might Not Need jQuery).

Many older sites embed multiple versions of jQuery, often as a transitive dependency.

I thought “wouldn’t it be funny if there was a page that loaded all versions of jQuery?”.

Behold:

jQueries: One page, every jQuery.

jQueries: One page, every jQuery.

Notes:

  • Embeds every version of jQuery from 1.2 to 3.6.1, including the slim builds.
  • Each version of jQuery animates it’s own row. See the unminified script.js for details.
  • You may have trouble hosting jQueries if you have a restrictive Content-Security-Policy (like mine). In particular, versions of jQuery older than 1.4.0 will fail to load.
  • Deliberately serves up old versions of jQuery with known security vulnerabilities. Be careful where you host this.

Source code, build instructions, and additional details are available in the GitHub repository.

Solar Panels

September 11, 2022

Yesterday the Solar Energy World team installed our solar panels. Here are a few pictures…

Unloading electrical equipment.

Unloading electrical equipment.

Almost as excited as me!

Almost as excited as me!

Van full of panels.

Van full of panels.

One of the panels up close.

One of the panels up close.

A couple of installed panels.  I need a ladder to show the rest.

A couple of installed panels. I need a ladder to show the rest.

New solar equipment near the meter.

New solar equipment near the meter.

Update (2022-10-01): Added a new post with the latest status and a video of the array.

Update (2022-11-11): New post with array performance for the first month.

OpenSSL Hash Benchmarks

June 10, 2022

I compiled OpenSSL 3.0.3 and then benchmarked the following cryptographic hash functions:

Note: MD5 and SHA-1 are included for reference, but both are broken and should not be used.

Test systems:

Detailed results, build scripts, and documentation are available in the companion GitHub repository.

Thinkpad X1 Carbon Results

Lenovo Thinkpad X1 Carbon (9th Gen) results.

Lenovo Thinkpad X1 Carbon (9th Gen) results.

Notes:

Raspberry Pi 4B Results

Raspberry Pi 4B results.

Raspberry Pi 4B results.

Notes:

  • BLAKE2b is clearly the fastest on the Cortex-A72.
  • BLAKE2b is faster than BLAKE2s because the former is optimized for 32-bit CPUs, and the latter is optimized for 64-bit CPUs.
  • SHA512 is faster than SHA256 because it has a slight edge on 64-bit systems.

Fastest JavaScript HTML Escape

March 9, 2022

What is the fastest JavaScript HTML escape implementation? To find out, I:

  1. Wrote 10 different JavaScript HTML escape implementations.
  2. Created a web-based benchmarking tool which uses web workers and the Performance API to test with a variety of string sizes and generates a downloadable CSV of results.
  3. Created a set of scripts to aggregate and plot the results as SVGs.

Results

The times are from 64-bit Chrome 99 running in Debian on a Lenovo Thinkpad X1 Carbon (9th Gen); the specific timing results may vary for your system, but the relative results should be comparable.

The first chart shows implementations' mean call time (95% CI) as the string length varies:

String Size vs. HTML Escape Function Call Time (μs)

String Size vs. HTML Escape Function Call Time (μs)

The second chart comparse implementations' mean call time (95% CI) for 3000 character strings:

HTML Escape Function Call Times

HTML Escape Function Call Times

The red, blue, and green bars in this chart indicate the slow, medium, and fast functions, respectively.

Slow Functions

Anything that uses a capturing regular expression.

Example: h2

const h2 = (() => {
  // characters to match
  const M = /([&<>'"])/g;

  // map of char to entity
  const E = {
    '&': '&amp;',
    '<': '&lt;',
    '>': '&gt;',
    "'": '&apos;',
    '"': '&quot;',
  };

  // build and return escape function
  return (v) => v.replace(M, (_, c) => E[c]);
})();

 

The capture is definitely at fault, because the call times for identical non-capturing implementations (example: h4) are comparable to everything else.

Medium Functions

Except for the capturing regular expression implementations in the previous section, the remaining implementations' call times were comparable with one another. This includes:

  • Reducing an array of string literals and calling replace().
  • Several variants of reducing an array of non-capturing regular expression with replace().

Example: h4

const h4 = (() => {
  // characters to match
  const M = /[&<>'"]/g;

  // map of char to entity
  const E = {
    '&': '&amp;',
    '<': '&lt;',
    '>': '&gt;',
    "'": '&apos;',
    '"': '&quot;',
  };

  // build and return escape function
  return (v) => v.replace(M, (c) => E[c]);
})();

Fast Functions

Three implementations are slightly faster than the others. They all use replaceAll() and match on string literals. Their call times are indistinguishable from one another:

  • h7: Reduce, Replace All
  • h8: Reduce, Replace All, Frozen
  • h9: Replace All Literal

Example: h7

const h7 = (() => {
  const E = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ["'", '&apos;'],
    ['"', '&quot;'],
  ];

  return (v) => E.reduce((r, e) => r.replaceAll(e[0], e[1]), v);
})();

 

The Winner: h9

Even though the call times for h7, h8, and h9 are indistinguishable, I actually prefer h9 because:

  • The most legible. It is the easiest implementation to read for beginning developers and developers who are uncomfortable with functional programming.
  • The simplist parse (probably).
  • Slightly easier for browsers to optimize (probably).

Here it is:

// html escape (replaceall explicit)
const h9 = (v) => {
  return v.replaceAll('&', '&amp;')
    .replaceAll('<', '&lt;')
    .replaceAll('>', '&gt;')
    .replaceAll("'", '&apos;')
    .replaceAll('"', '&quot;');
};

 

Notes

  • The benchmarking interface, aggregation and plotting scripts, and additional information are available in the companion GitHub repository.
  • I also wrote a DOM/textContent implementation, but I couldn’t compare it with the other implementations because web workers don’t have DOM access. I would be surprised if it was as fast as the fast functions above.
  • Object.freeze() doesn’t appear to help, at least not in Chrome.

Update (2022-03-11): I posted the benchmarking tool online at the following URL: https://pmdn.org/fastest-js-html-escape/.

Tiny Binaries Redux

March 2, 2022

I regenerated the Tiny Binaries chart with the following changes:

Results

All Static Binary Sizes

All Static Binary Sizes

Notes

See the tiny-binaries GitHub repository for additional details and a table of results.

Archived Posts...
© 1998-2022 Paul Duncan