Solar Panel Updates

October 1, 2022

Video of our solar panels:

Solar panels (video).

Solar panels (video).

The permit was approved on September 21st and our meter is in the net metering queue:

From: ...@dominionenergy.com
Sent: Friday, September 30, 2022 10:34 AM                                    
Subject: RE: Interconnection Certificate of Completion for Paul Duncan       
                                                                                
Reprogramed: A meter exchange schedule is being set for the week of 10/3
for this project. This exchange will be an over-the-air programmed
conversion and not physical.  If there are no issues with the
exchange, we will forward a PTO letter in the next 1 to 2 days.
Please let us know if you have any questions.

Kindest Regards

Net Metering
600 East Canal Street, 11^th Flr.
Richmond, VA 23219
Email: ...@dominionenergy.com

Enphase provides an optional, publicly-visible URL with solar array details and energy production. I set up a redirect here:

https://pmdn.org/solar

P.S. Here’s the high-tech method we used to create the video above:

Advanced Phone-Taped-on-Stick™ technology (patent pending).

Advanced Phone-Taped-on-Stick™ technology (patent pending).

jQueries

September 18, 2022

Some people dislike jQuery because it isn’t really needed for modern browsers (see You Might Not Need jQuery).

Many older sites embed multiple versions of jQuery, often as a transitive dependency.

I thought “wouldn’t it be funny if there was a page that loaded all versions of jQuery?”.

Behold:

jQueries: One page, every jQuery.

jQueries: One page, every jQuery.

Notes:

  • Embeds every version of jQuery from 1.2 to 3.6.1, including the slim builds.
  • Each version of jQuery animates it’s own row. See the unminified script.js for details.
  • You may have trouble hosting jQueries if you have a restrictive Content-Security-Policy (like mine). In particular, versions of jQuery older than 1.4.0 will fail to load.
  • Deliberately serves up old versions of jQuery with known security vulnerabilities. Be careful where you host this.

Source code, build instructions, and additional details are available in the GitHub repository.

Solar Panels

September 11, 2022

Yesterday the Solar Energy World team installed our solar panels. Here are a few pictures…

Unloading electrical equipment.

Unloading electrical equipment.

Almost as excited as me!

Almost as excited as me!

Van full of panels.

Van full of panels.

One of the panels up close.

One of the panels up close.

A couple of installed panels.  I need a ladder to show the rest.

A couple of installed panels. I need a ladder to show the rest.

New solar equipment near the meter.

New solar equipment near the meter.

Update (2022-10-01): Added a new post with the latest status and a video of the array.

OpenSSL Hash Benchmarks

June 10, 2022

I compiled OpenSSL 3.0.3 and then benchmarked the following cryptographic hash functions:

Note: MD5 and SHA-1 are included for reference, but both are broken and should not be used.

Test systems:

Detailed results, build scripts, and documentation are available in the companion GitHub repository.

Thinkpad X1 Carbon Results

Lenovo Thinkpad X1 Carbon (9th Gen) results.

Lenovo Thinkpad X1 Carbon (9th Gen) results.

Notes:

Raspberry Pi 4B Results

Raspberry Pi 4B results.

Raspberry Pi 4B results.

Notes:

  • BLAKE2b is clearly the fastest on the Cortex-A72.
  • BLAKE2b is faster than BLAKE2s because the former is optimized for 32-bit CPUs, and the latter is optimized for 64-bit CPUs.
  • SHA512 is faster than SHA256 because it has a slight edge on 64-bit systems.

Fastest JavaScript HTML Escape

March 9, 2022

What is the fastest JavaScript HTML escape implementation? To find out, I:

  1. Wrote 10 different JavaScript HTML escape implementations.
  2. Created a web-based benchmarking tool which uses web workers and the Performance API to test with a variety of string sizes and generates a downloadable CSV of results.
  3. Created a set of scripts to aggregate and plot the results as SVGs.

Results

The times are from 64-bit Chrome 99 running in Debian on a Lenovo Thinkpad X1 Carbon (9th Gen); the specific timing results may vary for your system, but the relative results should be comparable.

The first chart shows implementations' mean call time (95% CI) as the string length varies:

String Size vs. HTML Escape Function Call Time (μs)

String Size vs. HTML Escape Function Call Time (μs)

The second chart comparse implementations' mean call time (95% CI) for 3000 character strings:

HTML Escape Function Call Times

HTML Escape Function Call Times

The red, blue, and green bars in this chart indicate the slow, medium, and fast functions, respectively.

Slow Functions

Anything that uses a capturing regular expression.

Example: h2

const h2 = (() => {
  // characters to match
  const M = /([&<>'"])/g;

  // map of char to entity
  const E = {
    '&': '&amp;',
    '<': '&lt;',
    '>': '&gt;',
    "'": '&apos;',
    '"': '&quot;',
  };

  // build and return escape function
  return (v) => v.replace(M, (_, c) => E[c]);
})();

 

The capture is definitely at fault, because the call times for identical non-capturing implementations (example: h4) are comparable to everything else.

Medium Functions

Except for the capturing regular expression implementations in the previous section, the remaining implementations' call times were comparable with one another. This includes:

  • Reducing an array of string literals and calling replace().
  • Several variants of reducing an array of non-capturing regular expression with replace().

Example: h4

const h4 = (() => {
  // characters to match
  const M = /[&<>'"]/g;

  // map of char to entity
  const E = {
    '&': '&amp;',
    '<': '&lt;',
    '>': '&gt;',
    "'": '&apos;',
    '"': '&quot;',
  };

  // build and return escape function
  return (v) => v.replace(M, (c) => E[c]);
})();

Fast Functions

Three implementations are slightly faster than the others. They all use replaceAll() and match on string literals. Their call times are indistinguishable from one another:

  • h7: Reduce, Replace All
  • h8: Reduce, Replace All, Frozen
  • h9: Replace All Literal

Example: h7

const h7 = (() => {
  const E = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ["'", '&apos;'],
    ['"', '&quot;'],
  ];

  return (v) => E.reduce((r, e) => r.replaceAll(e[0], e[1]), v);
})();

 

The Winner: h9

Even though the call times for h7, h8, and h9 are indistinguishable, I actually prefer h9 because:

  • The most legible. It is the easiest implementation to read for beginning developers and developers who are uncomfortable with functional programming.
  • The simplist parse (probably).
  • Slightly easier for browsers to optimize (probably).

Here it is:

// html escape (replaceall explicit)
const h9 = (v) => {
  return v.replaceAll('&', '&amp;')
    .replaceAll('<', '&lt;')
    .replaceAll('>', '&gt;')
    .replaceAll("'", '&apos;')
    .replaceAll('"', '&quot;');
};

 

Notes

  • The benchmarking interface, aggregation and plotting scripts, and additional information are available in the companion GitHub repository.
  • I also wrote a DOM/textContent implementation, but I couldn’t compare it with the other implementations because web workers don’t have DOM access. I would be surprised if it was as fast as the fast functions above.
  • Object.freeze() doesn’t appear to help, at least not in Chrome.

Update (2022-03-11): I posted the benchmarking tool online at the following URL: https://pmdn.org/fastest-js-html-escape/.

Tiny Binaries Redux

March 2, 2022

I regenerated the Tiny Binaries chart with the following changes:

Results

All Static Binary Sizes

All Static Binary Sizes

Notes

See the tiny-binaries GitHub repository for additional details and a table of results.

Relaxed Content-Security-Policy for Go Code Coverage Reports

February 25, 2022

There is a conflict between my strict Content-Security-Policy and the CSS and JavaScript embedded in the HTML code coverage reports generated by go cover.

I tested a couple of methods of overriding the base Content-Security-Policy, without success:

  1. Add a relaxed <meta http-equiv='Content-Security-Policy' content='...'> element.
  2. Embed the script and style as data: URLs.

(Aside: I’m glad browsers don’t allow these workarounds, because they would be potential security holes).

In any case, the my solution was to relax the policy for a specific location via the Apache config:

#
# Relax style-src and script-src content security policies for content
# in the "/coverage-reports" directory so that the HTML coverage reports
# generated by `go cover` work as expected.
#
# Specifically the relaxed constraints allow:
#
# 1. The inline `<script>` element at the end of the generated HTML.
# 2. `style='display: none'` attributes on hidden elements.
#
# Notes:
#
# * You *have* to use `Header set` rather than `Header append` to
#   replace rather than append to the existing header.
# * Ideally we'd use `style-src-elem` and `script-src-elem`, but neither
#   are currently supported by Firefox.
#
<Location /coverage-reports>
  Header set "Content-Security-Policy" "default-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline'"
</Location>

 

The situation could be improved by making the following changes to the go cover HTML report source code:

  1. Replace style="display: none" attribute on elements with class=hide.
  2. Add .hide { display: none } to the inline <style> element.
  3. Hash the inner content of the inline <style> element.
  4. Hash the inner content of the inline <script> element.
  5. Add a <meta http-equiv ...> element to the header, using the hashes from the previous two steps.

Example:

<meta
  http-equiv="content-security-policy"
  content="default-src 'self'; script-src 'self' 'sha256-a43KCehRqYcFBGPJxgYD6a15e6CRFwVvwuDAe8rGbkM='; style-src 'self' 'sha256-8OTC92xYkW7CWPJGhRvqCR0U1CR6L8PhhpRGGxgW4Ts='"
/>

 

Bonus: These reports would automatically set a reasonable policy even in the absense of a more restrictive server policy.

Counter-argument to this post: These coverage reports are meant to be simple and only used locally. More elaborate or more secure coverage reports should be generated by a separate tool rather than complicating the default one.

Media Shrinkage

January 28, 2022

Recently I made the following site improvements:

  1. Minified SVGs with minify.
  2. Created lossless WebPs of PNGs in recent content.
  3. Created a Hugo shortcode for the <picture> element wrapped in a <figure>.
  4. Updated bitmap images in recent content to default to WebP with a fallback to PNG (progressive enhancement).
  5. Configured mod_deflate to compress SVGs (see note about BREACH below).
  6. Enabled HTTP/2 (how did miss I this before?).
  7. Added a Cache-Control header for static style, script, and image assets.

Note: Using HTTP compression (mod_deflate, mod_brotli, etc) with dynamic web pages can expose you to a BREACH attack. This site is statically generated (via Hugo) so BREACH is not an issue.

Results

68% cumulative reduction in the total size of an uncached front page load in Chrome:

DescriptionTotal Size (kB)Reduction (%)
Baseline uncached front page load596 kB0%
WebP and minified SVGs324 kB45%
WebP, minified SVGs, and mod_deflate update185 kB68%

Notes

Update (2022-01-29): Small formatting, grammar, and spelling fixes. Added HTTP/2 note, Cache-Control note, BREACH warning, and tdewolff/minify link.

Update (2022-08-18): Fixed typo in convert command.

Postgres Trigger Tests

January 23, 2022

We already know statement-level INSERT triggers are faster than row-level triggers in Postgres.

Out of curiosity I decided to time row-level and statement-level INSERT triggers with a variety of row counts.

Here are the results:

Row Count vs Query Time

Row Count vs Query Time

The takeaway here is that if speed is a concern, then you should prefer statement-level triggers to row-level triggers.

The scripts used to create the test database, run the tests, and generate the results are available in the companion GitHub repository.

Generics in Go 1.18

January 17, 2022

I spent some time trying the generics types and generic functions in Go 1.18, and I like what they’ve got so far.

Below is a Go 1.17 program which prints the results of the following functions:

  • SumInts(): calculate the the sum of of a slice of int values.
  • SumFloats(): calculate the the sum of of a slice of float32 values.
  • SumComplexes(): calculate the the sum of of a slice of complex64 values.
package main

import "fmt"

// Return sum of all values in slice of ints.
func SumInts(m []int) int {
  var r int

  for _, v := range(m) {
    r += v
  }

  return r
}

// Return sum of all values in slice of float32s.
func SumFloats(m []float32) float32 {
  var r float32

  for _, v := range(m) {
    r += v
  }

  return r
}

// Return sum of all values in slice of complex64s.
func SumComplexes(m []complex64) complex64 {
  var r complex64

  for _, v := range(m) {
    r += v
  }

  return r
}

var (
  // test integers
  ints = []int { 10, 20, 30 }

  // test floating point numbers
  floats = []float32 { 10.0, 20.0, 30.0 }

  // test complex numbers
  complexes = []complex64 { complex(10, 1), complex(20, 2), complex(30, 3) }
)

func main() {
  // print sums
  fmt.Printf("ints = %d\n", SumInts(ints))
  fmt.Printf("floats = %2.1f\n", SumFloats(floats))
  fmt.Printf("complexes = %g\n", SumComplexes(complexes))
}

 

Here’s the same program, written using Go 1.18 generics:

package main

import "fmt"

// Return sum of all numeric values in slice.
func Sum[V ~int|~float32|~complex64](vals []V) V {
  var r V

  for _, v := range(vals) {
    r += v
  }

  return r
}

var (
  // test integers
  ints = []int { 10, 20, 30 }

  // test floating point numbers
  floats = []float32 { 10.0, 20.0, 30.0 }

  // test complex numbers
  complexes = []complex64 { complex(10, 1), complex(20, 2), complex(30, 3) }
)

func main() {
  // print sums using generics w/explicit types
  fmt.Printf("ints = %d\n", Sum[int](ints))
  fmt.Printf("floats = %2.1f\n", Sum[float32](floats))
  fmt.Printf("complexes = %g\n", Sum[complex64](complexes))
}

 

You can use type inference to drop the type parameters in many instances. For example, we can rewrite main() from the previous example like this:

func main() {
  // print sums using generics w/explicit types
  fmt.Printf("ints = %d\n", Sum(ints))
  fmt.Printf("floats = %2.1f\n", Sum(floats))
  fmt.Printf("complexes = %g\n", Sum(complexes))
}

 

Generics can also be used in type definitions. Example:

package main

import "fmt"

// Fraction
type Frac[T ~int|~int32|~int64] struct {
  num T // numerator
  den T // denominator
}

// Add two fractions.
func (a Frac[T]) Add(b Frac[T]) Frac[T] {
  return Frac[T] { a.num + b.num, a.den * b.den }
}

// Multiple fractions.
func (a Frac[T]) Mul(b Frac[T]) Frac[T] {
  return Frac[T] { a.num * b.num, a.den * b.den }
}

// Return inverse of fraction.
func (a Frac[T]) Inverse() Frac[T] {
  return Frac[T] { a.den, a.num }
}

// Return string representation of fraction.
func (a Frac[T]) String() string {
  return fmt.Sprintf("%d/%d", a.num, a.den)
}

func main() {
  // test fractions
  fracs = []Frac[int] {
    Frac[int] { 1, 2 },
    Frac[int] { 3, 4 },
    Frac[int] { 5, 6 },
  }

  // print fractions
  for _, f := range(fracs) {
    fmt.Printf("%s => %s\n", f, f.Mul(f.Add(f.Inverse())))
  }
}

 

Interface type declarations can now be used to define the constraints that a matching type must satisfy. In addition to the ability to specify methods that a matching type must implement, a type constraint specified as an interface may also specify a union of terms indicating the set of matching types.

Type union terms can be tilde-prefixed (example: ~int), which indicates that the underlying type must match the given type.

For example, the Frac type declaration from the previous example could be written like this instead:

// Integral number type.
type integral interface {
  ~int | ~int32 | ~int64
}

// Fraction
type Frac[T integral] struct {
  num T // numerator
  den T // denominator
}

 

There are two new predeclared identifiers:

  • any: An alias for interface {}.
  • comparable: Any type which can be compared for equality with == and !=. Useful for the parameterizing map keys.

There is a new constraints package, which (not yet visible in the online Go documentation as of this writing) that provides a couple of useful unions, but it’s relatively anemic at the moment:

$ go1.18beta1 doc -all constraints
package constraints // import "constraints"

Package constraints defines a set of useful constraints to be used with type
parameters.

TYPES

type Complex interface {
	~complex64 | ~complex128
}
    Complex is a constraint that permits any complex numeric type. If future
    releases of Go add new predeclared complex numeric types, this constraint
    will be modified to include them.

type Float interface {
	~float32 | ~float64
}
    Float is a constraint that permits any floating-point type. If future
    releases of Go add new predeclared floating-point types, this constraint
    will be modified to include them.

type Integer interface {
	Signed | Unsigned
}
    Integer is a constraint that permits any integer type. If future releases of
    Go add new predeclared integer types, this constraint will be modified to
    include them.

type Ordered interface {
	Integer | Float | ~string
}
    Ordered is a constraint that permits any ordered type: any type that
    supports the operators < <= >= >. If future releases of Go add new ordered
    types, this constraint will be modified to include them.

type Signed interface {
	~int | ~int8 | ~int16 | ~int32 | ~int64
}
    Signed is a constraint that permits any signed integer type. If future
    releases of Go add new predeclared signed integer types, this constraint
    will be modified to include them.

type Unsigned interface {
	~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr
}
    Unsigned is a constraint that permits any unsigned integer type. If future
    releases of Go add new predeclared unsigned integer types, this constraint
    will be modified to include them.

 

Using the constraints package, the Frac type from the previous example could be written like this:

import "constraints"

// Fraction
type Frac[T constraints.Signed] struct {
  num T // numerator
  den T // denominator
}

 

And with constraints, the Sum() function from the first example could be defined like this:

// Numeric value.
type Number interface {
  constraints.Integer | constraints.Float | constraints.Complex
}

// Return sum of all numeric values in slice.
func Sum[V Number](vals []V) V {
  var r V

  for _, v := range(vals) {
    r += v
  }

  return r
}

 

Other useful tidbits:

Update (2021-01-19): Minor wording changes, add information about tilde prefixes in type constraints.

Update (2021-02-13): The constraints package was moved to x/exp for Go 1.18. LWN has an excellent summary of Go 1.18.

Update (2021-03-17): Go 1.18 released.

Archived Posts...
© 1998-2022 Paul Duncan