pablotron.org

COVID-19 in the United States

Mar 28, 2020

I put together a tool to track COVID-19 in the US by state.

It is available here: COVID-19 US.

Weather Sage 0.1.2 Released

Aug 25, 2019

A couple days ago I released Weather Sage 0.1.2.

Weather Sage is a command-line tool and Ruby library to get the weather forecast and current weather observations for a domestic United States street address.

It uses the Census Bureau Geocoding API to geocode street addresses, and the National Weather Service Weather API to get weather forecasts and current weather observations.

The command-line tool queries the APIs above to obtain the requested information for the given street address, and then prints the results to standard output in CSV format.

Here’s an example of using the weather-sage command-line tool to get the weather forecast for the address 1600 pennsylvania ave nw, washington dc:

> weather-sage forecast '1600 pennsylvania ave nw washington dc'
address,name,temperature,temperature_unit,wind_speed,wind_direction,short_forecast
1600 pennsylvania ave washington dc,Today,97,F,7 mph,SW,Scattered Showers And Thunderstorms
1600 pennsylvania ave washington dc,Tonight,76,F,2 to 7 mph,SW,Scattered Showers And Thunderstorms then Mostly Cloudy
1600 pennsylvania ave washington dc,Tuesday,94,F,6 mph,E,Slight Chance Rain Showers then Chance Showers And Thunderstorms
1600 pennsylvania ave washington dc,Tuesday Night,75,F,2 to 6 mph,S,Chance Showers And Thunderstorms
1600 pennsylvania ave washington dc,Wednesday,94,F,3 to 9 mph,SW,Chance Showers And Thunderstorms
1600 pennsylvania ave washington dc,Wednesday Night,76,F,5 to 8 mph,SW,Chance Showers And Thunderstorms
...

And here’s an example of using the weather-sage command-line tool to get the current weather observations from the weather station closest to the address 1600 pennsylvania ave nw, washington dc:

> weather-sage now '1600 pennsylvania ave washington dc'
address,name,type,value,unit,quality_control
1600 pennsylvania ave washington dc,timestamp,time,2019-08-19T06:52:00+00:00
1600 pennsylvania ave washington dc,textDescription,text,Mostly Cloudy
1600 pennsylvania ave washington dc,temperature,value,26.700000000000045,unit:degC,qc:V
1600 pennsylvania ave washington dc,relativeHumidity,value,81.65039907186703,unit:percent,qc:C
...

You can install Weather Sage via RubyGems like this:

# install weather-sage with rubygems
> gem install weather-sage

Note: This is the initial release, so the API documentation, test suite, and command-line interface are not yet complete.

What to do with an STM32?

Aug 25, 2019

stm32f103c8t6

Last week my bag of generic STM32F103C8T6 “Blue Pill” boards arrived.

So far I’ve flashed Pill Duck and a couple of examples from Beginning STM32.

Now all I need are some project ideas.

All I’ve come up with so far are another Pocket Jim or another BME280-based temperature monitor.

Releases: Luigi Template 0.5.0, ZipStream-PHP 0.3.0, and Mathy

Aug 25, 2019

Some stuff I released months ago but never posted:

Luigi Template, version 0.5.0: Simple multi-language string templating library inspired by Unix pipes. Version 0.5.0 includes better documentation, faster rendering, implementations for Java, JavaScript, PHP, and Ruby.
Mathy: Web-based LaTeX math renderer.
ZipStream-PHP, version 0.3.0: PHP library that allows you to dynamically generate streamed Zip archives. Version 0.3.0 is a clean rewrite for PHP 7. It includes a test suite, Composer support, and API documentation.

Pi 4 Benchmarks

Aug 25, 2019

world-renowned pablotron test facility

A few weeks ago I picked up a 1GB Raspberry Pi 4 Model B and decided to run some openssl speed benchmarks.

Test systems:

Raspberry Pi Zero W
Raspberry Pi 3 Model B
Raspberry Pi 3 Model B+
Raspberry Pi 4 Model B
Linode VM (1 Core)
Intel i7-8650U (Lenovo X1 Carbon, 6th Gen)
AMD Ryzen Threadripper 1950X

Test algorithms:

blake2b512
blake2s256
sha256
sha512
aes-128-cbc
aes-192-cbc
aes-256-cbc

Results are available here. The code used to run the tests and generate the charts is here.

Self-Contained C11 SHA-2 Implementation

Aug 25, 2019

Last month I wrote a C11 SHA-2 implementation.

Features:

Self-contained (no external dependencies)
C11 only (platform-agnostic)
MIT-licensed
Includes test vectors (via make test).

The code is available here.

It includes implementations of the following:

SHA-224
SHA-256
SHA-384
SHA-512
HMAC-SHA256
HMAC-SHA512

This implementation is faster than coreutils, but slower than the assembly-optimized, processor family-specific OpenSSL SHA-2 implementation:

> time -p ./sha256 ~/Videos/8x*avi > /dev/null
rleal 9.39
user 9.10
sys 0.29
> time -p sha256sum ~/Videos/8x*avi > /dev/null
real 12.04
user 11.73
sys 0.31
> time -p openssl sha256 ~/Videos/8x*avi > /dev/null
real 6.36
user 6.01
sys 0.32

Current BEV Range

Jun 1, 2019

Range of Battery Electric Vehicles (BEVs) currently available in the US, according to data from fueleconomy.gov.

BEV Range

Links:

range-years-current.csv: CSV of current BEV range data.
Github repository: Has the CSV above and the steps I took to create it.
Data source: Zipped CSV on fueleconomy.gov (current as of 2019-05-25).

TensorFlow on a Raspberry Pi Zero is a Bad Idea

Apr 6, 2019

A recent question from the Something Awful Forums:

Can I run TensorFlow on a Raspberry Pi Zero?

The answer? You can, but it’s a bad idea.

The Raspberry Pi Zero is a single core ARMv6, with no NEON. Which means it’s slow:

System	Package	Install Time	Test Time
Raspberry Pi Zero W	Virtualenv/Pip	>1 hour	40 seconds
Raspberry Pi 3 Model B+	Virtualenv/Pip	10 minutes	10 seconds
AMD ThreadRipper 1950X (KVM VM, 8 cores)	Docker image	2 minutes	1.8 seconds

Details

Below are the steps I took to install TensorFlow on a Raspberry Pi Zero W. Note: you have to use Virtualenv to install TensorFlow in Raspbian. If you try to install TensorFlow directly with Pip, the installation will bomb out with an error.

Raspberry Pi Installation Steps:

# install system pip, numpy dependencies, and virtualenv
sudo apt-get install python3-pip python3-dev libatlas-base-dev virtualenv

# at this point i tried to install tensorflow directly via pip, which does NOT work
# sudo pip3 install --upgrade tensorflow

# created virtualenv environment instead
virtualenv --system-site-packages -p python3 ./venv

# activate virtual environment "venv"
# note: after this command your shell prompt will be prefixed with "(venv) "
source ./venv/bin/activate

# install tensorflow (i also installed keras here, because I use it for other stuff)
# note: this step takes a comically long time (>1 hour)
pip install tensorflow keras

Test Results (Raspberry Pi Zero W):

(venv) pabs@zero:~> time python -c "import tensorflow as tf; 
  tf.enable_eager_execution();
  print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
...
tf.Tensor(1533.9042, shape=(), dtype=float32)

real  0m40.802s
user  0m38.283s
sys  0m1.150s

Test Results (Raspberry Pi 3 Model B+):

(venv) pabs@peach:~> time python -c "import tensorflow as tf;
  tf.enable_eager_execution();
  print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
...
tf.Tensor(800.62, shape=(), dtype=float32)

real  0m9.408s
user  0m9.227s
sys  0m0.360s

Test Results (AMD ThreadRipper 1950X, 8 core KVM VM, Docker image):

pabs@hive:~> time docker run --rm -it tensorflow/tensorflow:latest-py3 python3 -c \
	"import tensorflow as tf; tf.enable_eager_execution();
  print(tf.reduce_sum(tf.random_normal([1000, 1000])))" 
...
tf.Tensor(-173.73222, shape=(), dtype=float32)

real  0m1.745s
user  0m0.032s
sys  0m0.016s

Make KVM, Docker, and TensorFlow Play Nice

Apr 6, 2019

Notes on getting KVM, Docker, and TensorFlow to cooperate.

By default, a KVM VM does not have the necessary CPU flags set to run the TensorFlow Docker image. In particular, the TensorFlow Docker image is compiled with support AVX.

The solution:

Use virsh capabilities on the host to get a list of host CPU capabilities, then
Use virsh edit to manually add the necessary CPU flags as <feature> tags under the <cpu> tag.

I elected to add all of the SIMD capabilities, including FP16.

For an AMD Threadripper 1950X, the resulting <cpu> tag looks like this:

<cpu mode='host-model'>
  <model fallback='allow'/>
  <feature policy='require' name='sse4.1'/>
  <feature policy='require' name='sse4.2'/>
  <feature policy='require' name='avx'/>
  <feature policy='require' name='f16c'/>
  <feature policy='require' name='avx2'/>
  <feature policy='require' name='ssse3'/>
</cpu>

Test run:

pabs@hive:~> time docker run --rm -it tensorflow/tensorflow:latest-py3 \
  python3 -c "import tensorflow as tf; tf.enable_eager_execution();
              print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
2019-04-06 12:25:16.576095: I tensorflow/core/platform/cpu_feature_guard.cc:141]
 Your CPU supports instructions that this TensorFlow binary was not compiled to
 use: AVX2 FMA
2019-04-06 12:25:16.627588: I tensorflow/core/platform/profile_utils/cpu_utils.c
c:94] CPU Frequency: 3393620000 Hz
2019-04-06 12:25:16.629909: I tensorflow/compiler/xla/service/service.cc:150] XL
A service 0x395bf00 executing computations on platform Host. Devices:
2019-04-06 12:25:16.629968: I tensorflow/compiler/xla/service/service.cc:158]
StreamExecutor device (0): <undefined>, <undefined>
tf.Tensor(-95.5094, shape=(), dtype=float32)

real	0m1.780s
user	0m0.024s
sys	0m0.012s

Password Strength Tester

Apr 4, 2019

This weekend I put together a simple password strength tester. You can try it out here.

Features:

Simple interface.
Open source (MIT licensed).
Easy to inspect. The interface code is contained in a single, <300 line JavaScript file.
Self-contained. No external connections for assets, social media sites, analytics, etc.
Can be run offline (no internet connection) or locally (by downloading an archive of the the Git repository).

Built with:

zxcvbn: JavaScript password strength estimator.
Bulma: Open source CSS framework.
font-awesome: Open source font icons.

Links: