I wrote a blog post about painting my fence. This got me thinking a lot about colors and how I use them as a web developer.
Nathan Broadbent
So I asked myself a question: What's the perceptual difference between two adjacent HTML colors?
There are 16,777,216 possible colors (256 x 256 x 256), since there are only 256 possible values for red, green, and blue. I've been using this Delta-E calculator to calculate the perceptual difference between colors:
I chose a random green/blue color (rgb(50,200,180) and calculated some differences after adding/subtracting one from each color channel. Here are the results:
- 0.1192 = R:50,G:200,B:180 -> R:51,G:200,B:180
- 0.1168 = R:50,G:200,B:180 -> R:49,G:200,B:180
- 0.6929 = R:50,G:200,B:180 -> R:50,G:201,B:180
- 0.6943 = R:50,G:200,B:180 -> R:50,G:199,B:180
- 0.5714 = R:50,G:200,B:180 -> R:50,G:200,B:181
- 0.5711 = R:50,G:200,B:180 -> R:50,G:200,B:179
Subtracting one from the green channel seems to show the biggest perceptual difference.
Here's the difference when you subtract one from each color for pure white (rgb(255, 255, 255):
- 0.3561 = R:255,G:255,B:255 -> R:254,G:255,B:255
- 0.667 = R:255,G:255,B:255 -> R:255,G:254,B:255
- 0.5079 = R:255,G:255,B:255 -> R:255,G:255,B:254
So the green channel shows the biggest difference for both addition and subtraction. The next largest difference is in the blue channel, and red makes a relatively small difference.
Here's the progression from rgb(255, 255, 255) down to rgb(255, 232, 255)
You can see that it's very hard to tell the difference between any two colors that are right next to each other.
I thought of another question: What are the biggest and smallest perceptual differences between two adjacent HTML colors?
I decided to write a Ruby script to figure this out. I found this color_difference Ruby gem that uses the CIEDE2000 Color-Difference formula.
mmozurasPaper: https://hajim.rochester.edu/ece/sites/gsharma/ciede2000/ciede2000noteCRNA.pdf
require 'color_difference'
max_difference = nil
min_difference = nil
max_difference_colors = []
min_difference_colors = []
0.step(255, 1) do |r|
0.step(255, 1) do |g|
0.step(255, 1) do |b|
color = { r: r, g: g, b: b }
puts "Color: #{color.inspect}"
[1, -1].each do |diff|
%i(r g b).each do |channel|
new_color = color.dup
new_color[channel] += diff
new_color[channel] = [0, new_color[channel]].max
new_color[channel] = [255, new_color[channel]].min
next if new_color == color
difference = ColorDifference.cie2000(color, new_color)
if max_difference.nil? || difference > max_difference
max_difference = difference
max_difference_colors = [color, new_color]
end
if min_difference.nil? || difference < min_difference
min_difference = difference
min_difference_colors = [color, new_color]
end
end
end
end
end
end
def format_float(number)
sprintf('%.15f', number).sub(/0+$/, '').sub(/\.$/, '.0')
end
puts "Max difference: #{format_float max_difference}"
puts "Colors: #{max_difference_colors.inspect}"
puts "------------------------------------"
puts "Min difference: #{format_float min_difference}"
puts "Colors: #{min_difference_colors.inspect}"Results:
Max difference: 0.012461248722627
Colors: [{:r=>24, :g=>23, :b=>23}, {:r=>24, :g=>24, :b=>23}]
------------------------------------
Min difference: 0.000064328231603
Colors: [{:r=>0, :g=>255, :b=>255}, {:r=>1, :g=>255, :b=>255}]I've included the colors below. The large blocks are the two adjacent colors with the biggest/smallest differences. The smaller blocks are what it looks like if I continue adding one to their respective color channels (green for the largest difference, and red for the smallest difference.)
Note: It's easier to see the largest difference against a black background.
Largest perceptual difference:
Smallest perceptual difference:
You can clearly see the top blocks becoming more green, while the bottom blocks all look like the same color. (I had to double-check my HTML code to be sure I hadn't made any mistakes!)
I can't even really tell the difference between rgb(0,255,255) and rgb(30,255,255)!
I have to go all the way up to rgb(120,255,255) before I can really notice the difference.
(I'm not color blind! I did a test!)
So I think we now have enough information to answer the question: Do we need more HTML colors?
No, 256 possible values for each color channel is plenty. It's virtually impossible for a human to tell the difference between any two adjacent color codes.
]]>We're renting an old house that needs a lot of maintenance. The landlords are very easy-going
]]>We're renting an old house that needs a lot of maintenance. The landlords are very easy-going so we're allowed to have a dog and two cats. They also said we can pretty much do whatever we want to the house, so we've installed lots of home automation stuff, put up a garden shed, and hammered nails into the walls to put up pictures. We also take of the lawns and the garden and help to keep the place tidy.
"Tidy up the fence" had been on my todo list for a long time. There's a few marks and rusty nails, and it looks like someone had tried to paint a... smiley face?
I think they forgot the mouth on the smiley face.
I thought it would be nice to touch up the fence with a bit of paint. I guess the proper way to do it is to pressure wash the fence, sand it down, apply a coat of primer, then a coat of paint. I didn't want to do all of that since the fence is old and probably due to be replaced. I just wanted to paint over a few spots, so I needed to find a matching paint color.
I tried taking a small bit of painted wood to the hardware store. They have a computerized paint machine that can scan a color and mix some paint to match the color. Unfortunately this didn't work too well and the resulting paint was too dark. They said that my painted wood chip had a bit too much dirt and grime and it wasn't easy to clean it, so they couldn't get an accurate color.
I went over to the Resene color swatches and held up my paint flake. "Double Thorndon Cream" seemed like it was a good match, so I bought a small test pot. I also brought home a collection of 28 Resene color cards for white and neutral colors, just in case I needed a different color. (They were free!)
I held up the color swatches and double-checked that "Double Thorndon Cream" matched the fence. It seemed pretty close, so I tried it out.
Wait, what? Why is it so light? Maybe there's something wrong with the paint in the test pot. Even if it's a bit brighter when it's wet, I can't see how it will be the right color when it dries.
I took a picture and used the color picker tool in GIMP. I wasn't able to choose a single pixel since the wood has a lot of texture, so I checked the "Sample average" option and took the average color from a 50x50 square.
I also checked the color of the original paint:
I did some Googling and found an online "Delta-E calculator":
This tool can calculate the "perceptual distance" between two colors by using the Cie76 algorithm.
The difference between the original paint and the paint from my test pot is: 8.9811. Apparently that's quite a big difference. Humans can't tell the difference between colors if the difference is less than 1. That might be too hard to find, so I decided to aim for a difference of less than 2.
But how did I get it so wrong? Maybe I didn't mix the paint in the test pot well enough, or maybe it was a lot lighter than the swatch. So I checked the colors in GIMP.
- Swatch =
rgb(201, 203, 199) - Paint =
rgb(198, 203, 201)
The Delta-E difference between these two colors is only 1.5174. They're actually very similar colors.
So the paint does match the swatch. Maybe I shouldn't be relying on my eyes so much.
Contributors to Wikimedia projects
Anyway, now I know that the color swatch cards can be trusted. If the test paint matches it's corresponding swatch, then maybe I just need to find a swatch that matches the original paint. But I'm not going to rely on my eyes this time. I'm going to use the color picker and look at the Delta-E differences.
I took photos of all 28 swatches against the fence and went through all the colors one-by-one.
Resene Double Parchment™️cc - Y80-025-083
I found a color called "Double Parchment" that perfectly matched the paint. It was even the exact RGB value I was looking for: rgb(185, 186, 179). There was something very satisfying about finding a perfect match. (Although the color of the fence varies slightly, depending on where I measured it.)
I went back to the store to buy another test pot. Unfortunately they didn't have any small pots available, so I had to buy a slightly bigger one. No problem, I was pretty sure that I got it right this time.
Well that doesn't seem quite right.
You can see that I was getting a bit frustrated so I just went ahead and painted over all the graffiti anyway. But still want to figure this out, because I want to touch up some other parts of the fence and the house. If I manage to figure out the right color then maybe I'll paint over this section again.
New strategy:
I'll compare the new paint with the old paint and calculate percentage difference for each color channel. Then I'll apply that same percentage difference to the color swatch and look for a swatch that matches the darker color.
Color difference:
- Red: 202.6 / 221.6 = 0.9142599278
- Green: 195.7 / 215.4 = 0.908542247
- Blue: 186.2 / 204.0 = 0.912745098
So I need to find a paint color that's about 91% as bright as Double Parchment.
I finally stumbled onto the Resene website, and realized that I had been doing everything wrong up until this point. Using the GIMP color picker for my swatch photos wasn't a very reliable process.
This looks a lot more promising! Now I can rely on some official RGB values for Resene Double Parchment. I just need to multiply them by around 91%, and hopefully this next color will be the right one.
- Red: 199 * 0.9142599278 = 182
- Green: 187 * 0.908542247 = 170
- Blue: 163 * 0.912745098 = 149
They even have a search feature, so you can look for paint colors matching some specific RGB values:
I calculated the Delta-E differences for each color to see which one was the closest. Here are the results:
- Bison Hide: 2.0151 = R:182,G:170,B:149 -> R:181,G:172,B:148
- Half Malta: 2.6672 = R:182,G:170,B:149 -> R:181,G:166,B:149
- Half Grey Olive: 2.4561 = R:182,G:170,B:149 -> R:177,G:171,B:149
- Joss: 2.5412 = R:182,G:170,B:149 -> R:185,G:168,B:150
- Double Tea: 1.312 = R:182,G:170,B:149 -> R:178,G:168,B:148
- Mandrake: 2.1529 = R:182,G:170,B:149 -> R:176,G:169,B:148
- Eagle: 3.5444 = R:182,G:170,B:149 -> R:176,G:172,B:148
- Drought: 2.843 = R:182,G:170,B:149 -> R:184,G:168,B:143
- Quarter Gargoyle: 3.795 = R:182,G:170,B:149 -> R:178,G:164,B:149
- Half Rickshaw: 4.1016 = R:182,G:170,B:149 -> R:189,G:167,B:147
- Quarter Pravda: 4.5576 = R:182,G:170,B:149 -> R:180,G:171,B:158
- Quarter Stonewashed: 5.0726 = R:182,G:170,B:149 -> R:178,G:169,B:157
"Double Tea" was the closest match with a difference of only 1.312.
So maybe I'll go back to the hardware store tomorrow and buy another test pot.
The Next Day:
I went outside to look at the fence again. The paint had dried. Turns out that "Double Parchment" was pretty close after all.
There's still some room for improvement, but I don't think it will be noticeable after I paint the rest of the boards in this section. The posts spaced every ~2 metres will help to hide the color difference, especially when the sun is hitting them and casting shadows.
Lessons Learned
- Don't trust your eyes.
- It's very easy to get fooled by optical illusions. Your eyes can be tricked by all sorts of things, including shadows, patterns, textures, perspective, background color, color temperature, etc. Don't use your eyes to compare two colors. Use a color picker tool and check the Delta-E difference.
- Be patient.
- Wait for paint to dry before you make any decisions. Sometimes paint gets lighter after it dries. Sometimes it gets darker. Maybe some paint even stays the same color.
- Don't be lazy.
- I should have washed the fence first. The color of the old paint will probably change if I ever clean it. (Maybe I'll get lucky and it will match the new paint.)
Things I might try next time
If I ever need to color match some paint in the future, I might try using one of these ColorChecker charts:
Contributors to Wikimedia projects
Maybe I can hold this card against the wall and take a photo. Then I can use the chart to calibrate the photo and get a reliable RGB color for the wall. I could also hold up a ColorChecker chart against some Resene color swatch cards, and then calibrate them against the official RGB values from the Resene website.
Update:
I pressure washed the fence and painted a few more spots. I tried taking a timelapse video, but the sun is too bright and you can't really see any difference. Oh well.
It's close, but not really!
I used the color picker on this photo:
- New paint:
rgb(193.8, 195.8, 181.8) - Old paint:
rgb(181.4, 183.4, 171.4)
Differences:
- Red: 181.4 / 193.8 = 0.9360165119
- Green: 183.4 / 195.8 = 0.9366700715
- Blue: 171.4 / 181.8 = 0.9427942794
I took the official RGB values for Resene Double Parchment from the website, and multipled the values by this ratio to find a slightly darker color.
- Red: 199 * 0.9360165119 = 186.2672858681 (186)
- Green: 187 * 0.9366700715 = 175.1573033705 (175)
- Blue: 163 * 0.9427942794 = 153.6754675422 (154)
Then I searched for this new color: rgb(186, 175, 154)
- Akaroa: 1.9844 = R:186,G:175,B:154 -> R:190,G:178,B:154
- Half Hillary: 3.8147 = R:186,G:175,B:154 -> R:185,G:179,B:152
- Triple Parchment: 2.5177 = R:186,G:175,B:154 -> R:189,G:175,B:150
- Half Nomad: 3.4854 = R:186,G:175,B:154 -> R:183,G:175,B:160
- Silk: 4.9492 = R:186,G:175,B:154 -> R:187,G:173,B:161
- Half Napa: 3.747 = R:186,G:175,B:154 -> R:180,G:173,B:158
- Joss: 3.4876 = R:186,G:175,B:154 -> R:185,G:168,B:150
- Serene: 4.4225 = R:186,G:175,B:154 -> R:189,G:176,B:163
- Antidote: 3.2984 = R:186,G:175,B:154 -> R:193,G:176,B:159
- Coral: 4.857 = R:186,G:175,B:154 -> R:192,G:176,B:147
- Eighth Stonewall: 4.1965 = R:186,G:175,B:154 -> R:178,G:173,B:158
- Quarter Pravda: 4.6182 = R:186,G:175,B:154 -> R:180,G:171,B:158
- Bison Hide: 2.251 = R:186,G:175,B:154 -> R:181,G:172,B:148
Akaroa looked promising. I picked up a test pot and tried covering up a few more rusty nails.
The color was perfect! I couldn't see any difference.
... as long as I didn't look at it from the side.
I forgotten to consider the "sheen", or high shiny the paint is.
Resene paints come in a few different gloss levels:
- Gloss
- Semi-gloss
- Satin
- Low sheen
- Flat (or matt)
The test pot I tried was "made using Resene Lumbersider low sheen exterior and interior paint." It turns out that I needed a flat / matt paint with zero sheen.
I did some more research and saw that Resene has a "Lumbersider Matt" paint. I couldn't find any matt test pots for sale online, and it looks like Lumbersider Matt might not even be available in New Zealand. But I will go back to the Resene store on Monday and ask them about it.
Otherwise, I might need to start over with a different paint brand. Maybe Dulux or Wattyl.
]]>My wife is a counselor who does "play therapy" at a primary school. She has a sand tray and some toys that kids can play with during their sessions. Sometimes she asks me to 3D print things for the kids to play with. For example, I printed this little shovel for the sand tray:
One kid really likes trains, so I was asked to 3D print a toy steam train:
Another kid really likes pretending to make fried rice in the sand, so my wife asked if I could 3D print some plastic ingredients and a toy spatula.
I found a few 3D models:
freepik 6.10m assets
The wok model was tricky. I used this tool to convert the OBJ to STL:
The resulting STL had some errors, so I used this tool to fix them:
Formware B.V.I also added some basic shapes in Bambu Studio (spheres, cylinders, and cubes) to peas, bacon, omelette, and green onions.
Some of these small parts were quite difficult to print. I had to experiement with the support and raft settings, especially for the peas and corn. You can see that I still lost a few.
Here's the finished product:
I had a lot of fun making this. I hope they have fun playing with it!
]]>I bought it from the creator's online store:
PricelessToolkit
The design is also open source and available on GitHub:
I bought this awesome device that sends me notifications whenever we get a new letter or package in our mailbox.
I bought it from the creator's online store:
PricelessToolkitThe design is also open source and available on GitHub:
PricelessToolkitThey included STLs for a 3D printed case. I also designed a custom mailbox flap with embedded magnets. The flap opens whenever a letter is pushed into the mailbox. This triggers a reed switch that turns on the Mailbox Guard and sends a notification to the receiver.
Sensor case and custom mailbox flap
I bought this LILYGO TTGO LoRa32 board to receive messages and send them to Home Assistant via MQTT.
I 3D printed a case for the receiver: TTGO Lora32 T3 Slimline Case on Printables
It turns out that most of our mailbox notifications are flyers and leaflets, and sometimes the wind. I will probably turn off the real-time notifications and just show the status on a dashboard somewhere.
I've noticed that LoRa is more reliable than some of my Zigbee devices, and has a much longer range. I might try designing some of my own LoRa devices in the future.
Future Ideas
I would like to add a second reed switch on the main door of the mailbox, so that I can automatically reset the count to zero whenever we take the letters out of the mailbox. But I don't know if this is possible due to the design of the board.
I would also like to add some LoRa transmitters to our cars, so that I can automatically open and close our garage doors. I've already set this up with the Home Assistant mobile app for home presence detection, but sometimes there's a delay, or my wife is driving my car (or vice versa.)
]]>
I'm extremely happy with this 3D printer and highly recommend it! It's more expensive than entry-level printers, but it's a very high quality machine and is very reliable. I can focus on designing
]]>I bought a Bambu Lab X1-Carbon 3D printer in March 2023.
I'm extremely happy with this 3D printer and highly recommend it! It's more expensive than entry-level printers, but it's a very high quality machine and is very reliable. I can focus on designing and making things instead of spending lots of time tinkering with my 3D printer.
I find a lot of models on Thingiverse, Printables, and MakerWorld. I also design my own parts in OpenSCAD and Fusion360.
This page is where I log some of the things that I've printed.
2024
Aeria Mechanica Mini
Leviathan Steam Train
Fried Rice Sandbox Toy
Nathan Broadbent
See the project blog post for more details
Replacement Generator Cap
My cousin works for a company that supplies generators for events. They lost one of these generator caps and asked me if I could print a new one, since it's quite expensive to buy a replacement. I copied the design in OpenSCAD and printed it using ASA filament for strength and UV resistance.
This took a few attempts because there was significant warping with ASA. You can see the build plate jump up a little bit as the plastic cools and shrinks, but it was able to finish printing without any issues.
Dune Ornithopter Model
The Moon
This print took 33 hours.
Replacement Sunvisor Clip for Audi
Lighter Fluid Tracks
Raspberry Pi 4B Cases
This my new favorite case design. It looks really cool and snaps together nicely.
Other Stuff
josteb
2023
Desk Cable Organization
My first big 3D printing project was to organize all the cables under my desk. I found some of these models on Thingiverse, and I also designed a few of my own things in OpenSCAD.
Mailbox Guard and Lora Receiver Case
Nathan Broadbent
See the blog post for more details
Michaelangelo's David Bust
I printed and spray-painted this David bust to display on a shelf. I paused the print half way and filled it with pebbles to give it some weight.
Figurines for Airbrushing and Painting Practice
I printed this duck, Crash Bandicoot, and Pikachu to get some practice with my airbrush. My printer can do multi-color prints using the AMS, but it takes a long time, wastes a lot of plastic, and doesn't always look very good. Another good option is to print all the parts separately and glue them together.
Crash Bandicoot
3D Tic-Tac-Toe
JetSpool
Tri-Hex Holder For Pens, Paintbrushes, etc.
Replacement Rack Ears for Ubiquiti Switch
I bought a used Ubiquiti Switch that didn't come with any rack ears. I designed and printed these little brackets so that I could mount it in my server rack. The brackets prevent it from slipping out of the rack, but they are not strong enough to support the switch. It needs to be placed on top of another rack-mounted device for support.
Battery Organizers
I printed lots of organizers for coin cell batteries, AA, AAA, 9V, etc.
Orleans Board Game Organizer
Curved Honeycomb Vase
I printed this to try out "Spiral vase" mode, where it prints in one continous line (instead of printing one layer at a time.)
Filament Spool Switcher & Winder
This machine can be used to wind filament onto a new spool.
Bambu Lab P1 / X1 / X1C / X1CC Filament Spool Switcher & Winder on Printables
Turntable for 3D Scanning
I ended up using this for airbrushing.
Tablet Wall Mount
LED Strip Channel Holders for Vegepod
I designed and printed these clips to hold an LED strip grow light.
Pendant Lamps
I printed 8 of these for lights around our house. They look really nice.
小厮工作室
Glass Sphere Lamp Adapters
I printed these adapters so that an existing light socket would fit into a new spherical glass lampshade.
Angled Holder for Nintendo Switch Dock
This dock holder makes it a easier to take out my Nintendo Switch from the TV cabinet.
Bolt Thumb Screws for Drill Press Vise
I use these to quickly reposition or remove a vise from my drill press.
Alien Egg Chair
This was designed for a chicken egg, but I scaled it up to fit an ostrich egg. Now my wife can display her ostrich egg on a shelf in her office.
Cat Key Hook
Vinyl Record Wall Brackets
Fluxx Card Game Box
Replacement Pool Vaccum Adapter
My cousin asked me to fix this thing for his pool vacuum.
Vaccuum Hose Adapters
I designed some adapters for my shop vac so that I could extend the hose. I also printed adapters for my circular saw and cyclonic separator.
Rubik's Cube Tissue Box
PLA was a bad choice. The tissue box started warping and the color pieces fell off when my wife parked her car in the sun. I might try printing this again using ASA filament (UV resistant.) I might also print the color pieces slightly smaller and super glue them (instead of hammering them in.)
Fairy Door & Window
Raspberry Pi 4 Cases
I really like this Raspberry Pi case design.
NodeMCU ESP-32S Case
I've printed lots of these cases for ESP-32s around my house running ESPresense.
QuinLED Dig Uno Case
QuinLED Dig Uno v3.1 on Printables
Car Water Bottle Holder
I designed this water bottle holder for my car in OpenSCAD.
Benchy
The "Hello World" of 3D printing!
Other Stuff
James Tai
Vision2Design
MasterCake
I haven't been able to write any blog posts since I got an M2 MacBook. I was using an old version of Hugo and got it running in a Docker container, but the Docker container only ran on my old Intel Mac. It was too painful to update my hugo theme to support the latest version, and I couldn't figure out how to get it to run on my new Mac. So I didn't write any blog posts for a while.
I was using Ghost for my company blog and I really loved the editor. It's so much nicer than editing Markdown files in VS Code, especially when it comes to embedding screenshot images and videos. I loved being able to take a screenshot and just drag it into my blog post, instead of messing around with image resizing, file paths, Markdown image syntax, etc. So I decided to set up a Ghost instance on my home Proxmox server.
I found this really helpful post on the Ghost Forum that helped me build an archive of all my old posts and import them into the Ghost database:
diegoeis
See my reply for the JS script that ended up (mostly) working for me. I still had to spend a few hours fixing minor issues, getting syntax highlighting to work, and converting some image slideshows to Ghost galleries.
You can run Ghost in production and use it to serve a blog dynamically, but I didn't want to open up my home server to the public internet. I also didn't want to give up all the benefits of a static site:
- Extremely fast and 100% cached
- No need to worry about traffic spikes or DDoS attacks
- No need to worry about security updates
Some downsides:
- You can't use Ghost's subscription and commenting features
- It takes a little bit more time and effort to publish a new blog post
I've turned off Ghost's subscription features and I'm using Disqus for comments.
I found a tool that can generate a static site from a Ghost blog:
SimonM0I wrote a little script that will fetch all the HTML and assets, make a few changes and add a few things, commit the changes, and push to my GitHub pages repo.
#!/bin/bash
set -e
if ! [ -d "static" ]; then
git clone [email protected]:ndbroadbent/ndbroadbent.github.io.git static
git checkout gh-pages
fi
rm -rf static/*
node src/index.js --domain https://ghost.ndbroadbent.com --productionDomain https://madebynathan.com
# Replace all remaining instances of https://ghost.ndbroadbent.com with https://madebynathan.com in static files
grep -rl "https://ghost.ndbroadbent.com" static | xargs sed -i '' 's/https:\/\/ghost.ndbroadbent.com/https:\/\/madebynathan.com/g'
echo madebynathan.com > static/CNAME
cp pubkey_38E63C0A.txt static/
cp -R sudoblock static/
cd static || exit
git add .
git commit -m "Update static files"
git push origin gh-pages
I always like to use Cloudflare, even though GitHub pages already has a CDN:
Cloudflare provides a little more flexibility, so I can set up custom redirect rules, force SSL, etc. They also have a nice analytics dashboard.
I stopped using Google Analytics a while ago. Now I use Plausible.io for analytics on all of my websites. I use their hosted cloud platform for my blogs and personal sites, and I self-host my company instance on Render.
This blog post is my first real test run on my self-hosted Ghost instance, and I've really enjoyed it! It's such a game changer to have a WYSIWYG editor where I can drag-and-drop images. I love using Ghost as a headless CMS and generating a static site on GitHub pages + Cloudflare. I think it's the best of both worlds.
I'm excited to write some more blog posts about 3D printing, home automation, and all the other stuff I've been up to lately.
]]>You can use this PDF to make your own ring binder:
]]>The template code and generated PDF document are released
I used a ring binder and card sleeves to organize 1,700 through-hole resistors.
You can use this PDF to make your own ring binder:
The template code and generated PDF document are released under the MIT license. You can find a copy of the MIT license at the bottom of this page.
What Are Resistors?
Resistors are a fundamental component in electronics. They are used to limit the current flowing through a circuit. Resistors have colored bands that represent their value. The first two bands represent the first two digits of the resistance, and the third band represents the "multiplier" value, or power of 10. For example, a 10KΩ resistor has a brown band, a black band, and an orange band. The first two bands are brown (1) and black (0), which is 10. The third band is orange (1,000Ω), so the resistance is 10 x 1,000 = 10,000Ω.
The "tolerance" value is the fourth band. This is the maximum deviation from the nominal resistance. For example, a 10KΩ resistor with a 5% tolerance can have a resistance between 9,500Ω and 10,500Ω.
The E24 Series
The E24 series is a logarithmic series of 24 values for each power of 10: 1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1.
You can multiply each value in the series by the multiplier (power of 10) to get the 24 resistance values for each order of magnitude. For example, these are the first ten resistor values with a brown multiplier band (10Ω): 100Ω, 110Ω, 120Ω, 130Ω, 150Ω, 160Ω, 180Ω, 200Ω, 220Ω, 240Ω.
How I Organized 1,700 Resistors
I bought a resistor pack that included 1,700 resistors in the E24 series, in sets of 10 (170 different values.) I bought 1/4 W carbon film resistors with a 5% tolerance.
It's very useful to have a wide range of resistors for breadboard circuits and prototype PCBs. It's not so useful when they're all mixed together in a single bag. The resistors I bought didn't have any labels or markings on the paper strips, so I had to read the colored bands or use a multimeter to find the resistance.
For my first attempt at organizing them, I sorted them by the multiplier band (0.1Ω, 1Ω, 10Ω, etc.), and put them in labeled drawers:
This didn't really help at all. I still needed to sort through 24 different values to find the one I was looking for. It got even worse once I started pulling out individual resistors to use in breadboard circuits. Instead of 24 sets of 10, I could have up to 240 individual resistors to sort through.
Ring Binder Design
I had some spare card sleeves, and I realized that I could use these to create a 3x3 grid of pockets on an A4 ring binder sheet.
I used DocSpring to create the layout for all the pages. DocSpring is an API for filling out and generating PDFs. You can upload an existing PDF form, or create your own PDF templates using HTML and CSS. DocSpring's HTML/CSS templates support the Liquid template language, so I wrote some Liquid code to calculate the resistor values and generate all the pages for the E24 series.
(Disclaimer: I'm the founder of DocSpring.)
This template didn't need any fields, so I didn't need to use the DocSpring API. I just downloaded the generated PDF from the Preview tab.
I included a resistor graphic that shows the colored bands for each value. This was invaluable when I was sorting resistors into sleeves.
I used Sleeve Kings 63.5mm X 88mm Card Sleeves for the 3x3 grid of pockets.
I cut them to make them a little shorter, and stuck them onto ring binder sheets using a glue stick. Then I sorted all the resistors into sleeves, which felt like doing a jigsaw puzzle. It was a fun activity for a rainy Sunday afternoon.
I'm pretty happy with how this turned out, and I think my version is even better than the expensive ring binders I found for sale. The PDF and code are MIT licensed, so anyone is more than welcome to use this for personal or commercial purposes.
- Github Gist with Liquid and SCSS code
- Try a DocSpring demo with this template
- Sign up for a free DocSpring trial account to customize the template
Gallery: Finished Ring Binder
This ring binder system makes it much easier to see when I need to order more resistors. (I've got some more 330, 360, 1K, and 1.1K ohm resistors on the way!)
How To Organize Your Own Resistors
Option 1: Buy Card Binder Sheets
After publishing this blog post, I found out that you can buy card binder sheets that already come with 9 pockets.
You can purchase these on Amazon.
I found another option that has some flaps at the top of each pocket.
This would prevent the resistors from falling out if you drop the binder or tip it upside down. I'll probably use this option next time I need to organize electronic components. It would also be very handy for SD cards and USB sticks.
Supplies
- Color printer
- Ring binder
- 9 Sleeve Card Binder Sheet, or 9 Pocket Binder Sheets With Flaps
- Scissors or paper cutter
Instructions
- Print the PDF
- Cut out the cards on each page
- Insert the cards into the pockets
- Insert resistors in front of the cards in each pocket
Option 2: Make DIY Card Binder Sheets
Supplies
- Color printer
- Ring binder
- Plain ring binder sheets
- Card sleeves (e.g. Sleeve Kings 63.5mm X 88mm Card Sleeves or similar)
- Glue stick
Instructions
- Print the PDF
- Insert pages into ring binder sheets
- Use glue stick to glue the card sleeves to the outside of the ring binder sheets
- Insert resistors into card sleeves
Option 3: Raaco storage drawers
@impulse7 posted a comment on Hacker News where they shared their own solution to this problem. They created PDF labels to organize resistors in Raaco storage drawers. They have shared their PDF generation code on GitHub. This is a great option if you have plenty of space in your workshop.
Option 4: Buy A Finished Ring Binder With Sorted Resistors
If you want to skip all the work and just buy a pre-made ring binder, you can buy one from an electronics supplier such as RS Components. This will save a lot of time, but they can be quite expensive.
$504.52 New Zealand dollars = $323.72 USD
I found a few cheaper options but they were still hundreds of dollars. So I thought it would be fun to make my own ring binder.
P.S. Digi-Key's Resistor Color Code Calculator was very useful. It's a great tool for quickly looking up resistor values.
]]>Plex-Cleaner is a Python script that can clean up your Plex directory by deleting movies and TV shows after you've watched them. This will stop your disk from filling up with lots of old media files over time. You can run this script on your server (or Raspberry]]>Plex-Cleaner is a Python script that can clean up your Plex directory by deleting movies and TV shows after you've watched them. This will stop your disk from filling up with lots of old media files over time. You can run this script on your server (or Raspberry Pi, or wherever you run Plex Media Server.)
I like to put things in the /opt directory on my servers.
-
Run
git clone https://github.com/ndbroadbent/Plex-Cleaner.git /opt -
Run
python3 PlexCleaner.py --dump Cleaner.confto create a new configuration file -
Configure an authentication token in
Cleaner.conf, under the"Token"key. -
Call
python3 PlexCleaner.pyto test it out.
The script will run in test mode and won't delete anything if your Cleaner.conf contains "test": true, or if you call python3 PlexCleaner.py --test.
- After everything looks good in the logs, set
"test": falseinCleaner.conf.
You'll probably want to run this script regularly as a cron job. Run crontab -e to edit your crontab, then add the following line to run the script every day at 4am:
0 4 * * * /usr/bin/python3 /opt/Plex-Cleaner/PlexCleaner.py
I always like to set up monitoring for any scheduled scripts so that I get a notification if when they stop working. healthchecks.io is my favorite monitoring tool and they have a generous free tier. (No affiliation, they're just an awesome service.)
To set up monitoring with healthchecks.io, you can call the run_plex_cleaner.sh script from my fork. This script will ping healthchecks.io on success or failure. healthchecks.io will also send you a notification if it doesn't receive any ping at the expected time (e.g. if your server has been turned off.)
- Sign up for a free healthchecks.io account.
- Create a new check. Choose the "Cron" schedule and enter your cron expression, e.g.
0 4 * * *. - Make sure you also set the correct Time Zone in the options.
- Save the healthchecks.io check ID to a
healthchecksio_idfile in thePlex-Cleanerdirectory - Set the log file option in
Cleaner.confso that it writes logs to a file:
"LogFile": "/opt/Plex-Cleaner/plexcleaner.log",
- Set up this cron job entry (instead of the one above):
0 4 * * * /opt/Plex-Cleaner/run_plex_cleaner.sh
Now you'll get an email alert whenever something goes wrong:
- The script exits with an error
- The
plexcleaner.loglog file contains anyERRORentries - The script doesn't run at all
SATA and SAS use very similar connectors, but a SATA connector has a gap in the middle. This means that you can physically plug a SATA drive into a SAS connector, but you can't plug a SAS drive into a SATA controller (because it won't
]]>
SATA and SAS use very similar connectors, but a SATA connector has a gap in the middle. This means that you can physically plug a SATA drive into a SAS connector, but you can't plug a SAS drive into a SATA controller (because it won't fit.) SAS controller software can support SATA drives, but SATA controllers don't support SAS drives (even if you have an adapter.)
Why do I have a SAS controller?
I was running Home Assistant and Plex Media Server on a Raspberry Pi 4 Model B. I kept adding more and more Home Assistant add-ons (which run in Docker containers), so it started to get a bit slow. It took a long time to restart Home Assistant, and we started to notice some speed and reliability issues.
I already had a server rack in my closet, so I wanted to get a proper server to put in it. I didn't need anything too fancy. It just needed to be a little bit faster than a Raspberry Pi, so I found a 10 year old IBM server for $249 NZD ($160 USD.) I found it on TradeMe, which is similar to eBay in New Zealand.
- Model: IBM System X3250 M4 E3-1270v2
- CPU: Intel Xeon E3-1270v2 4-core, 8-thread Processor
- Memory: 8GB DDR3 Ram
- Storage: 4x 2.5" SAS / SATA drive bays, 1x 240GB SAS hard drive
- Controller: LSI SAS2004 IR SAS / SATA HBA
- Power: 2x 460w power supplies
The new server has been working great, and Home Assistant is noticeably faster. The 10-year-old Intel Xeon E3-1270v2 CPU has a CPU Mark benchmark score of 6429, which is 8x better than the Broadcom BCM2711 processor in a Raspberry Pi 4B (834.)
The only problem is that the server came with a 320GB SAS hard-drive. It was a bit too small, and it made a lot of clicking noises. I wanted to replace it with a 1TB SATA SSD, which would be faster and silent. I just wasn't sure if my old server would be compatible with a modern SATA SSD.
I did a bit of research, and it seemed like it would work, so I bought a new Samsung 870 EVO 1TB SSD. It was the same size and had the same screw holes as the old 2.5" SAS drive, so I was able to swap the drives and use the same drive bracket. I put in the new drive and installed Debian 11 (with LVM), and everything worked great!
Conclusion: The latest Samsung SATA SSDs are still compatible with SAS controllers in servers that were made in 2012.
]]>favicon.ico file for your website or blog. Things have changed a bit since I wrote that blog post in May 2010.
I've been updating the CSS for my blog recently, and I
]]>It's been a while since I wrote a blog post about creating a favicon.ico file for your website or blog. Things have changed a bit since I wrote that blog post in May 2010.
I've been updating the CSS for my blog recently, and I wanted to update the favicon. Here's the old green square icon that I was using before:
I wanted something a bit more interesting, so I tried using some of the new text-to-image AI tools that have been getting a lot of attention lately. I used Midjourney and DALL·E 2. (Midjourney uses Stable Diffusion.)
I played around with some different prompts in Midjourney and DALL·E 2, and I iterated on a few different ideas. I enjoy spending time in Midjourney's shared Discord channels (e.g. #general-*), where you can see what other people are doing and get some inspiration for prompts.
Here's a few examples of the prompts that I was trying:
website favicon logo, circuit board PCB design, vector, SVG, blue purple gradient, hexagon, bolt and tools icon
small favicon logo for a blog website, circuit board pattern, electronics, vector, epic ultra wide aerial shot from atmosphere, cool gradients, ultra high contrast, blue background, psychedelic color, vortex, hyperrealism, intricate details, cinematic lighting
Here's some of the images that I generated using DALL·E 2:
And here's some from Midjourney:
I eventually found some shapes and colors that looked pretty cool, so I generated some variations using the Midjourney Discord Bot.
Eventually I settled on this one. It kind of looks like an abstract "N". I tweaked the colors and contrast a little bit in GIMP.
I used realfavicongenerator.net to generate a favicon package with lots of different sizes and (mostly unnecessary) features.
They provide this HTML to include in my <head> tag:
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png" />
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png" />
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png" />
<link rel="manifest" href="/site.webmanifest" />
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5" />
<meta name="msapplication-TileColor" content="#603cba" />
<meta name="theme-color" content="#ffffff" />
So here's the new favicon:
P.S. Here's one I didn't end up using, in case you want it:
Wet Bectolon. Crdoruf Ptaogoatuy
(wetbectolon.com is available!)
]]>I originally posted this write-up on the Home Assistant Community Forums. I thought I would post it here as well, since I haven't written anything on this blog for a while. I've been getting more serious about Home Automation lately, so I might start writing some
]]>
I originally posted this write-up on the Home Assistant Community Forums. I thought I would post it here as well, since I haven't written anything on this blog for a while. I've been getting more serious about Home Automation lately, so I might start writing some more blog posts and making a few videos.
I figured out how to control the light and fan in the rangehood above my stove. I learned a lot from this project, and I wanted to share some of my experiences.
- I figured out how to read the state of 4 buttons and light up 4 LEDs with only 5 wires
- I discovered some powerful circuit simulation software
- I used optocouplers for the first time (instead of relays)
- I wrote some C++ code for a custom ESPHome component
- I bought an oscilloscope
Introduction
I wanted to be able to control the light and extractor fan above my stovetop, so I decided to automate it using an ESP32. This would be a similar concept to the Raspberry Pi Microwave project that I built many years ago: I would put a "proxy" circuit between the buttons and the main board that controls the light and fan. The circuit would simulate button presses to control the main board, and it would read the state of the original buttons so that they would still work normally. The circuit would add WiFi connectivity so that the buttons could be controlled remotely and via automations in Home Assistant.
I still use Raspberry Pis, but now I prefer using ESP32 and ESP8266 chips. ESPHome makes it so much easier to build little devices like this, and the OTA (over-the-air) updates are really convenient. I love all the built-in components that you can easily add to your YAML configuration files.
Here's a photo of the rangehood, with some of my tools. You can see the main controller board and the button board hanging out.
This is the button board, which has 4 buttons, and an LED for each button:
How to read 4 buttons and light 4 LEDs with only 5 wires
I wanted to preserve the original functionality of the rangehood buttons, so that you wouldn't even know there's an ESP32 chip inside. This button board was the first obstacle. If there were only 4 buttons, then that would be quite easy. You can just attach them directly to GPIO pins with an internal pull-up resistor. The LEDs were a bit confusing at first, but it was a fun puzzle to solve.
I took a photo of the PCB and traced it using different colors, so I could where each of the wires was going:
I found that button has a 1K ohm resistor, and each LED has a 148 ohm resistor.
Circuit simulation software
I'm a software developer, so I really like having fast feedback loops and the ability to try lots of ideas quickly to see if something works. I started wondering if it might be possible to do this with circuits. I wanted to see if I could figure this out with trial and error and use some kind of circuit simulation software.
I found TinkerCad, and my mind was blown! Now I had a virtual Arduino wired up to a virtual breadboard. I recreated this button/LED board inside TinkerCad. I could set up virtual multi-meters to see how much current was flowing through LEDs, and easily figure out values for resistors.
This is so awesome! It even tells you when too much current is going through an LED:
Here's a link to my TinkerCad project where you can run the circuit simulation.
I started playing around with some code in an Arduino sketch. I figured out how to light up the LEDs. I figured out how to read the buttons. But I couldn't figure out how to do them both at the same time. If I wanted to light up the LEDs, then I'd need to be using 3.3V on the shared wire. But if I wanted to read the buttons, then the shared wire needed to be connected to GND.
Then I finally had a breakthrough. Whenever I had worked with GPIO pins in the past, I had just assumed that "high" and "low" meant "on" and "off". But I realized that "high" literally just means 5V (or 3.3V), and "low" literally just means GND. So you can just change the direction of a circuit if you set your output pins to normally "high", and you swap the GND pin with a 3.3V pin.
So instead of needing to choose between the 3.3V pin and the GND pin, I could use a GPIO pin on the shared wire and toggle it between "high" and "low". I set it to "low" whenever I needed to read the buttons, and set it to "high" whenever I needed to light up the LEDs.
I got something working in my virtual Arduino. The other side of the circuit was quite easy (to simulate the button presses.) I used some optocouplers as relays, where you send current through an IR LED and it turns on a transistor on the other side, but it keeps both of the circuits separated. Later on I realized that I should have been looking at "solid state relays", which are optocouplers that are specifically designed for this purpose and can switch much higher voltages and currents.
I tried it out with a real Arduino, and it worked!
It was a pretty cool experience to simulate a whole Arduino circuit inside my browser, and then see it work in real life.
https://everycircuit.com is also a really cool option for circuit simulation:
It can handle some things that TinkerCad can't do, such as astable multivibrator circuits where you need to inject a little noise to get it started. (TinkerCad just crashes with an "infinite loop" error.)
Custom C++ ESPHome Component
Alright, on to the next step. I've got something working with an Arduino, but how do I port this to ESPHome and get it working over WiFI?
I needed to set an GPIO pin to output mode for a while, briefly switch it to input mode, read the state of a button, and then set it back to output mode again. I couldn't figure out how to do this in a YAML configuration file for ESPHome, so I decided to write a custom component using C++.
The ESPHome documentation is pretty good, and I was able to get something working. Here's my YAML configuration and my custom C++ module: https://gist.github.com/ndbroadbent/63aa5f105e21631bee4ff9a62c9b1608
The most important difference between Arduino and ESPHome is that a component's loop() function is only called once every 16 milliseconds. (Arduino restarts the loop instantly.) This was actually perfect for me, since it meant that I could read the buttons once every 16ms, and leave the LEDs on before I exit the loop.
I ran out of pins on my ESP8266 (Wemos D1 Mini)
I should have made sure that everything worked on a breadboard. Instead, I started to get a bit impatient, and I jumped straight to soldering a prototype PCB. I soldering some pin headers for my ESP8266 development board, and then started with one input, and one optocoupler for the output. I tested it and it was all working, so I kept going.
Then I checked the ESP8266 pinout and realized that I had run out of GPIO pins.
I tried to get away with using some of the yellow "OK" ones, but the boot started failing (I guess I was pulling some of the pins high or low.) I needed 4 pins to control the optocoupler outputs, and 5 for the button/LED board. That's 9 GPIO pins, and the ESP8266 only had 7. I thought it had more!
This can be solved with shift registers. You only need 3 GPIO pins to control a 74HC595 8-bit shift register and get 8 digital outputs. You only need one additional GPIO pin if you want to add a CD4021BE shift register (parallel-in, serial-out) and get 8 digital inputs. This is because the input and output registers can share the same clock and latch pins. You can then daisy-chain both of these to get as many inputs and outputs as you need (or use bigger shift registers with more pins), and use only 4 GPIO pins.
Anyway, I switched to using an ESP32 development board which has 18 usable GPIO pins, and a few more with caveats. I used breadboard jumper wires to wire it up.
My resistors were too big
I had put 1k resistors on the optocoupler outputs, to mimic the original button/LED board. What I didn't realize is that the optocoupler itself seemed to add about 500 ohms of resistance, so the resistance was slightly too high, and the rangehood controller board couldn't reliably read the simulated button presses.
So I unsoldered them and switched to 560 ohm resistors. Should have tested on a breadboard.
I bought a USB oscilloscope
After doing a bunch of unsoldering and resoldering, putting random hookup wires all over the place, and swapping out ESP dev boards, I ended up with the ugliest prototype PCB you've ever seen. Of couse, stuff started breaking and one of my outputs wasn't working properly or reliably. It seemed like there was a problem with one of my optocouplers, and it was a real pain to figure out what was broken since I had only had a multimeter.
I realized that a oscilloscope would be really useful, and I should have bought one a long time ago. I bought a USB BitScope 10. It's pretty cool!
This helped me figure out where some things needed to be resoldered. One of the optocouplers was only working when I pushed on it with my finger. I originally thought it might have been a broken chip or something to do with capacitance, but it was just a broken PCB trace and some dodgy soldering.
I'm still learning how to use the BitScope software and have barely scratched the surface of what it can do. This is going to be extremely useful for future projects, especially for reverse engineering how stuff works. (P.S. You'll need their pre-release version for the latest version of Mac OS.)
WiFi was very weak on my "Duinotech Wearable ESP32 Development Board"
I initially switched to a Duinotech Wearable ESP32 Development Board that I had bought a while ago. I got everything working, but then it really struggled to connect to WiFi and would cut out regularly, and my entities would become unavailable. Even though I had a Ubiquity access point only 10 meters away in the same room.
I found a few other people who seemed to have problems with WiFi as well, but they might be for different reasons:
- https://www.esp32.com/viewtopic.php?t=16311
- https://forum.micropython.org/viewtopic.php?t=11550&p=63020
- https://www.esp32.com/viewtopic.php?t=18159
- https://www.esp32.com/viewtopic.php?t=16622
- https://www.esp32.com/viewtopic.php?t=6073
- https://www.reddit.com/r/esp32/comments/bbjrir/very_poor_wifi_performance/
Then I found the "WiFi Power Save Mode" section in the ESPHome documentation:
NONE(least power saving, Default for ESP8266)LIGHT(Default for ESP32)
So I tried:
wifi:
power_save_mode: none
output_power: 20
I think this seemed to help a little bit, but it still wasn't very reliable. I had ordered some more electronics stuff and threw in another ESP32 Development Board. I tried this out once it arrived and it was an instant improvement. I just threw away the board with poor WiFi. I also ordered a bunch of shift registers so I can get back to using ESP8266 boards (adafruit huzzuh feathers), and they seem to have much better WiFi as well.
I also ordered a few extra Wemos D1 Mini Pros that support external antennas. They haven't arrived yet, but I might try these out for car presence detection. I've been struggling with WiFi range for this as well.
Putting it all together
I put everything into a little black box. Connected all the wires up and stuck it inside the rangehood. I chopped the power cable for the rangehood and added a screw terminal, and wired up a USB charger to power the ESP32 board.
So now I've got the fan and light in Home Assistant. (And the physical buttons still work as well.)
I've set up an entity controller to turn on the light.
Next steps
The rangehood controller board has a piezo buzzer that beeps every time a button is pressed. That's pretty annoying. I might try to desolder or destroy the buzzer.
I want to put an air quality sensor in the kitchen and automatically turn on the extractor fan based on AQI. I've ordered some ZigBee air quality meters on AliExpress, and they should arrive in a few weeks.
Update from Sep 30, 2022: I've put the air quality sensor in the kitchen, and have set it up to turn on the rangehood fan when the PM 2.5 level is above 12 µg/m³. (It usually hovers around 6 µg/m³.) It's been working really well!
I also bought a current clamp sensor that can measure AC current. I want to set this up for the stovetop so I can detect when it's on and automatically turn on the fan (in advance, instead of waiting for the AQI to get bad.) It would be easier if I could use an energy monitoring wall plug, but it looks like the stovetop is wired directly into the circuit breaker, and it uses a lot of power so I don't want to mess around with those wires. I'll just separate them and put a clamp around one of them. I'll follow this guide to set it all up and get it working on an ESP32.
I have this power meter that I'll use to calibrate it.
I also want to learn how to make a proper PCB design in KiCad and order a cool purple PCB from OSH park.
I'm really enjoying this self-directed crash course in electronics. It's really fun to learn so much while working on practical projects that we can use every day in our house.
Thanks for reading, it was fun to write up everything I learned. If you have any questions, please feel free to ask in the comments!
]]>The following Docker solution broke when I started using a M2 MacBook. I wasn't able to run the older version of Hugo in a Docker container (even with Rosetta.)
I haven't written any blog posts for
]]>The following Docker solution broke when I started using a M2 MacBook. I wasn't able to run the older version of Hugo in a Docker container (even with Rosetta.)
I haven't written any blog posts for a while. One reason is that I've been hard at work on DocSpring for the last few years, and I haven't had a lot of time to work on personal projects. But the main reason is that my blog uses an older version of Hugo, which is a "static site generator" [1].
I switched from Jekyll to Hugo in 2017, and the current version of Hugo at the time was 0.21. I found a cool theme called hugo-sustain. Everything was great for a few years.
Time passed. One day, I tried to update a post or write a new post (I can't remember which.) I realized that my build and develop scripts were broken. I was a new computer at that point, and I had updated my macOS version. I installed the latest version of hugo and saw a bunch of interesting and confusing error messages when it tried to compile my old themes and layouts. I tried to downgrade hugo to version 0.21, and it crashed with a segfault (it was built for an older version of macOS.) I cloned the hugo repository and tried to compile it from source, but my Go version was too new, so it failed to compile. Finally, I downgraded my Go version to an older version that was released around the same time in 2017. I held my breath as I tried to compile hugo one last time. Go tried to fetch all of the required dependencies, and crashed with a bunch of 404 errors. Apparently some of the packages had been renamed and moved around, and the older versions had been removed from the Go package index.
So I gave up for a while. Instead of generating my blog from the source, I switched to editing the static files directly. Sometimes I would need to correct a typo or adjust some styles, so I'd go into the generated ./public directory and manually modify the raw HTML and CSS.
Time passed. One day, I started to notice some activity on a blog post that I had written 11 years ago. This post is about a GIMP plugin called deskew that makes it easy to scan old photos in batches on a scanner and automatically rotate them. I had dropped this plugin file in my Google Drive and had pasted a link to the file. The link worked great for 10 years, and people were able to download the file without any issues. But eventually Google changed something and the link was no longer working. I started to receive emails from people who were requesting access to this file.
I manually shared the file a few times. Then I decided to download the file and check it in to the blog repo. I started going into the ./public directly to update the HTML, but I decided it was time to have another crack at this Hugo problem and fix my blog.
Should I switch to Ghost? I've loved using Ghost for the DocSpring blog. It's a really nice blogging platform that I self-host on Digital Ocean for $5/mo, and I enjoy the WYSIWYG writing experience a little more than editing plain Markdown files. (Images are a bit annoying.) But I didn't want to migrate all my posts over to ghost.org and get locked in, or spend $5/mo for the rest of my life. I just want some static HTML/CSS that I can put on GitHub Pages forever.
Should I upgrade to the latest version of Hugo and spend hours fixing up the themes and tweaking all my posts until everything is working again? No thanks. Hugo runs on my local machine and produces static HTML and CSS content. It's a pure function. There are no security vulnerabilities to watch out for. As far as I'm concerned, Hugo version 0.21 is "finished software." It generates my blog, and I'm happy with my blog. I will continue to be happy with my blog for many years to come. I don't need the latest and greatest features. I just want something stable that I can use over the next few decades without the constant grind of updating packages, breaking things, and debugging random issues. Give me Hugo 0.21 from May 2017!
I was even tempted to throw everything away and start from scratch with something old and stable. Preferably written in Bash, C, or Perl. There's a lot of cool new languages out there but they often "move fast and break things." The POSIX standard was created 33 years ago in 1988, so I could still run some shell scripts that are over 30 years old. (I asked Hacker News for some examples: Ask HN: What is the oldest Posix script that I can still run in a modern shell?.)
I had a sudden burst of inspiration:
Docker!
I could run Hugo in a Docker image! If I can get Hugo 0.21 running in a Docker image, then I can save that Docker image into a *.tar.gz file and store it right in my git repo. Then I have a static hugo binary that comes packaged with everything it needs to run in a consistent environment, and I can run it anywhere (Linux, Mac OS, Windows.)
I found a Dockerfile in this docker-alpine-hugo repo, and I just needed to change 0.55.3 to 0.21. Everything worked on the first try! [2]
How I use Docker
Instead of running hugo, I run a ./hugo wrapper script that runs hugo inside a Docker container:
#!/bin/bash
docker run --rm -v "${PWD}:/src" hugo-alpine hugo "$@"
I have a build_docker script [3] that builds the Docker image and saves it to hugo-alpine.tar.gz:
#!/bin/bash
set -euo pipefail
echo "Building Dockerfile..."
docker build . -t hugo-alpine
echo "Saving image to hugo-alpine.tar.gz"
docker save hugo-alpine > hugo-alpine.tar.gz
If I'm on a new computer, I can just run docker load -i hugo-alpine.tar.gz to load the hugo-alpine image into Docker.
I have a dev script that starts the hugo server and makes it available on port 1313:
#!/bin/bash
docker run --rm -v "${PWD}:/src" -p "1313:1313" hugo-alpine hugo --watch serve --bind 0.0.0.0
Finally, I have a deploy script that generates the static site into ./public, then pushes the result to a gh-pages branch:
#!/bin/bash
set -euo pipefail
if [ ! -d public/.git ]; then
rm -rf public
REMOTE="$(git remote get-url origin)"
git clone "${REMOTE}" public
(cd public && git checkout gh-pages)
fi
docker run --rm -v "${PWD}:/src" hugo-alpine hugo
cd public
git add -A
git commit -m "$(date)"
echo "Pushing build..."
git push
cd ..
echo "Pushing source..."
git push
You can view the source code for my blog here: https://github.com/ndbroadbent/ndbroadbent.github.io
I'm very happy with this workaround. Now I'm back in business, and I can update or write new blog posts to my heart's content. This new Docker-based setup should last me for the next few decades, if not longer. I still love how Hugo is super fast and generates my entire blog in about 6 seconds (even version 0.21!) I'm in no hurry to switch to anything else.
A static site generator converts a folder full of Markdown files into a plain HTML/CSS website that you can host for free on GitHub Pages or Netlify. ↩︎
I think it's generally much easier to get older Linux packages running, especially when it's a single Go binary with no dependencies. I wish it was this easy on Mac! ↩︎
Hopefully I never have to run this again! ↩︎
I bought a wifi-connected kettle and flashed it with a custom firmware.
myle
This is a "Tuya" product which uses an ESP32 chip. Tuya provides hardware and software for a lot of white-labelled products, and it's a very open platform with a lot of documentation. This means that a lot of off-the-shelf devices can be flashed to run custom firmwares such as Tasmota and ESPHome.
We used to be able to use a tool called tuya-convert, which would hack devices wirelessly and flash a custom firmware over-the-air (OTA). I tried tuya-convert, but I got this error:
Your device's firmware is too new.
Tuya patched the PSK vulnerability that we use to establish a connection.
You might still be able to flash this device over serial.
So I had to open it up, connect some wires, and flash it via serial. I used the GPIO pins on a Raspberry Pi.
It worked! I flashed the kettle with Tasmota and saw data coming into MQTT.
I set up a flow in Node-RED to monitor and control the temperature setting on the kettle.
Hyper Text Coffee Pot Control Protocol
I was worried that I might accidentally ask the kettle to brew coffee, so I implemented support for the Hyper Text Coffee Pot Control Protocol. I did this by returning the HTTP status: 418 I'm a teapot.
(I guess it's not a teapot, but it's certainly not a coffee machine.)
The kettle now responds to an API request at /bc (which stands for "brew coffee"), and will respond with status 418.
~/code/Tasmota $ curl -v 192.168.1.113/bc
* Trying 192.168.1.113...
* TCP_NODELAY set
* Connected to 192.168.1.113 (192.168.1.113) port 80 (#0)
> GET /bc HTTP/1.1
> Host: 192.168.1.113
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 418
< Content-Type: text/plain
< Content-Length: 12
< Connection: close
<
* Closing connection 0
I'm a teapotThe commit that adds this important functionality
FAQ
- Q: Why would you want to flash your kettle with a custom firmware?
- A: The original Tuya app uses a cloud service, and I don't like relying on third-party services. Tasmota is fully local and doesn't make any requests to the public internet. This means that I don't need to worry about hacks, leaked data, or the company going out of business and shutting down the app.
- Q: Ok but why do you have a smart kettle anyway?
- A: Honestly, because I thought it would be funny. But my wife does like to have it turn it on automatically when she gets out of bed.
I used React Native to build a cross-platform game for iOS, Android, Windows, and the web. SudoBlock is a cross between Sudoku, jigsaw puzzles, and Tetris.
You can find SudoBlock on the web, App Store, Google Play, and the Microsoft Store.
React
]]>
I used React Native to build a cross-platform game for iOS, Android, Windows, and the web. SudoBlock is a cross between Sudoku, jigsaw puzzles, and Tetris.
You can find SudoBlock on the web, App Store, Google Play, and the Microsoft Store.
React Native only supports iOS and Android, but I used react-native-web for the browser, and react-native-windows for Windows desktop and phone. The UWP app can also run on Xbox One and HoloLens. I also experimented with react-native-macos and react-native-appletv, but they're not being maintained.
Here's some of the things I learned while building SudoBlock:
You should probably use a game engine
I've heard good things about Godot and Unity.
These game engines support iOS, Android, Windows, and Linux. Unity supports
many other platforms.
Making a simple game was a great way to learn React Native, but it's not the best tool for the job.
However, people have started working on some game libraries for React Native, such as react-game-kit and react-native-game-engine.
It's not the easiest way to make a game, but it's also not impossible.
The built-in Animated library is great,
and I also used react-native-animatable.
I used a library to play sounds.
I wrote native code to integrate with iOS Game Center and Google Play Game Services.
I used libraries to integrate with InApp Billing on Android,
and in-app purchases on iOS.
I also wrote native code for ads and in-app purchases on Windows.
There's a RN library for particle effects
(although you'd have to add support for Android),
and react-game-kit provides a way to manage sprites and animations.
I want to make some more simple 2D games, and I'm going to stick with React Native for now. I can fork SudoBlock and reuse a lot of the code that I've already written.
If your goal is to quickly build a cross-platform mobile game, then I'd recommend learning Godot or Unity.
Don't worry about supporting Windows
This is a no-brainer. No one uses Windows Phone and Microsoft have abandoned it.
I wanted to explore and learn new things, so I decided to do it anyway. I enjoyed the process
of installing Windows, working with Visual Studio, and writing some C#.
I also figured out how to write cross-platform npm scripts using scripty.
react-native-windows gives you a UWP app that can
run on Windows Phone, tablets, desktop, Xbox One, HoloLens, and other Windows platforms. But:
- Nobody uses Windows Phone.
- Mobile games don't usually make sense on an Xbox.
- If your game runs in a browser, then you probably don't need a desktop app.
React Native might be better than Electron for desktop applications
When I was running SudoBlock as a Windows desktop application, I realized that
React Native could be a better choice than Electron.
Electron apps are notorious for being huge and using a lot of memory (e.g. Slack and Spotify.)
They have to package and load an entire WebKit browser.
React Native apps are much smaller and use far less memory, because they only need a JavaScript engine.
You can also use responsive design, so that a single codebase works on desktop, mobile, and the web. The only problem is that react-native-macos
is unmaintained and out of date, so it would be great if a company sponsored development.
You'll probably write some native code
Some people are attracted to React Native because they've heard that you can write a mobile
app with only JavaScript. In practice, this is only true for extremely simple applications.
Most of your code will be written in JavaScript, but you'll probably have to write some native code.
At the very least, you must be prepared to fix some bugs in third-party libraries.
I started contributing to react-native-admob, and sent a pull request to allow multiple test devices. That was my introduction to native code in React Native, and I had to work with JavaScript, Objective-C, and Java.
I also did a lot of work on react-native-blur. When I first tried to use it, it was completely broken on Android, and there were lots of problems on iOS.
It took a lot of work to get everything running. I could have just skipped the blur and used a darkened overlay, but I enjoyed the work and learned a lot.
I also had to write native code to integrate with iOS Game Center and Google Play Game Services,
and for ads and in-app purchases on Windows. I also wrote a small library to manage vibrations and haptic feedback across iOS, Android, Windows, and the web (using the Vibration API.)
There will be bugs
React Native is pretty stable, but there's a lot of unmaintained libraries,
and most libraries don't have any tests. React Native is a bit like jQuery, in that it smooths over a lot of quirks and inconsistencies and provides a consistent API. But there's some tricky edge cases,
and I often had to read the React Native source code to figure out why something was happening.
Some examples:
- There was a bug with the native animation driver for Android.
- There were crashes in the
react-native-sound library. - The Android GC seemed to be clearing some memory that shouldn't have been cleared.
Android was particularly unstable. Not just React Native, but Android itself.
I didn't have too many problems with iOS.
React Native vs. iOS / Swift
I wrote an iOS app with Swift a few years ago, and I've actually had a much better experience with React Native.
When I was working with UIKit, I remember constantly fighting with things like layout, contraints, and font rendering. I uncovered some actual bugs, and found long threads on the Apple forums that were being ignored.
It was really nice to let React Native handle all of the rendering. I had no rendering issues on iOS or Android,
and just a few problems that I fixed on Windows.
Swift was also very unstable at the time, and Xcode upgrades took a lot of effort.
My Obj-C code still compiles a year later on the latest version of Xcode. If I was using Swift,
I think it would take at least a day to upgrade to Swift 4.1 and update all of the third-party libraries.
I believe Swift is more stable now, and I love the language, so I might start using it again on future projects.
I had a lot of headaches with React Native, but it wasn't as bad as Swift v1 and UIKit.
Marketing is really hard
-
I tested Facebook ads with $50. I reached about ~7,000 people and got ~50 clicks. One person
ended up buying the game for $2.99, so I made $2. You can't spend $50 to make $2. -
I posted on Reddit a few times:
-
A German website
published an article about SudoBlock. -
I tried to capitalize on #covfefe,
which didn't work at all. But I repurposed that new code into an Emojidoku mode. -
I found a game publisher who was going to handle all the marketing and split the revenue.
They even promised to get the game featured on the App Store.
We signed the contract and I took the apps down for a while, but the publisher fell off the radar and stopped replying to my emails. -
I finally got around to writing this blog post.
Other Notes
-
I switched from Sublime Text to VS Code near the beginning of the project.
VS Code is awesome. It's super fast and very customizable. -
I set up Flow and started using Immutable.js.
I love having static type checking for JavaScript. -
I used Airbnb's eslint config, and spent about a day
fixing all the issues. Every time I saw a rule that I didn't understand, I looked it up to understand their reasoning. I read through a lot of great discussions on Github. This was a great way to learn more about
JavaScript, and especially some of the new ES6 features. -
I starting doing some functional programming with lodash/fp and ramda. I had fun refactoring some code in a more functional style.
-
I really enjoyed working with redux-saga, which helped me clean up a lot of messy code.
-
I learned a lot about Reactive programming. This post is amazing: The introduction to Reactive Programming you've been missing. I started playing with RxJS and
redux-observable. -
I set up CodePush, so that I could push JS changes without releasing a new version to the App Store. The setup guide is very helpful.
-
I had to debug some memory issues on Android, and this article was really helpful: React Native Android App Memory Investigation
-
I learned about the webpack DLL plugin, which made development a lot faster. You can compile everything in
node_modulesas a separate bundle. You only need to do that once, so it saves a lot of time. -
I released a boilerplate project with my webpack config for react-native-web.
-
I learned about the Babel AST while working on an issue in a Babel plugin, related to react-native-web.
-
I wrote a script that stripped unused glyphs from icon fonts, to reduce the app size.
-
I tested the app while simulating a slow network connection in Chrome. This revealed a bug where the counter started ticking before everything had finished loading.
-
I discovered that it takes a huge amount of effort to actually launch a game. Once I had a playable game, it was another 2 months before everything else was finished. Things like in-app purchases, ads, analytics, high scores, achievements, tutorials, app store listings, screenshots, icons, social media accounts, etc.
This was about 3 months of work, and I was in a state of flow most of the time. I learned a lot of new things, and I really enjoyed the whole process.
The game has only made about $50 so far, but I have some more ideas for grid-based games, and
I can reuse a lot of the SudoBlock code. I'm also a freelancer, so this is the only way I can pick up new skills.
Thanks for reading! If you have any questions, please leave a comment on Hacker News.
- I'm currently working on DocSpring, which is an API for filling out PDFs.
I thought this was a pretty cool idea, and that it
]]>A few months ago, I was inspired by this post on Reddit, titled: Food items should have QR codes that instruct the microwave exactly what to do. Like high for 2 minutes, let stand 1 minute, medium 1 minutes..
I thought this was a pretty cool idea, and that it would be a fun project for a Raspberry Pi. I agreed with the people who thought using UPC barcodes would be better, since products already have them, so I went with a barcode scanner + online product database.
Here's a summary of the features that I've added to my microwave:
- Re-designed touchpad
- Nicer sounds
- Clock is automatically updated from the internet
- Can be controlled with voice commands
- Can use a barcode scanner to look up cooking instructions from an online database
- There weren't any online microwave cooking databases around, so I made one: https://microwavecookingdb.com
- The microwave has a web page so you can control it from your phone (why not), and set up cooking instructions for products
- Tweets after it's finished cooking something (See https://twitter.com/rbmicrowave)
Demo Video
Using a Raspberry Pi to cook a Raspberry Pie
Hardware
I used a microwave with a touchpad, and discovered that the touchpad was a button matrix. I took photos of the touchpad and traced the wires, so that I could tell which pins corresponded to which buttons.
I initially wanted to put everything in a case outside the microwave, but I decided that it would be more challenging and fun to try and fit everything inside. Here's all the PCB revisions, before I settled on a design that would fit neatly on top of the microwave's original PCB.
I used shift registers and optocouplers to control the touchpad pins. To listen for touchpad presses, an output shift register scans one line at a time on the first touchpad layer, and an input shift register listens for connections to the second layer.
I unsoldered the touchpad connector from the original circuit board, and replaced it with a row of pin headers. I then used the original touchpad connector on my PCB, so that my circuit acts as a kind of proxy for button presses.
Here's the final product after transferring toner, etching, drilling, and soldering. (I had to use the ribbon cables to save space.)
And here's how it fits on top of the microwave controller:
When I peeled off the old touchpad overlay, it became wrinkled and ugly, so I thought I may as well have a go at redesigning the interface. My goal was to get rid of the features I don't really use, and make the basic functions more convenient. The top two rows of buttons are now dedicated to "one touch" cooking times, for either "high" or "medium" power. You can also set the time and power manually.
You might have noticed that I started the project with the intention of using an Arduino Nano plugged into a Raspberry Pi USB port. This was because I was a) familiar with the Arduino, b) not familiar with the Raspberry Pi GPIO, and c) thought it would make testing and debugging a bit easier, since I could just plug the Arduino into my laptop.
However, it turns out that my Raspberry Pi had some issues with the arduino's FTDI chip if the Raspberry Pi was turned on while the Arduino was plugged in. It wouldn't recognize the Arduino until I unplugged it, and then plugged it back in again.
So I decided to make an Arduino Nano => Raspberry Pi GPIO adapter, and port my Arduino code to the Raspberry Pi GPIO using the WiringPi library.
The Raspberry Pi is powered by a USB hub, which is also plugged into the Raspberry Pi's USB port. To power the hub, I wired up a power adapter to the microwave's power source. There's also a USB powered speaker, USB microphone, wifi adapter, and barcode scanner.
Microwave Software
All the software running on the Raspberry Pi is hosted at: https://github.com/ndbroadbent/raspberry_picrowave.
There are 4 main components:
Microwave Daemon
This runs the code that listens for touchpad button presses, and controls the microwave. It also accepts TCP connections so that other programs can send commands or request information about the microwave's status.
Barcode Instructions
This program listens to the barcode scanner, and requests product information from the Microwave Cooking Database. It also runs the cooking programs.
Voice Control
I used PocketSphinx for voice recognition, which worked very well with my small corpus. I embedded Ruby in the pocketsphinx_contiunous C program, so that it would be easier to script voice commands and send commands to the microwave daemon. It turns out that the acoustics of my kitchen seem to mess up the recognition, so it won't be used very often.
Sinatra App
There's a simple sinatra web application that lets you control the microwave from your phone or computer. This may not be a big selling point. It uses a JavaScript EventSource to push updates to the browser, so you could have hundreds of users connected to your microwave at once.
If any barcodes are scanned that can't be found on the Microwave Cooking Database, this webpage will display the unknown barcodes and provide a link to add the new product.
Microwave Cooking Database
I couldn't find any existing websites with a database of microwave cooking instructions, so I made one.
UPDATE: It used to be live at https://www.microwavecookingdb.com/, but the domain has now expired. Sorry!
If cooking instructions are posted for a 1000W microwave, you can request the instructions for a 700W microwave, and the cooking times will be automatically adjusted.
So if you're also planning on making an internet connected microwave with a barcode scanner, please feel free to sign up and add some products.
Thanks for reading!
I'd be interested to hear if you build something similar!
]]>