Today I Learned (TIL)
TIL stands for today I learned. This is a collection of small, practical things I’ve learnt while writing software, which I thought were worth remembering and sharing with other people.
If you want to follow along, these posts have their own RSS feed.
Filter by tag
- alpine
- amazon-dynamodb
- applescript
- aws
- bash
- buildkite
- colour
- css
- csv
- curl
- datetime-shenanigans
- digital-preservation
- discord
- docker
- dreamwidth
- ffmpeg
- fish
- flask
- flickr
- git
- github
- github-actions
- home
- http
- https
- imagemagick
- images
- interesting-words
- java
- javascript
- jekyll
- json
- jxa
- liquid
- live-text
- macos
- mastodon
- naming-things
- netlify
- networking
- nginx
- obsidian
- photokit
- playwright
- podcasts
- python
- rsync
- ruby
- rust
- safari
- scala
- screenshots
- shell-scripting
- sparql
- spreadsheets
- sqlite
- svg
- swift
- theatre
- trains
- tumblr
- unicode
- wayback-machine
- web-dev
- wikimedia-commons
- wordpress
- youtube
These are the current filters.
-
Use
$_to get the last argument to the previous bach commandThis allows you to write commands like
mkdir myfolder && cd $_orgit init myrepo && cd $_. -
With Flask-Login, you want
current_user == None, notcurrent_user is Nonecurrent_useris a proxy object that happens to be wrappingNone, but isn’t actuallyNone. -
How to get the expiry date of an HTTPS certificate
-
Write to the middle of a file with Python
Open the file with mode
r+to be able to seek around the file and write to it. -
Create solid-colour image placeholders to show before an image loads
-
Get an image from a video with ffmpeg
-
Get the embedded artwork from an MP3 file
Use the command
eyeD3 [MP3_FILE] --write-images [FOLDER]. -
Convert an animated GIF to an MP4 with ffmpeg
-
Using the Wikimedia Commons API to tell if a file has been deleted
-
Build a URL with query string parameters with curl
A combination of
--getand--data/--data-urlencodeallows you to write curl commands which are readable and expressive. -
How to get the selected item in Finder using AppleScript
-
Removing letterboxing from a video screenshot with ImageMagick
Using
-trimwill remove the black portions and leave you the unletterboxed image. -
How to check the quality of a network connection
Using an
NWPathMonitorand inspecting the value ofNWPath.status,NWPath.isExpensiveandNWPath.isConstrainedcan tell you what sort of connection you’re running on. -
How to highlight Python console sessions in Jekyll
Adding a couple of options to the
consolelexer (console?lang=python&prompt=>>>) gets you syntax highlighting for a Python console session. -
Not all coal is the same
-
How to simulate an
[Errno 54] Connection reset by peerwhen using pytestYou can run a TCP server in the background using a fixture, and using the
SO_LINGERsocket option can reset the connection. -
What does “insomnolent” mean?
-
Open a Safari webarchive from Twitter/X without being redirected
Disabling JavaScript when you open the webarchive file will prevent you from redirecting you to twitter.com.
-
Writing a file in Swift, but only if it doesn’t already exist
Adding
Data.WritingOptions.withoutOverwritingto yourwrite()call will prevent you from overwriting a file that already exists. -
What’s inside a Safari webarchive?
The inside of a
.webarchivefile is a binary property list with the complete responses and some request metadata. -
Get a Palette colour as a command-line argument with Clap
Wrapping a
Palette:Srgbin a struct and implementingFromStrfor the struct allows you to take hexadecimal colours as command-line inputs. -
How to get the filename/size without downloading a file in curl
You can do some fun stuff with the
--write-outflag and variables. -
There’s a musical that tells you the number of minutes in a year
The song Seasons of Love from Rent starts with the line “Five hundred twenty-five thousand, six hundred minutes”.
-
How to get a tally of tally counts in SQLite
Using a nested query allows me to perform a two-level aggregation of the values in a column – how many values appear once, how many twice, and so on.
-
How to get a list of captures from the Wayback Machine
Use the CDX Server API to get a list of captures for a particular URL.
-
How to take a screenshot of a page in the Wayback Machine
Using Playwright to take screenshots and adding some custom styles gets a screenshot of a page without the Wayback Machine overlay.
-
How to count how many Discord messages were sent on a given day
Using the
Duringfilter gives me a count of how many messages were being sent. -
How to see the HTTP requests being made by pywikibot
To see exactly what HTTP requests were being made, I modified the library so that betamax would record requests.
-
How to embed an inline SVG in a CSS rule
Modern browsers allow you to embed the SVG almost as-is, with just a couple of characters that need escaping – no base64 required!
-
How to shuffle an array in a Jekyll template
If you want an array in random order, you can use the
samplefilter to get a random sample of the same size as the original array. -
Checking if a URL has changed when you fetch it over HTTP
When you make an HTTP request, you can use the
If-Modified-Sinceheader to get a 304 Not Modified if nothing has changed since your last request. -
The
opencommand can ask questionsIf you pass an argument that can’t be easily identified as a file or a URL,
openwill ask you what to do next. This may be a surprise if you were trying to use it in a script. -
Why is Pillow rotating my image when I save it?
Images can have orientation specified in their EXIF metadata, which isn’t preserved when you open and save an image with Pillow.
-
How to change the name of an internal link in an Obsidian table
Escaping the pipe like
[[filename\|display text]]allows you to customise the of a link in a table. -
Getting a boto3 Session for an IAM role using Python
Why I use Sessions in boto3, and the Python function I use to create them.
-
My config for running youtube-dl
The flags and arguments I find useful when I’m using youtube-dl.
-
How much will Mastodon instances try to re-send messages?
-
Get my Netlify bandwidth usage from the API
-
How to restrict a page to specific IP addresses
-
Use the
-n/-iflags to avoid overwriting files withcpandmv -
How to parse URLs in JXA
-
Using
errexitand arithmetic expressions in bash -
How to check when an HTTPS certificate expires
-
The COUNT(X) function only counts non-null values
-
What happens when you replace a photo on Flickr?
-
Exclude files from Time Machine with
tmutil addexclusion -
You need to call
resp.close()to close the file opened bysend_file() -
Add the
-vflag to see whatrmis deleting -
Custom error pages in Flask
You can use
app.error_handlerto add custom responses for HTTP status codes, so the errors match the look and feel of the rest of the site. -
Use the
{% raw %}tag to describe Liquid in LiquidIf you’re trying to write about using Liquid tags in a Liquid-based site, wrapping your tags in the
{% raw %}tag will prevent them being rendered. -
What characters are allowed in titles on Wikimedia Commons?
-
Run a script on macOS on a schedule using a LaunchAgent
-
Beware of using
test -nwith command expansion -
How to do offline geo-lookups of IP addresses
MaxMind offer databases you can do to look up IP addresses without sending the address off to a remote service.
-
How to tally combinations of values across multiple columns
-
How to move files when you need sudo on the remote server
-
How to gracefully restart a gunicorn app
-
How to find the longest common suffix in a list of strings in Python
-
How to simulate shell pipes with the subprocess module
-
Use the
{% capture %}tag to assign complex strings to variablesIf you want to get a string that’s semi-complicated to construct, you can put a “mini-template” in the
{% capture %}tag to build it over multiple lines. -
How to create a footer that’s always at the bottom of the page
-
WordPress URLs that get hammered by spammers
-
Manage MP3 metadata from iTunes with eyed3
-
Sort a list of DOM elements by sorting and calling
appendChild() -
How to create flag emojis for countries in Python
-
Use the IMAGE function to insert an image into a spreadsheet
-
How to style a
<details>element differently depending on whether it’s open or closed -
Pausing the animation of <svg> elements can affect child <svg> elements differently in different browsers
-
Create a directory before you
cpormva file to it -
Running the Netlify CLI in GitHub Actions
-
Use shlex.split() to parse log files quickly
-
Use the regex library to get Unicode property escapes in Python
-
Get and manipulate the contents of a page in Safari with
"do JavaScript" -
SVGs are only rendered on GitHub if you use an
<img>that points to another file -
Animate an attribute of an element with <animate>
-
Run a randomly selected subset of tests with pytest
By reading the code for the
pytest-random-orderplugin, I was able to write a new plugin that runs a random subset of tests. -
Getting a tally of SQLite values as a CSV
-
Using sqlite-utils to convert a CSV into a SQLite database
You can use sqlite-utils on the command line to create a SQLite database from a CSV file.
-
Python’s sqlite3 context manager doesn’t close connections
The
sqlite3.connect(…)context manager will hold connections open, so you need to remember to close it manually or write your own context manager. -
How to delete albums
-
Live Text is aware of how hyphenation works (kinda)
-
Go between M-IDs and filenames on Wikimedia Commons
-
Telling mechanize how to find local issuer certificates
Calling
browser.set_ca_data(cafile=certifi.where())will tell where mechanize can find some local SSL certificates. -
Why I prefer XML to JSON in the Wikimedia Commons APIs
The XML-to-JSON conversion leads to some inconsistent behaviour, especially in corner cases of the API.
-
How to do resumable downloads with curl
-
Find files that use a particular SDC field
-
Why the term “snak” keeps appearing in the Wikidata API
-
The acronym “woe” in the Flickr API stands for “Where On Earth”
-
Use Unicode property escapes to detect emoji in JavaScript
-
Finding the original page for a post on Mastodon
Following the logged-out 302 Redirect takes you to the original post.
-
Use concurrency gates to prevent concurrent deployments
-
How to profile Swift code
-
How to use hex colours with the palette crate
You can use
Srgb::from_str()to parse a hexadecimal string as a colour in the palette crate. -
Calley-ope (calliope) Syndrome is pronouncing a word wrong because you’ve only ever read it on the page
-
Use
git check-ignoreto debug your.gitignoreRunning
git check-ignore --verbose <PATH>will tell you which rule applies to a given path, and where that rule is defined. -
Installing mimetype on Alpine Linux
-
How do Dreamwidth posts IDs work?
They were deliberately non-sequential as an anti-spam technique. It’s no longer required, but it’s in the codebase now and hasn’t been changed since it was written.
-
DynamoDB: Conditional updates on nested fields
-
How to iterate over the lines of an InputStream in Scala
-
Manipulating URL query parameters in JavaScript
-
The Content-Disposition header can be used to name a downloaded file
-
How to suppress installing rdoc/ri docs when running
gem install -
Replace black/white parts of an image with ImageMagick
-
How to set the clock on a Horstmann Electronic 7 water heater
-
Pushing with
--force-with-leaseis safer than with--forceIt checks the remote state of the origin hasn’t changed since you last fetched, so you don’t risk overwriting anybody else’s commits.
-
Create compact JSON with Python
Calling
json.dumps(…, separators=(',', ':'))will reduce the amount of whitespace in the final string. -
List all Git object IDs and their type
-
Why does Hypothesis try the same example three times before failing?
-
How to set the clock on a Tricity Bendix SIE454 electric cooker
-
How to set the clock on a Worcester 28CDi boiler
-
Get an RSS feed of external audio posts on Tumblr
If you add
/podcastto a Tumblr site, you get a podcast-like RSS feed for all the external audio posts on that site.