Song: Coming Up for Air

I was trying to make some background music for It's EELectric!, but I failed.  I had various ideas, mostly working around an undersea theme.  I originally wanted something like the "Evan's Room" song from Legend of Zelda: Majora's Mask.  (digression: I always associated that song with being underwater because that's the context in which you hear it in the game, but I just learned from Zeldapedia that it's also the "Game Over" music from the original Legend of Zelda.)

Anyway after various ideas didn't seem to work, I settled on a sequence of 9th chords.  I expanded that into a more complete piece with a solemn, contemplative verse and a hopeful chorus.  Then I realized that it didn't at all fit the whimsical theme of the game for which I was writing it, and completely gave up on using it in that context.


Now I don't have anything to attach it to, so here it is by itself:

Coming Up for Air

Maze generators

I'm working on another game for the kids, called "Mr. Floppypants", of which I'll say no more in this article, except that I'm using node.js, and that for it I required some code to generate a maze.  For once not wanting to write my own code for that particular subfeature, I searched around npm for "maze generator" packages.  There were a few.

Watch out for bull-men!

I set about looking into each one in turn to see if it would suffice, but I discovered for each one a reason not to use it.  One of them simply didn't run, another used Ecmascript 6 (and so wouldn't work in some browsers without being transpiled), one only worked for square mazes, and some (and I admit I was a bit picky) just weren't written to my taste.  Maze generation is a fun little project, and it's natural that many people have made their own generators, in varying degrees of quality.  Yes, I gave up and wrote my own.

Here's what I learned about mazes... wait, no.  Instead let me point you at some much better sources of information:

Jamis Buck writes a mostly-programming blog, and wrote a great series on maze generators in 2010 and 2011.  This in-browser presentation sums up the whole maze generation concept very nicely, and this recap of the blog series is a great place to get more details.  He describes how to implement a handful of algorithms, and provides in-browser demos so you can watch the algorithm run.

Walter Pullen has such a variety of interests that I refuse to summarize his site, in one part of it you can find a very thorough maze generator program he wrote, and he has written extensively about maze generation.  Also I just can't keep myself from mentioning his extensive files about the movie Labyrinth.

Now you know everything about mazes.  But just in case you didn't spend hours reading through all that stuff, here are some highlights:

A maze is basically equivalent to a spanning tree of a graph based on a rectangular grid.  Consequently, algorithms that compute spanning trees can be adapted to make mazes.  Kruskal's algorithm is like this, and builds mazes in an seemingly magical way:  It connects random adjacent cells, taking care not to create a loop, until a maze basically appears out of the noise.

Speaking of Kruskal's, the hardest part is quickly deciding whether two adjacent cells are already connected.  It can be done with a disjoint-set forest, and the tricks they use to make that fast (union by rank and path compression) are neat.

One of the simplest algorithms, recursive-backtracking, is my pick for making the best-looking mazes.  While generators like Kruskal's tend to make lots of clustered dead-ends, recursive-backtracking prefers meandering passages.  It works as basically a randomized depth-first-search of the grid.

Eller's algorithm can generate the maze one entire row at a time.  It works by keeping track of which cells connected to the previous row have already been connected (possibly by an earlier, now forgotten row).  On the very last row, it patches up any not-yet-connected regions of the maze.  In doing so, it may end up with a long straight passage along the bottom.

The Aldous-Broder algorithm generates all possible mazes with equal probibility, but you probably don't want to use it because it's very slow.  It's interesting to think about though: Start in a random cell and perform the well-named  "drunkard's walk", digging a passage when (and only when) you walk into a cell you've never visited before.  Assuming you have enough patience, you'll eventually visit every cell and oh hey, there's a maze!

It's not a generator, but "dead end filler" is an amusing way to solve some mazes without ever visiting cells in the solution path.  Find all the dead ends in the maze and fill them in.  Then find any new dead ends you created, and fill those in too.  Repeat until there are no more dead ends.  The remaining unfilled spaces are the solution.

Back to my project:  I implemented three algorithms (recursive-backtracking, modified Prim's, and Kruskal's).  I also used this as an opportunity to learn about node and npm, so the algorithm packages also have unit tests, benchmarks, and demos, and I spent some time trying to optimize them for speed and also make them easy to plug in to another project without needing to bring a lot of dependencies along.

Demos of my maze generator algorithms

@sbj42/maze-generator npm package

Update (10/15): I added a maze "puzzle" to Probably Puzzles.
Update (6/17): Added a link to Mr. Floppypants

Generating Shikaku-like puzzles

A very small
Shikaku
in progress.

In my last post, I talked about adding Nonograms to Probably Puzzles.  Now let's talk Shikaku.

In Shikaku your task is to divide a grid into rectangular regions, such that every cell is assigned to a region, every region has exactly one number in it, and that number is the area of the region.

This one turned out to be harder to generate than Hidato, though maybe I haven't found the right algorithm.  The first step was to generate a random division of the grid into rectangles (i.e. to generate the solution).  I didn't find a good algorithm for that, so I made up a mediocre one:

1. Choose a random cell that isn't yet in a rectangle
2. Randomly decide on a major axis (horizontal or vertical)
3. See how far we can stretch from that cell along the major axis (e.g. for horizontal we see how many empty cells there are to the left and right)
4. Choose a random range from that span, with length no greater than 8
5. See how far a rectangle of that width can go along the minor axis
6. Choose a random range from that span, no longer than 8, where the rectangle's area is no bigger than 16
7. If we were able to pick a rectangle bigger than 1x1, then go back to 1 and repeat until all cells are consumed.
8. If not, then pick a random rectangle adjacent to the cell and remove it, then go back to 1.

That last step was the key to speed.  At first I had it just give up and start over if it hit an isolated cell, but it had to do a lot of resetting before it randomly ended up perfectly dividing the grid.  The last step allows it to make a small adjustment and try from there, rather than losing all the work it put into the earlier rectangles.

A grid, divided.

 This gives us a divided grid.  A solution in search of a problem.  So now we choose a random cell from each rectangle to use as the hint.  Here things get a little tricky.  To make sure there is only one unique solution for a given puzzle, we need to go ahead and solve it.

I wrote a brute force solver, but it was taking far too much time for puzzles of moderately-large size.  One trick I used to speed it up was to do a first pass on each hint, seeing how many possible rectangles could go there at the beginning of the puzzle.  Then we sort the hints from fewest to most possible rectangles, ensuring that we try all the "forced" ones first.

I suspect I can make the solver even faster if I resort at each decision in the solver tree, because placing one rectangle frequently reduces the number of options for another, and I find that for most of the Shikaku I'm generating, every rectangle is forced at some point or another, leaving the best kind of brute-force tree ever: a straight line to the solution.

There was another problem.  Too often the generator would put two 2x1 rectangles next to each other, and too often the hint-picker would select kitty-corner hints from those rectangles, giving the puzzle multiple solutions.

An ambiguous Shikaku.

When this happens, the solver takes longer to run, and in the end we have to throw all that work away because we won't accept a puzzle with multiple solutions.  So I threw in a pass to detect when this was happening and auto-correct it (by moving one of the hints).

That left me with a fast-enough generator and a not-so-fast-but-maybe-ok solver to verify the puzzles.

A generated Shikoku-like puzzle.

For the other puzzle types, I have a feature that allows you to adjust the "difficulty" of the puzzle, but so far I don't have that for Shikaku.  I could go by the number of choices made in the solver (which is what I did for Numbrix and Hidato), or I might even be able to use "number of initially-forced regions".  However I would need to speed up the solver before I attempted that.  At the highest grid-size setting, 15x15, it currently takes a while just to generate one puzzle, and if it started getting picky about difficulty it might take way too long.

Solving in progress.

The UI is pretty straightforward.  You click and drag to make rectangles.  It colors a rectangle red if it's invalid (i.e. if it contains no number, or more than one number, or doesn't have the right area for the number it contains).

Probably Puzzles

Generating Nonograms

In my last post, I discussed a generator for Numbrix-like puzzles.  Since then I've extended Probably Puzzles with three more puzzle types.

Hidato is very similar to Numbrix.  The only difference is that the path can move diagonally.  To get that I had to drop the Hamiltonian path generator I scrounged up, and replace it with one that generalizes to accept the diagonal connections.  The new path generator is based on this stackoverflow answer.  Hidato is interesting, the path can cross itself and that seems to lead to some surprises while I'm solving it.

Another puzzle type I added is Nonogram (a.k.a. "Paint by Numbers", "Griddler", and lots of other names).  This one is the most popular of the ones I've done so far (but will certainly drop to second place if I add Sudoku).  In this puzzle you color in the cells of a grid to draw a "picture".  There are numbers along the left and top of the grid which constrain which cells you can fill in.  They tell you how many sequences there are on each row and column, and the lengths of those sequences, but not exactly where those sequences go.

A randomly-generated 9x9 nonogram.

On the one hand, generating these is easier than Numbrix/Hidato.  We don't need to pick a subset of the hints that leaves the puzzle uniquely solvable, we just need to generate a picture.  The hints are determined directly from that.  On the other hand, how do we generate "a picture"?  I considered pulling from a library of images, or scattering various geometric shapes around, but I though it would be better for this particular site if the puzzles were completely random.  In the end, I decided to use simplex noise, which generates basically a bunch of random blobs.  At low difficulty, you get very big blobs, and a high likelihood that some rows or columns are completely determined by the hints.  At the highest difficulty, each cell is independently random, and it ensures that no rows or columns are completely determined.

Nonogram in progress.

For the play interface, I took some advice from my wife, who is an avid nonogram puzzler.  You left click to fill the cells, and right-click to place little dots in the cells which you can use as a reminder to yourself that you've decided that particular cell must be empty.

The third puzzle type I added was Shikaku, which I'll describe in the next post.

Probably Puzzles

Generating Numbrix-like Puzzles

Very tiny
Numbrix-like
 example.

We got the kids interested in a simple kind of logic puzzle called Numbrix.  I showed my son a multiplication table and he became obsessed for a time with filling out more and more tables with bigger and bigger numbers.  Since he seemed to like writing numbers into boxes, we looked around for a puzzle we could print off the web that he could do, and found some kids' Numbrix at this "Math in English" site.

Numbrix is apparently a puzzle published in Parade magazine, developed by Marilyn vos Savant, who had the Guinness Book of World Records "Highest IQ" in the late eighties and also a serendipitous surname.  The goal is to complete a Hamiltonian path on a grid with consecutive numbers.  If you've heard of Hidato, it's like that but you can't move diagonally.  The ones at Math in English seem to be randomly generated, but the ones made by vos Savant are cleverly constructed by hand.

Either way, there are a limited number of Numbrices at both places.  I haven't found anything that generates them "while u wait", so I set about writing some code to do that.

A randomly generated Numbrix-like puzzle.

The first step is to generate a random-ish Hamiltonian path.  With a little googling I came across this "HamPath" library, which does the trick well enough.  Next we need to take away random numbers until we have a puzzle we like.  The puzzle has to have a unique solution, so each time we take away a number we have to check to see if there's another solution, so we need a solver.  The solver just runs all the possibilities and stops early if it finds a second solution.

I wanted it to have a difficulty knob, so we need some kind of measurement of difficulty and then we need to make sure when we take a number away we haven't made it too difficult.  I use two measures of difficulty: "choice count" and "stretch".

Choice count is a measure of how many branches were encountered in the solver.  This vaguely represents the number of ways you can go wrong if you just jot down numbers without thinking or looking ahead.  It's not perfect, because sometimes the path forward is obvious to a human but not to the solver.  It measures a choice at each number, but it would be better if it considered sequences of numbers that link two given cells.  For instance, starting with 1 in the above picture, you need to reach the 6.  There are a few ways to get to the 6, and the solver counts those, but it also counts ridiculous choices that never reach the 6, and I don't think humans would count those as making the puzzle more difficult.

The other measure is stretch.  Stretch is just the maximum distance between two given numbers.  A smaller stretch generally means you need to look ahead a smaller distance to find the next number, and it tends to mean fewer paths between one number and the next.  This one also has the benefit that it speeds up the solver; a very large gap in the numbers causes the solver to consider a large number of possible paths and while that might make an interesting puzzle, we need the solver to finish in a reasonable amount of time.

So we take away numbers, making sure the solution is unique and keeping the difficulty in check, and then stop after a certain number of tries, and now we have a puzzle.  The rest is user interface.

Puzzle in progress.

I tried to keep the interface simple.  The Parade site requires you to type the numbers, which I found annoying.  Instead I wanted to just click to place the next number.  There are some edge cases to consider, however.  The user should be able to start at any number, and go in both directions.  Also, the user needs to be able to remove numbers they've entered.  And when you hit a sequence you've already placed, it should skip over that to set you up to put the next number after the sequence.

As you'll guess from the intro menu, I'm thinking of adding more puzzle types later.

Probably Puzzles

It's EELectric! (Junior)

My kids really wanted to play It's EELectric!, but it's way too hard for them.  So I made a new version with simpler rules and easier levels.  You know, for kids.  Or adults who couldn't stand the punishment of the previous version.

It's EElectric! (Junior)

Music Theory, Sound Science and Digital Audio

This isn't the demo.
This is just a screenshot.

I've been working on some JavaScript music code for a possible future project.  So far I've learned a few interesting things about music theory, sound, and digital audio, and I've reached a milestone so here's Keyboard Demo, a little electronic keyboard demo.

Here are some things I needed to learn to get this far:

Generating audio with FFmpeg

I wanted to be able to play a range of pitches; the standard 88-key piano keyboard runs from C0 to C8, so at least those.  I also wanted to support IE.  There don't seem to be a lot of good baked-in options for that sort of thing, so I knew the lowest common denominator would be to have a bunch of tiny audio files, one for each pitch.

I considered generating all the pitches with MIDI in Anvil Studio, recording with Audacity, and then cutting that audio up into individual files, but that would have taken forever.  After poking around, I discovered that you can generate audio in batch using FFmpeg.

FFmpeg is a very useful tool for audio and video manipulation.  It can transcode audio and video files, change sample- and frame- rates, apply filters, crossfade, and probably hundreds of other tricks I don't even know about.  I didn't know it could generate audio (or video) until I ran into this superuser.com question.  The idea is to use ffmpeg's filtergraph feature as an input, with a command line like this:

ffmpeg -f lavfi -i "<filtergraph string>" <output>

FFmpeg's filtergraph strings are pretty complicated, but making a sine wave is pretty easy:

sine=frequency=440:duration=1

Yup.  That's a sine wave.

A quick python script to generate 88 calls to FFmpeg and now I've got all the pitches I need as individual m4a files (IE won't play ogg, sadly).  There's some interesting music theory in just the choice of frequencies, of which I'll give a taste here:

I generated pitches using equal temperatment, meaning that each pair of adjacent pitches has the same frequency ratio.  As temperaments go, this is the one that all the cool kids are using.  It beats out well and meantone temperament, and generally makes sense if you want to play in multiple keys using the same set of pitches.  Apparently it produces some slightly impure intervals, though I haven't yet compared it to just intionation with my own ear.

Then I based the frequencies around the notion that A4 (the A above middle C) is 440 hertz.  That's called concert pitch, or at least that's what we call it lately, and it's not the only pitch reference out there.  It's nice that it's a round number like that - makes it easier to remember. I've heard it said that music played in different keys can "feel" different, as though simply transposing a piece upward a semitone can turn it from melancholy to hopeful, or something.  But the fact that the exact frequency of a pitch has changes through the ages, and that different ensembles might choose something other than the standard, makes me a little skeptical.  Some people have perfect pitch, of course, but I don't know how that works with respect to different pitch references.  I guess both the feeling-of-the-key and perfect-pitch concepts are relative to contemporary practice.  Anyway, 440.

My ears

The range of frequencies that humans can hear is (very roughly) 20 Hz to 20 kHz, which are nice round numbers that should make you suspicious about the error bars around them.  I don't know what the standard deviation is, but I was surprised when I went to listen to those audio files I made.  I couldn't hear most of the low octave (C1-A1, ~32-55Hz)!  I was sure that I had made a mistake in my script.  I've played an 88-key piano before and while my memory, like my pitch, isn't perfect, I seem to recall being able to hear those low notes.

I ran a highly scientific test using this YouTube video, and sure enough, my ears don't kick in until around 55 Hz.  I didn't get to check on the high end of my range because the dogs started barking and I can't blame them because those are some annoying frequencies up there.

So what's going on?  Well, as you know, a piano doesn't sound like a sine wave.  It produces a crazy mess of noise that goes well beyond the fundamental frequency.  Some of the biggest frequencies in the spectrum are multiples of the fundamental frequency (called "harmonics"), but also the whole audible range is spattered with low-amplitude impurities, and all that stuff together makes the timbre of the note.  That's why those frequency spectrum visualizers are never very satisfying to me - it's all a mess, all the time.  I guess it turns out that when I play C1 I'm not usually hearing the fundamental frequency at all (although more volume can help), but my brain is putting the harmonics and other noice together and I end up with a good approximation.

Adding harmonics

So I went back to those sound files and added some harmonics.  This stackoverflow answer has some numbers in it for relative frequencies of harmonics for a piano, supposedly.  I don't know where they got those numbers from, but I figured it was better than just guessing.  I plugged those into my script and generated some more complicated FFmpeg filtergraphs.  For example, heres A4 with just the first three harmonics:

sine=frequency=440.00000:duration=1[s1];
[s1]volume=volume=3[i1];
sine=frequency=880.00000:duration=1[s2];
[s2]volume=volume=1.197[i2];
sine=frequency=1320.00000:duration=1[s3];
[s3]volume=volume=0.897[i3];
[i1][i2][i3]amix=inputs=3

The actual filtergraph I used was a little more complex: more harmonics, plus I added a delay to each harmonic so that I didn't end up with a sawtooth-like shape.

Now I can hear the low notes.  Plus, the notes are a little less annoying.  It's interesting that adding frequencies other than the one I want to play - impure, dirty, frequencies - ones that don't belong - actually make the sound more real.

Aliasing

Hey wait a minute.  I can hear the low octave now, just fine, but the high octave sounds just terrible.  The notes I hear don't even seem related to the ones I'm asking for.  Luckily, I remembered something from college about the Nyquist frequency (so I guess it wasn't a total waste).  I had picked a sample rate of 16 kHz, because I wanted to save bandwidth and it made the audio files nice and small.  Sure enough, if your sample rate is less than twice some frequency you'd like to hear then sorry, that's just not gonna happen.  Instead you're going to hear a bizzaro mirror-world frequency, and you probably won't like it.

To me, the best analogy is to that old experiment where you look at a fan in a strobe light: at a certain strobe frequency the fan appears to stop spinning, and then if you strobe faster the fan seems to rotate backwards.  It's not a perfect analogy, but basically this is the kind of crazy stuff that happens when analog meets digital.  I probably could have fixed it by filtering out the high frequencies or increasing my sample rate, but these audio files are just for IE users.  I can just drop those pitches from the range.  The IE users probably won't miss them.  What are the non-IE users going to get?

The Web Audio API

Modern browsers have the Web Audio API built-in, and with that I don't need any silly audio files laying about.  I can generate my noises in real time!  It's as simple as this:

    var context = new AudioContext();

    var oscillator = context.createOscillator();

    oscillator.frequency.value = 440;

    oscillator.connect(context.destination);

    oscillator.start();

That's produces a pure A4 sine wave.  Add more oscillators for the harmonics, watch out for aliasing, put a "gain" node in there because the default oscillator is super loud, and we're in business.  There are some cool tricks like amplitude envelopes that I could use to make better or crazier sounds (see this web synthesizer site for instance).  Really, the engineering of synthesizers is an enormously deep rabbit-hole, and I'm tempted to jump in, but this will do for now.

Volume

Another thing about sounds:  Loudness.  I used to think it was mostly about how far up and down the waveform moved.  Nope.  It's also not about the rate-of-change of the waveform at any given point.  If you're thinking about amplitude of sine waves, that's closer, but still not right.

Uh-uh.

Nope.

From what I now understand, it's a combination of the amount of energy in the wave over a certain span of time, and the sensitivity of the human ear to the frequencies involved.  A big spike in the wave means the speaker (for example) has to push a bunch of air molecules really hard, but it has to keep working on that air for a while before it will affect how loud we think the sound is.  Plus, even a very energetic sound may be very quiet if it's near the edges of the range of our hearing, like those low-octave pure-sine-wave notes I couldn't hear.  Loudness is a messy concept, and, when it comes to commercial music, thank goodness for ReplayGain.

In my demo you'll notice that some of the notes are louder than others, due partly to the fact that I didn't vary the oscillator amplitudes as the pitch went up, but also due to a phenomenon represented well by the Fletcher-Munson curves, which show that some frequencies just sound louder than others even when the "sound pressure level" is the same.

The interface

The interface for this demo is pretty simple:  Click and hold a key to play it.  If you want to get fancy, you can hold down shift and click different notes to make chords.  I took a tape ruler to the little electric keyboard I have for some measurements and then used a little dynamic SVG to generate the keys.  There's a little dot on middle C just to get you oriented.  That's about it.  I have some bigger plans for this code, so stay... tuned.  See what I did there?  Tuned?  It's.. oh, ok.  Here's the link again:

Keyboard Demo