Author Archives: tbabb

How the swimming pool got its color

New York Times

NYT Article: EPA science advisors to be fired and replaced with industry-friendly representatives

Once upon a time there was a public pool. Everyone used the pool and enjoyed it, but very soon it became apparent that a few of the pool-goers were relieving themselves in it. The pool quickly turned yellow and smelly.

So the community got together and formed the pool-peeing committee; the goal of which was to cut down on the general amount of pool-peeing that was being done. They would do this by hiring some local experts from the town to measure the pool water regularly, and tell everyone when somebody had taken a leak in it, and to the best of their ability, who was responsible.

Right away, some chronic pool-pissers were caught and yanked from the pool. This was very embarrassing for them, so everyone paid attention when it happened, and so the new standards of pool play more or less caught on and were known by everyone. Soon the water cleared up, and people were able to enjoy the pool again. It wasn’t perfectly clean by any means, but it was much, much better than before, and it was improving every day. This worked pretty well for a long time.

A few people didn’t like the pool-peeing committee. Some didn’t like the idea that someone else could tell them what they could and couldn’t do in a pool. Others were mad because every once in awhile, the committee would accuse them of having peed in the pool when they had only peed a little bit, while Jimmy over there drank a whole 2 liter bottle of Mountain Dew before he swam and let it all out through his bladder, but the committee didn’t catch him and that wasn’t fair. Many had started coming to the pool long after the pool committee had formed, and didn’t understand how smelly the pool had been before the committee came along.

But by far, the people who hated the pool-peeing committee the most were the biggest pool-pissers. The pool-peeing committee was always bothering them, they complained, embarrassing them in front of their friends, and cruelly yanking them out of the pool. All they wanted to do was play in the pool, and didn’t they have a right to do that? So what if a little pee leaks out every now and then. Worse (they argued), if the committee was allowed to yank anyone who peed out of the pool, then pretty soon the pool would be empty and the community center would be bankrupt. Pool-pissers gave a lot of money in entrance fees, they pointed out.

Of course the solution was simply to not pee in the pool (which the rest of the community was able to handle just fine), and to hold it until afterwards, but that really cuts into our pool-playing time, the pool-pissers whined.

So the pool-pissers got together a plan: They would band together and take over the pool-peeing committee— but first they had to convince the other pool-goers that this was a good idea.

“The system is rigged!” the pool-pissers squawked. “The pool-peeing committee gets paid to test the pool! So you see, they all have a stake in the outcome of the pee tests! This is a conflict of interest! They’re on the take!”

A lot of the swimmers began to nod their heads— this sounded really unfair. They started to worry if the pool-peeing committee could be trusted.

“We’re being paid to do our jobs,” said the pool committee. “That’s not a conflict of interest. And we all signed up for this job because we care about having a clean pool. We swim in it too, you know.”

But the swimmers didn’t hear them, or maybe they didn’t care because they were all very worried that something unfair might be happening. And they were right, something very unfair was happening, but it wasn’t what they were thinking of.

“We should kick out these crooked pool experts from the pool committee,” said the pool-pissers. “They don’t know the reality of what it’s like to be a swimmer, like exactly how hard it is to hold your pee. Besides, us swimmers have the biggest stake who stays in the pool. It would be much more fair to put swimmers on the pool committee.”

This sounded reasonable to everyone and soon enough the pool experts were sent away, the pool-testing equipment was thrown out, and the pool committee was re-staffed with “regular swimmers”. Someone noticed that it just so happened that everyone on the new pool committee had been caught peeing in the pool many times, but it was decided that this was okay, because they clearly knew the most about pool-peeing, so it made sense that they were on a committee about pool-peeing. Everyone was very satisfied with this arrangement, and congratulated themselves for having solved the conflict of interest.

Almost immediately, the pool turned bright yellow and smelled like a lot like a subway station, only more so. Nobody was really sure why, or how to fix it. Some swimmers kept saying something about strengthening the pool committee, but it seemed clear that pool committees didn’t work, because we have a pool committee, and look how yellow the pool is.

Many people got very sick, and eventually the community pool lost all of its revenue and had to close after everyone stopped coming to it. The mystery of why the pool turned yellow remains to this very day.

Kwol’s promotion

Chancellor Kwol’s outspoken stance against imperial policy had clearly been irritating the Emperor’s inner circle.

That is why it was a surprise to him when he learned that, in recognition of his initiative and heartfelt vigor, he had been given the position of Imperial Right Hand. In the interest of fostering fresh ideas, the Emperor would sometimes reward an official who took a vocal stand in service of his district. The Emperor wishes every voice in the galaxy to be heard (so the announcement went), and for having the courage and leadership to give voice to such unorthodox ideas, the Chancellor was to receive a conspicuous promotion.

It was also strange that the Emperor’s inner circle had suddenly become polite and deferring almost overnight, and their former frostiness and grinding skepticism had disappeared. The Chancellor had chalked it up instinctual deference to his newfound power, and meekness for having guessed wrong about the Emperor’s opinion of him. But had he been a more astute man, he might have noticed a patronizing note in their mannerisms.

Something was unmistakably amiss in the first briefing session with the Imperial Third Hand, a thin and over-composed man named Cohrer. Kwol was instructed that it was customary for a man in his new position to affirm everything the Emperor said. It was not that he wasn’t allowed to express dissent, per se (the Emperor valued dissent greatly, he was reassured), it was just a customary way of speaking, like saying “please” and “thank you”, or “good day”, or “bless you” when a person sneezes. When the Emperor speaks to you or asks a question, it is polite to affirm it.

And then you can discuss the matter at hand?

The Third Hand’s face concealed a flash of both irritation and patronizing patience.
No. You must never volunteer that. That would be an irreconcilable mistake.

The phrase “irreconcilable mistake” had a kind of compressed cadence which identified it as a jargon code word.

“And to fail to affirm the Emperor when it is your turn to speak, by the way, that would also be an irreconcilable mistake.”

When he asked how he could perform his duties as hand to the Emperor if he couldn’t express a position, much less a contrary one, he was told that he would come to learn that later. First, he must master and internalize the proper etiquette.

“Things are different in the Palace. You are a smart man and a deft politician. I expect you should have no trouble picking up and adapting to new social decorum.”

The rest of the day was spent drilling this into the Chancellor, with the Third Hand role playing as the Emperor, and the Master of the Hand’s Affairs instructing him on his every blunder.

And so it was, after a week of conditioning, that Kwol was allowed to meet with the Emperor and accompany him on his outing to Aluvia.

Upon their arrival it slowly dawned on him what, exactly, Aluvia was.

The landing green was immaculately manicured; every blade of grass cut to the imperial standard 20mm, at the edges of the walkway tapered to an even 60 degree angle. Everywhere lush plants of all kinds from across the galaxy intermingling in regular arrangements, nowhere a brown leaf. Every tree tied off to regulation height, clipped flawlessly to 1-meter poly form, and planted precisely into its place within an elaborate Khanner-Badorff pattern.

What was alarming was that the attention to detail did not diminish as his gaze wandered out into the fields, and moreover not beyond the fields, up into the hills and further to the mountains— the Khanner-Badorff pattern was still visible, distantly, in what he might have called forests.

Cohrer had told him that there were no droids on Aluvia. In fact, since his appointment as Hand, he had not seen a single droid at all.

In the context of Aluvia, Kwol’s mind reeled. Not only was the entire planet a manicured garden, but it was maintained, every inch of it, by people.

The industry that would have to support them— just the gardeners alone— would fill another planet or two. Billions of families would live out their lives, in service of polishing tiny plot after tiny plot of the planet, keeping it perfect, all on the off-chance (which would certainly never come) that one day the Emperor might decide to fly in and gaze upon it.

Of course it could not be the immaculate sight of all those roses and fecundias that the Emperor cared about— he wouldn’t ever see them, after all, and if he wanted a perfectly manicured planet, he could have one staffed by droids at a far less mindbogglingly gargantuan cost. No, it was the knowledge that his subjects were obediently tending it for him that mattered. The planet was as much a garden of keepers as it was of plants.

And it was clear that this was the same reason for the absence of droids elsewhere in the upper echelons of the Empire— though droids were cheaper and more reliable, the human help was a far more conspicuous luxury.

“Good day for chints, isn’t it?”

Chints is a complex game that takes days to play out. The rules eluded Kwol, but he understood that the main appeal was that it revolved around the suffering of something small and helpless.

It looked like it was going to rain, but the Chancellor caught himself from pointing that out. The Emperor is here to play chints, Cohrer’s voice admonished him. That is why you are holding his chintsing sticks. That is why the hands dressed him for chints this morning. That is why you flew out to this planet today. He is clearly in the mood for chints. Are you suggesting that he do something else?

That would be an irreconcilable mistake.

“Lovely day for chints, Lord.”

Kwol had planned to keep an eye out for the moment in conversation, sometime after the two men had gotten acquainted to each other, when he could begin to edge past the oppressive etiquette and elucidate his ideas to the Emperor.

With every passing moment of the game, that opportunity seemed further away. The Emperor spoke only in brief commands to him, with a superficial civility completely devoid of interest or compassion— requests for this stick or that one, or to re-bind the cuckmouse please, or that’s enough thank you (which he quickly learned was jargon for “stand over there”). The intervening time was mostly silent, and when the Emperor broke it with a self-congratulatory “That’ll do” or a huff about his swing, his entourage would affirm him in a melodic round robin, and Kwol was humiliated to observe himself participating in it.

Kwol was embarrassed, too, to realize that he had failed to notice the difference between the entourage and the inner circle. The Emperor would disappear behind heavy oaken doors and argue with the inner circle, but with the entourage he almost acted as though he were in perfect solitude— Kwol now perceived that the illusion was never to be disturbed. It was almost as if the Emperor had created a small flock of perfectly silent hands to carry his things and tend to the objects surrounding him.

Ugh, it was right there in the job title. So stupid.

The night before he had planned to put in his resignation, the Chancellor dreamt that he was the cuckmouse, and he was placing himself on the tee while the Emperor loomed overhead and practiced his swing.

“This is a lifetime position,” Cohrer told him.

“Yes, I understand that. But I would like to decline it.”

“This is a lifetime position”, Cohrer reiterated. “The position and your lifetime are to end coincidentally.”

And so it became crushingly clear to the Chancellor— after all, he had a lifetime to observe and think about it— the nature of his role. Aluvia was a conspicuous display of subservience, and so was he.



It begins working when the scientist is not expecting it. It records eight silent seconds of him in his home, interacting with his wife.

He is laughing and joking with her. We can’t see her at first; only that he is happy, and talking to some unseen person. Then she dips into the recording area and says something. He listens. They joke some more. Then he steals a kiss. The recording ends.

The initial playback happens in real-time, as expected, and in glowing, ethereal miniature on the scientist’s desk. What the scientist did not anticipate is that his particular use of quantum feedback would create infinitely many more copies of this same image. They repeat themselves fractally, in space and in time. The real-time image was only a base case: A few moments later, a double-size copy will play back at the same location. Then a four half-sized copies, two of them mirror images. Then eight sixteen-sized copies, some mirror images, some playing backwards, arranged in a cube.

The process is out of control. It is not harmful; it is only made of light. But once it has started, it will play out until the end. It is now a natural phenomenon.

These images appear all over the region near the scientist’s lab; they become like a weather phenomenon. Some of these events are very short-lived and localized. Others play out in the immensity of the sky; partially submerged in mountains, illuminating cloud banks, silently stretching high out of the stratosphere into low orbit, glowing against the twilight.

The fractal nature of these events give them a kind of poetic symmetry; a beginning, a middle, and an end; sometimes with a bright, triumphant climax— maybe an unusually large, single, clear apparition, or perhaps a rapid, accelerating explosion of symmetrical, replicating copies. When in sight of a specter, human activity tends to pause.

As time progresses, the phenomenon becomes less localized. As it evolves, it becomes more warped and complex— the copies are no longer simple reflections and translations of the original; they are bent into strange, warping coordinate systems. Some are unrecognizable. Some copies have their colors compressed into shimmering, pure spectral slices of the rainbow.

While many forms are too complex to analyze, some statistical predictions can be made about the frequency and location of appearances. Overall, it can be seen that the specters are getting larger and less frequent. After about one and a half years, humanity will have to wait five years for a single appearance. Then 10 years. Then 20. After a sufficient time, the time between occurrences will be longer than the predicted age of the universe, and the event will be over.

This eight seconds of beauty becomes first overfamiliar, and then an icon. Some come to believe it is the single most beautiful thing ever created by man.

On Brexit and crumbling structures

At high risk of adding to the prattle, two thoughts on Brexit:

The failure of political engineering, IMO, happened long before the vote or even the campaign, in that it was possible for Britain to secede by a simple majority instead of a supermajority (e.g. a vote of two thirds). Imagine if a U.S. state could secede from the union with a vote of 51%? All that is necessary is a slight, momentary imbalance in public favor— mere noise on a neutral signal— and permanent irreversible damage is done. The States’ founding fathers evidently recognized this danger when requiring a two thirds supermajority for constitutional amendments, and the result is a more failure resistant system. Perhaps this will be a Tacoma Narrows-esque lesson for the world of government: That we should not build large structures which can be overturned in a gentle breeze.

Second, when it comes to public discourse, it seems that there are no longer any “adults in the room”. I’d guess this is both because earnest, measured, and thorough journalism isn’t really a viable business model anymore, and because in an un-curated medium such as the internet(1)Says I, of my own soapbox; irony duly noted, sensible ideas can be easily out-shouted, and simple, “truthy” but wrong ideas can far outbreed the long-gestating ones that are more faithful to a complex reality.

I used to think the collapse of traditional media was a natural consequence of the disruptive power of the internet, that the Darwinistic forces of the free market would naturally find and settle on an updated business model, and that the increased democratization of information and ideas was unequivocally a good thing. Now I am not so sure.

Do not misread this to say that any ideas— even the very bad ones— should be silenced. I very much think it’s a good thing that one person can speak, for free, and be heard by millions. But serious thought should be given to how we, as a society, prioritize and disseminate our information. How do we keep the “snack food” information off to the side, and the “hearty meal” in the center? How do we make deep research and careful insight rewarding for content creators, the pool of which is as large as ever? How do we make it rewarding for people to consume it? And how do we do it all without adding friction to the wondrous speed and freedom of the internet? Decades down the line, the health of our civilization could depend on how successfully we’ve solved that problem, if we even make a concerted effort to solve it at all.

The solution to the former seems obvious in principle, though I don’t know enough about European politics and government to have any idea whether it is practically realistic, especially now. I have no clue what the solution to the latter is, and I don’t think it will come by itself without earnest, directed effort.

Footnotes   [ + ]

1. Says I, of my own soapbox; irony duly noted

12 trail runs under 10 miles in SF and the East Bay

I can’t overemphasize the rewards of starting a running club. About a year ago, I started one nearly by accident— a routine of solo running morphed into a weekly social event. Every Saturday or Sunday five or six of us gather in the mid morning, run five to ten miles, and then have brunch together afterwards. The way I see it, this is a quadruple refill for the soul; a combination of four things that have immense positive impact on your mood:

  • You’re getting exercise
  • You’re being social
  • You’re outside getting sunshine and nature
  • You’re accomplishing something

The delicious food (and, frequently, drinks) at the end of it don’t hurt either.

With that plug out of the way, I offer below a catalog of twelve routes we’ve found over the last year. Continue reading

Infinitely Aged Christmas Pudding

Astronaut reaching

While somewhat difficult to make, this recipe for extra-long aged Christmas pudding is rich in flavor!

  • 1 lb raisins, currants, chopped apples, prunes
  • 1 oz candied peel
  • 4 tbsp brandy
  • 4 oz bread crumbs
  • 2 oz flour
  • 1 oz chopped almonds
  • 2 solar masses dark matter
  • 1 3/4 Yottajoules of negative energy
  • 1/8 Earth-mass of lead (or other heavy elements) for radiation shielding. If you can find a small moon, it may be best to hollow out the core.
  • 1 Penrose drive-capable spacecraft
  • Provisions for 6 trillion years of space travel
  • 1 handgun loaded with two bullets
  1. Stoke the oven with the dark matter, set it to “closed timelike curve”, then turn it on and allow it to preheat. If anything strange happens at this point, don’t worry about it. It’s fine.
  2. Mix the fruits, peel, and other stuff into a bowl or whatever.
  3. Once the oven is preheated, retreat to a safe distance of about 1.5 AU. Apply the negative energy.
  4. Wait for the gamma radiation to die down, emerge from your shield-moon, and board the ship. Don’t forget your provisions and gun!
  5. Boost toward the gaping black hole where your oven used to be.
  6. Approach the ergosphere, taking care to maintain infinitesimal separation from the closed timelike curve. When you’re close enough, chuck the pudding into it. Be careful not to run into any of the infinite copies of yourself now orbiting nearby.
  7. Draw your gun and fire on the copy of yourself in the orbit just below you. The copy in the orbit just above you should be doing the same.
  8. Your infinity of guns and infinity of bullets should take care of the infinity of self-copies. With any luck, you’ll be the guy at the top. Good job!
  9. Descend toward the event horizon, and nab the pudding on your way down. Check that it’s an eigenstate of the universal wavefunction, and that the fruit has firmed up well. Insert a toothpick; it should come out clean of any protons, which will have completely decayed after a literal eternity trapped in a repeating loop.
  10. Pass through the singularity and emerge from the pre-heated oven with the pudding. Draw your gun again and fire on your past self, who will have just switched the oven on.
  11. Serve and enjoy. There will be no need to ignite the pudding, as it will probably already be on fire from all the proton decay.

Second Order FizzBuzz

The FizzBuzz problem requires the construction of a well-known list of interleaved numbers and nonsense words. The list has been studied extensively, and recent work at CERN has constrained its utility to less than 10-16 of a mosquito’s left nut. In modern times, the task of its production is typically relegated to computer programs. It is an open question why some institutions continue to use FizzBuzz as a test of programming ability, given universal knowledge of the test and the wide availability of solutions on Google and StackOverflow.

Programs that solve FizzBuzz are typically constructed by hand in an ad-hoc manner, often by inexperienced computer scientists, and entail a hazard of boredom-induced mortality. Below is presented an algorithmic solution to the second-order problem of generating such programs. In general, we define a class of nth-order FizzBuzz problems which require the production of a program which solves the (n-1)th-order FizzBuzz problem.

We use a variation on the stacksort algorithm due to Munroe (2013) and first implemented by Koberger (2013), which mimics a technique commonly employed in the wild. Here, the Continue reading

How a Kalman filter works, in pictures

I have to tell you about the Kalman filter, because what it does is pretty damn amazing.

Surprisingly few software engineers and scientists seem to know about it, and that makes me sad because it is such a general and powerful tool for combining information in the presence of uncertainty. At times its ability to extract accurate information seems almost magical— and if it sounds like I’m talking this up too much, then take a look at this previously posted video where I demonstrate a Kalman filter figuring out the orientation of a free-floating body by looking at its velocity. Totally neat!

What is it?

You can use a Kalman filter in any place where you have uncertain information about some dynamic system, and you can make an educated guess about what the system is going to do next. Even if messy reality comes along and interferes with the clean motion you guessed about, the Kalman filter will often do a very good job of figuring out what actually happened. And it can take advantage of correlations between crazy phenomena that you maybe wouldn’t have thought to exploit!

Kalman filters are ideal for systems which are continuously changing. They have the Continue reading

How to pronounce hexadecimal

This bit from HBO’s Silicon Valley cracked me up:

Some kid is pitching his revolutionary startup idea to entrepreneur Elrich Bachman:

Kid: Here it is: Bit… soup. It’s like alphabet soup, BUT… it’s ones and zeros instead of letters.
Bachman: {silence}
Kid: ‘Cause it’s binary? You know, binary’s just ones and zeroes.
Bachman: Yeah, I know what binary is. Jesus Christ, I memorized the hexadecimal times tables when I was fourteen writing machine code. Okay? Ask me what nine times F is. It’s fleventy-five. I don’t need you to tell me what binary is.

We infer that “fleventy-five” is a hexadecimal number, commonly used in coding; presumably it’s 0xF5 (which is not 0x9 times 0xF, as it happens). But instead of saying “eff-five” for the byte 0xF5, Bachman has come up with some kind of novel system for the English-ification of hex digits.

He’s on to something. We have ordinary English words for decimal numbers, with names based on the digits and their place-value. “Seventy” is the word for two-digit chunks starting with the digit seven, for example. It might appear in the number “seventy-three”, or “five hundred seventy-one thousand”; both numbers with a 7 digit in an appropriate place. “Fleventy”, then, would be the number for two-digit chunks starting with F.

Hex only adds more kinds of digits (the symbols A through F). Could we follow Bachman’s lead and add more number-names for the extra digit symbols, and pronounce hex just like a decimal number? Could we have a system that attains the unwieldiness and syllable count of spoken English numbers, with all the respectable seriousness of saying the word “fleventy”?

I’m glad you asked. This has never been done before(1)probably, but fear not; here are the official new number-words for hexadecimal. You may start using them immediately:

Hex Place value  Word
0xA0 “Atta”
0xB0 “Bibbity”
0xC0 “City”
0xD0 “Dickety”
0xE0 “Ebbity”
0xF0 “Fleventy”

I think it would help to solidify this with some examples:

0xB3 “bibbity-three”
0xF5 “fleventy-five”
0xE4 “ebbity-four”
0xA7 “atta-seven”
0xC5 “city-five”
0xDB “dickety-bee”

Higher place values

But really, we must go further. What about numbers larger than a byte? We have the words “hundred” and “thousand” for decimal place values higher than ten, so why not something for hex place values higher than 0x10? Say, for multiples of 0x100?

For this, I propose “bitey”.

Resulting in:

0xDAF1 “dickety-A bitey fleventy-one”
0xE137 “ebbity-one bitey thirty-seven”
0xA0C9 “atta-bitey city-nine”
0xBBBB “bibbity-bee bitey bibbity-bee”

Naturally, we could make yet larger numbers by devising some names for larger place values, and combine them with intermediate values, in the same way that we compose decimal numbers like “thirty thousand” or “six hundred fifty one million”. As in English, place value names could be dropped if you’re feeling brief. We’ll also need hex-digit words for the “teens”.

Try it

I’ll leave you to explore the new naming system with this toy:

Say place values

How we’ve conversed about hex without a system like this is beyond me. You’re welcome.

Footnotes   [ + ]

1. probably

Improving IMU attitude estimates with velocity data

This was last week’s project: Building a Kalman filter-based IMU.

IMUs (inertial measurement units) are clever little devices which try to estimate an object’s absolute orientation (and sometimes its position) by examining the forces on the object.

Hobby drone and computer input IMUs generally look at acceleration data (which informs where “down” is), compass data (which points toward north in 3D space), and rate gyro data (which tells the axis and speed of spin). “Down” and “north” in combination can give a pretty accurate constraint on orientation, but unfortunately if there are any lateral forces (wind; turning), they will get mixed in with “down” and distort the estimate. Kalman filters use matrix math to make good use of the gyro data to correct for this. However, a constantly-accelerating drone could still be fooled about where down is.

I’ve tried here to find out whether we can try to model the drone’s translation and take this into account when estimating the orientation. It turns out that even relatively poor and infrequent data about velocity can constrain acceleration— and thus “down”— quite well. The difference in the quality of the estimate is plainly visible.

This is all done by mathematically simulating a 3D moving object using ordinary dynamics and battering it with Gaussian random forces, and then predicting what data a noisy sensor might return. A predictor algorithm using a Kalman filter (which has no knowledge about the original state) attempts to recover the true state to the best of its ability. The truth is rendered white here, and the estimate in red.

At the end you can see the same algorithm running on actual sensors. The real thing doesn’t use GPS yet, but the prediction is still pretty decent! (There is not that much sustained acceleration to throw it off in this video).

I’ve done some other cool things with the code which perhaps I’ll write up in the future: Among them are a process noise model based on Gaussian random walks, and a nice extension of it to quaternions (which has dramatic impact on the quality of the estimate!) I also make use of automatic differentiation, which makes a linearized extended Kalman filter (EKF) particularly easy and robust to implement.