Ruby’s magic underscore

I discovered today that Ruby treats underscores a little bit differently when it comes to variable names.

Suppose, for the sake of argument, that we have a set of data that looks like this:

people = {
  "Alice" => ["green", ""],
  "Bob"   => ["brown", ""]

and we want to ignore eye colour and age and convert each element to a simple array containing name and email address. We might do this: { |name, fields| [name, fields.last] }

but it’s not very clear what fields.last is. Ruby has some fairly powerful destructuring assignment, so we can reach inside the structure with parentheses to make things more explicit: { |name, (eye_color, email)| [name, email] }

That’s better, but that unused variable is noise in this context. It doesn’t add to our understanding of what the code does.

In many languages (Haskell, Go, Clojure, and plenty more), it’s conventional to use an underscore to represent an unused variable. This is also recommended by Ruby style guides: 1, 2.

With that in mind, we can write: { |name, (_, email)| [name, email] }

and be fairly satisfied. But what if we’ve more elements to ignore?

people = {
  "Alice" => ["green", 34, ""],
  "Bob"   => ["brown", 27, ""]

No problem. Just re-use the underscore: { |name, (_, _, email)| [name, email] }

You can’t do it with any variable, though, at least in Ruby 1.9. It only works with variables that are called _: { |name, (x, x, email)| [name, email] }
# SyntaxError: (eval):2: duplicated argument name

The reason for this is found in Ruby’s parser, in shadowing_lvar_gen. All the normal checks for duplication are skipped iff the variable name consists of exactly one underscore.

Multiple underscores work in Ruby 1.8, too, but for a different reason: Ruby 1.8 doesn’t care about duplicated arguments in block parameters.

Meanwhile, neither 1.8 nor 1.9 care about duplication in multiple assignments:

a, a, b = 1, 2, 3
a, a, b = [1, 2, 3]

Both forms work without error (although I wouldn’t recommend relying on the value of a).

(Thanks to Ben for finding the pertinent line in parse.y.)

The East Coast Delay Repay scam

One of the tiring things in life is the way that everyone is trying to pull a fast one. East Coast Main Line operates a Delay Repay scheme that promises to repay ‘100% of the cost of a return ticket’ if you’re delayed by more than two hours. As you can probably guess, this isn’t exactly the case.

The Delay Repay information page promises:

Customers delayed on East Coast services for 120 minutes or longer will receive compensation of at least 100% of the cost of a single ticket or at least 100% of the cost of a return ticket (i.e. both ways, not just one way).

Thus, when my train to Edinburgh was cancelled at Newcastle due to extreme weather, and I arrived about 14 hours late, I was looking forward to getting £105.65 back from East Coast. I filled in the form and sent it off.

A week or so later, I received a refund for £58.15, i.e. only for the outbound leg of the journey.

But I’d bought a return ticket, I thought. The website described it as a return journey. The confirmation email described it as a return journey:

Ticket type: EC First Advance Valid on chosen train only. Non refundable. Changeable for a fee. No access to 1st Class Lounge.
Route: East Coast & other TOC connecting services.
Return journey: 1 Jul 2012
departs Edinburgh at 14:00 travel by Train service provider East Coast to station London Kings Cross arrives 18:43

I wrote to them. They didn’t reply. So I phoned them, and they explained the scam to me: If you buy a ‘return’ ticket via East Coast’s website, they tell you that it’s a return journey, but they actually sell you two single tickets, ensuring that they’ll never actually have to pay out on the return ticket repayment promise.

Oh, and they don’t repay you in cash, but in Rail Travel Vouchers in blocks of £25, which you can’t use anywhere except in a station or travel agent, and which you can’t get change from. But that’s another issue.

A life less soundbitey

I’m not dead. I’m just not really using Twitter any more. That old reflex action—oh, I’ve got a picosecond idle, better catch up on Twitter—has gone. It’s not that hard to break a habit.

I used to listen to the radio every morning to wake up—to the Today Programme on Radio 4, in fact. Waking up to aggressive journalists interviewing dissembling, vapid politicians often just made me angry, but I didn’t understand it until I left my last job and stopped setting the radio alarm, and realised that ignorance—not knowing what stupid idea the Home Office had come out with before I’d even had my morning coffee—had a distinct improvement on my mood.

If you can’t have any effect on politics—and we manifestly can’t—then why worry? Paradoxically, my spell working for the government just made me even more convinced that almost nothing can be done to fix the system.

I know that sounds cynical, but what can you do about it? Vote? There’s not much choice, your vote won’t have an effect unless you live in a marginal constituency, and there’s no guarantee that the party elected will actually follow its manifesto. It’s not just the politicians, either, but the whole rotten towering edifice.

I don’t think that meaningful political change can be effected in the UK without some massive external event—something on the scale of the Black Death. And I don’t think that’s much to look forward to.

So I decided to forget about it. I’ve made the excellent life choices of being white, male, and privately educated, so I’ll be all right as long as I’m careful not to get sick or poor.

But Twitter’s full of political outrage. That’s not surprising: politicians do astoundingly awful things for the cheapest and most banal reasons. However, I just can’t be outraged all the time. I know they’re egregious shits doing egregiously shitty things, but I don’t need a shot of righteous outrage a hundred times a day. It’s as useful as an erection on a neutered dog.

It wasn’t just the politics, though.

Maybe I’m just getting on in years, but I prefer the old internet. The decentralised one, of interoperable systems and open protocols. Not the endless venture-capital-funded attempts to re-intermediate all human communication through a few central portals that we see these days.

Twitter was useful. Twitter still is useful. But it’s not what it was: look at this post on the developer blog. See that quadrant chart! The old Twitter was a message bus for the internet. That’s dying in favour of something far more anodyne and nakedly commercial, and maybe just a little bit evil.

But …! You may say, Twitter has to make money! But therein lies the problem. That’s only the case because we’re funnelling everything through a central service. You can get email from any number of providers for free, or as part of a hosting or internet access service, or you can set up your own server and bear the running costs yourself. The same is true of websites. I don’t have to make money off this website, not does anyone else. I pay for the running costs.

There isn’t a real decentralised alternative to Twitter yet, unfortunately. There are several contenders, but none of them has achieved any significant penetration, and Twitter’s rules make it hard for anyone else to try.

I didn’t flounce off Twitter, though. I just stopped one day. I closed the browser tab, removed the client from my phone’s screen, and just stopped thinking about it after a while. Whenever I go back now, I scroll down a bit, get bored, and close it. The old compulsion has been replaced with ennui.

Instead, I’ve sought out the RSS/Atom feeds of everyone I know and added them to my reader, though 140-character writing seems to have quieted many a blogger.

It may change, but, for now, I’m quite enjoying the life less soundbitey.

Chana dal

I made a particularly good chana dal the other day. As requested by James, here’s the recipe.

Chana dal on rice

Serves 4 to 5 as a main course.

400g chana dal (dried)
1 large onion (or equivalent), finely chopped
4 or 5 garlic cloves, chopped
2 tsp cumin seeds, lightly crushed
1 tsp fennel seeds, lightly crushed
3 cm ginger root, peeled and roughly sliced
1 tsp turmeric powder
4 cloves
2 black cardamom pods
1 star anise
1 tsp salt (approx.; to taste)
1 tsp chilli sauce
4 tsp gram flour
½ tsp garam masala

You’ll also need oil (I use groundnut oil) and water, and a deep pan with a lid.

The chilli sauce mentioned is my own, made by blending together about ten scotch bonnet peppers, a clove of garlic, half a teaspoon of salt, and white wine vinegar. It keeps well in the fridge—better than raw chillies, anyway.

Wash the dal and soak for several hours. Rinse again and drain.

Heat enough oil to cover the base of pan, and add cumin and fennel seeds. Stir for a few seconds, then add onions and garlic. Fry until the onion is soft, then set aside in a bowl for later.

Put dal, ginger slices, turmeric, cloves, cardamom, and star anise in pan with enough water to cover with a cm or so to spare. Bring to boil, then cover and simmer until dal is soft. Add extra water as required. Add salt near the end.

Remove the whole spices (I find chopsticks helpful).

Mix gram flour with a little water, and whisk until smooth. Add to dal mixture, stir, and cook for a few minutes.

Add the onion mixture (set aside earlier) to the dal, and stir in. At this point, you may need to add a little water to get the constituency you want.

Add chilli sauce, garam masala, and any additional salt to taste. Grind some pepper over the final dish.

Mail Cleaner: a Chrome extension to mitigate the effects of accidental exposure to the Daily Mail

I came across a Safari extension by Andy Beaumont that mitigates the effects of accidentally following a link to a Daily Mail page by replacing the page with the same content filtered through Instapaper’s page cleaner. This eliminates comments, adverts, the awful misogyny of the Sidebar of Shame, and saves you from the self-hatred that comes from being lured into following links to other parts of the digital rag. I decided to make a similar extension for a browser I actually use: Chrome.

It’s all part of my ongoing campaign of attrition against the toxic influence on society that is the Daily Mail.

You can install the extension via the Chrome Web Store—I had to pay $5 for the privilege of being able to put extensions on there, so I hope you appreciate it!

The source code is on GitHub in case you want to improve it, modify it, or do anything else you wish.

Ephemeral computing

How much does the physical object that is your computer actually matter?

I went to the Clojure Dojo on Tuesday evening, opened my laptop, and … nothing. After unscrewing the bottom and pulling out various components in turn (bless ThinkPads and their easy maintainability!) I established that it was the (solid state) disk drive. It wan’t just dead; it was so dead that just having the thing connected caused the BIOS to hang.

It was unfortunate, really, especially since I’d spent the preceding afternoon setting up a good Clojure editor + REPL environment. Luckily, I’d pushed those configuration changes to a Git repository just before leaving.

It seems that dying on sleep is a known problem with this model of SSD (an OCZ Vertex2). I wish I’d known earlier. I’d been led to believe that the principle failure mode of SSDs was dead cells, and it may well be, but this one apparently died because of buggy firmware.

What may surprise you is that I don’t back up my laptop. At least, not in the sense most people understand it. I don’t copy it all to a safe place: most of it doesn’t matter. Almost everything on there exists elsewhere, mostly on the internet. The rest is just junk, resting there temporarily to serve a specific purpose at a moment in time.

My email is hosted remotely. All my work—and the configuration of my working environment—is version controlled using Git, and pushed to public or private repositories every time I make a change. Even the initial setup of the computer—installing software and configuring the environment—is done by a script that’s saved elsewhere. However, that was the theory. Tuesday’s failure was the first time I really put it to the test.

I had to tweak the slightly out-of-date setup script a bit, but it worked. I can take a standard installation of the operating system (Xubuntu in my case), run a script, and have a full operating environment ready to go. It’s particularly easy with Linux as all the software installation can be scripted, too: there’s no need ever to see a wizard or a download page. I’ve made the script idempotent now, so that I can modify and run it over and over when I want to make changes to my working environment. If I’m disciplined, it should never really be out of date again. (You could always use Puppet or Chef to do this. I think that’s probably overkill.)

What I now know for sure is that I can reproduce my working environment on a fresh laptop with almost no interaction in about an hour. The nice thing about this is that the hardware is almost unimportant. With the addition of disk encryption, I don’t really have to worry about the consequences of loss, theft, damage, or failure beyond the obvious financial calculation.

Solid state drives are a huge improvement over platters in terms of access time, but they seem to be quite prone to failure. It’s surprising, really, that a rapidly spinning disc of rust should be more reliable, but I suppose it’s just a more mature technology.

Are the advantages of an SSD worth it? Yes, probably. All technology can fail, and frequently does. If your computer is just an ephemeral incarnation of a reproducible working environment, it doesn’t matter so much.

A letter from IOC-occupied London

It was the best of times

Yesterday evening, I finally watched the Olympic opening ceremony, about a day later than everyone else did. At the risk of tarnishing my reputation for unmitigated cynicism, it was actually excellent.

I’ve been really cynical about the Olympics, and I still am. I doubt most of the claims of economic benefit from all the money being spent on it—and what money! You could run the entire nation of Rwanda, all eleven million of them, for a year and a half on what’s being spent on this two-month amateur sporting contest. I also wonder why it’s necessary to give such wide-ranging and unique legal favour to an organisation with a history of corruption. I think you probably can run a track-and-field meet without restrictions on freedom of movement, speech, and assembly, or exclusive traffic lanes for the nomenklatura.

Nonetheless, I can’t pretend it’s not happening. The transport disruption has made that obvious enough. I did think about leaving London for the period, but, despite not being tied here by work, here I still am.

So I thought I should see what everyone else was talking about. By watching the opening ceremony online, I was able to choose a version without the commentary, which I’ve heard was rather annoying and vacuous. (Like most commentary, then, really.)

I thought it was excellent. It was slightly eccentric, and spectacular in a very different way from the slightly creepy massed ranks of Beijing’s ceremony. Better still, it seems to have annoyed exactly the right kind of person.

If you’re the kind of person who lost his last job after being photographed at a Nazi-themed party, I suppose you might see all those non-Aryans and decry it in outrage as ‘leftie multi-cultural crap’. The Daily Mail saw politically-correct reds under the hospital bed (FreezePage), and resorted to outright racism:

This was supposed to be a representation of modern life in England but it is likely to be a challenge for the organisers to find an educated white middle-aged mother and black father living together with a happy family in such a set-up.

You couldn’t really ask for a better recommendation that the opprobrium of bigots. To me, it just looked like London: the people that I see every day, who come—yes!—in more than one colour; the music (often rather too loud and much too bassy!) that comes from cars driving past. It was inclusive, and, at a time when politicians rely ever more on petty xenophobia to distract the populace from the causes of domestic misery, that’s important.

The NHS spot seems to have particularly upset not just the Daily Mail, but a certain type of American (the kind that sees universal healthcare as evil Socialism) too. I’m rather glad that I didn’t have to decide whether I could afford to go to hospital after being concussed. I’m glad that I could choose to leave my job without having to worry about whether I’d be covered by health insurance. I know the NHS isn’t perfect, and I know people who’ve had bad experiences, but I also know that getting seriously ill or being shot at the cinema doesn’t lead to destitution, and that is something to be proud of. As this government, like its predecessor, appears to be confusing improving the NHS with selling off the profitable parts to their donors (just wait for the current crop of ministers to pop up on the boards of those companies in a few years’ time!), it’s timely to be reminded of the enormous good that comes from universal healthcare.

It even managed to sneak in the first ever lesbian kiss on Saudi television.

I’ve never really bought into this whole thing of being proud of your country. I even went so far as to answer ‘human’ to the question of ‘what is your national identity’ on the last census. I mean, the whole notion of your country is ridiculously arbitrary, based on where you were born, or who your parents were—the exact opposite of the meritocratic ideal, in fact. That doesn’t mean that I don’t have an attachment to the places I choose to live, or the people with whom I share them, or the cultures that exist there—and I do mean cultures in the plural. I do. But those ties are ones of choice, not of passport.

I thought Danny Boyle’s opening ceremony did a great job of describing and celebrating a city and a country in which I would like to imagine that I live.

It was the worst of times

And I say ‘would like to imagine’, because whilst the world was watching a cyclist open the ceremony and winged cyclists swoop around the stadium, hundreds of police were arresting 182 cyclists for the crime of, well, cycling.

I’ve never participated in Critical Mass (although I did once happen to cross Waterloo bridge at the same time). I’m not entirely sure how I feel about it: on one side, it does seem unnecessarily antagonistic towards other road users.

On the other side, though, it raises an interesting philosophical point: if there are too many cars on the road, we accept the congestion as inevitable. If there are too many bicycles, it’s ‘blocking traffic’. The question that Critical Mass poses is: are cyclists not just traffic too? Even if not, the disruption to motorised traffic is ephemeral: the swarm blocks a junction, then passes through, delaying other traffic by no more than a few minutes. Realistically, it’s not a big hold-up to drivers, but I know that the very act of driving, and of having the capacity for speed, inculcates the need to exercise that capacity. Rare is the driver who, presented with a laggardly cyclist, does not transform into Mr Toad.

Critical Mass takes place on the last Friday of every month, so there was nothing special about the day. That some people would ride in the Zil lanes was inevitable, but the penalty for doing so is a fine, not imprisonment. In legal terms, it’s a customary procession. It’s not a protest. If the police had just let them ride around, they’d have ridden around for a bit, then dispersed.

But no. The authorities had to Make A Point. What is the point? That ordinary civil liberties are suspended so some people can run around a track?

I hope we get our city back after the occupying forces leave.

Retrofitting furigana to browsers

I’ve written a small shim script that adds support for <ruby> markup to browsers that don’t support it natively, viz. Firefox and Opera.

Some time around the middle of the last decade, when Internet Explorer 6 was still a pretty good browser and Firefox had just come out, I noticed that Firefox didn’t support the HTML <ruby> markup used to put furigana pronunciation glosses above Japanese text. IE had supported it since version 5, and the W3C recommendation on Ruby Annotation had been around since 2001, so I assumed that it would arrive shortly.

And so to today. It’s 2012. I’m working on a project that uses <ruby> to annotate pronunciation guides against Japanese text. This works in Internet Explorer. It works in Chrome, and other WebKit browsers. It still doesn’t work in Firefox. There are a couple of add-ons (HTML Ruby and XHTML Ruby Support) that give Firefox this functionality, but that’s not particularly helpful as a publisher, as you can’t rely on them being installed.

Without browser support, it’s not a complete catastrophe: <ruby> is designed to fall back gracefully, so that you end up with the pronunciation after the character in parentheses instead of above it, but it’s definitely a worse reading experience.

Fortunately, Firefox’s support for CSS is pretty advanced, so it’s possible to retrofit the necessary styles quite easily. With the addition of a small shim script that adds those styles to the document only when necessary, all that’s needed is to add one shim script to the head of the document, and <ruby> just works.



Opera doesn’t support <ruby> either, but the shim script works there too. As usual, Opera is weird, so the shim has to use slightly different CSS for Opera that doesn’t give quite as good results at every font size: characters may occasionally be a pixel or so off the baseline. Given that Opera is both idiosyncratic and relatively unpopular, I think that’s a good effort.

You can find the script and the instructions for use on the furigana-shim project page on GitHub.

Taking back control

I left the Government Digital Service last week, and, since a lot of people have been asking me why, I thought I should explain.

On one hand, I feel that I’ve left something undone by departing before goes fully live. On the other, I’m relieved. I’ve found working in a large organisation profoundly difficult, and I suspect I’ve not been an easy person to work with. I know that some people are glad that I just handed in my notice and left quietly; I think they rather expected some kind of ‘going postal’ massacre or similar. I’d hesitate to call myself a professional, but I’d like to think I’m not that unhinged.

I was there for nine months. Not a long time to stay in a job, but it was a busy period, full of change. I always knew that I’d only be working for the government for a short time, but I was still taken by surprise by quite how quickly it grew from fifteen or twenty people in a small room in Lambeth to something like two hundred in a massive (though, apparently, not big enough) office in Holborn. I never really got used to the huge open-plan environment, and in the end decided that it was just too much for me. It was a shame, because I really loved working in the small team in the old office, but with success comes growth. I’d never worked for a big company before. I’m now pretty sure I won’t do it again!

I finally realised—though it took me quite a while—that I didn’t actually have to stay working in an environment that didn’t fit me. It was psychologically liberating to realise that I could take back control of my own life.

Having said all that, I wouldn’t want anyone to conclude from this that GDS is a terrible place to work. It’s full of very talented people working hard on something socially important that everyone in the country will see and use. I want them to succeed. And, for the most part (when I wasn’t watching Bundler run, or fighting with Puppet), I enjoyed my work there.

The conventional thing would have been to find another job, but I’ve come to the conclusion that that’s not really what I want, so I’m taking a bit of time off instead. If you’re not a programmer, this probably seems like a stupid and self-indulgent course of action in the current economic climate, but I’m lucky to work in a field in which there’s no shortage of work, so it’s not quite the crazy gamble it may seem. And, when I say time off, I do have some plans in development, about which I hope to write more soon. I’ve done quite a lot of writing various kinds of content management systems in Rails over the past few years, and I fancy a bit of a change.

From now on, I’m looking forward to trying to implement my own version of the future. One with less commuting, I hope.

Testing CSS

I decided to take a look at automated testing of CSS, and I think it might be a useful thing to do. I’m not suggesting laboriously testing every last style, but checking some of the important properties seems to be both possible and potentially time-saving.

For about six years now, most of my programming has been test-driven development. It’s no longer a thing that I do because I feel I ought to: I do it because it’s an effective and efficient way to build and maintain working software.

Nonetheless, there are a few areas of software in which F5-driven development (the make-a-change-and-reload approach) is the norm, and that’s especially true for visual stuff. To some extent, that’s sensible: the important thing about styling a website or application is that it looks good, and that’s very hard to test objectively.

I’ve been working on some CSS over the past few days. One of the typographic principles that I want the stylesheet to conform to is the notion of vertical rhythm, so, to assist me, I set up a bit of JavaScript to toggle a striped background on and off, and peered at my elements to see if they were lined up.

After spending a while doing this, and making fixes that broke things elsewhere, it occurred to me that I could probably automate this process, and that it would be easier, more accurate, and less frustrating if I did so. I knocked together a quick implementation, and asked around to see if anyone else was doing something similar, and was pointed at a few examples, though nothing that seemed to be actively maintained or used.

Today, I’ve expanded my initial implementation into a test library that I’ve called tcsst (I suppose it’s a bit unpronounceable).

The approach that I’ve taken is that tests are defined in terms of selectors to which they apply. The test body will be run against every matching element, and one or more assertions can then be made against each element.

For example, to go back to my vertical rhythm check, I define a test that applies to almost every block element, and assert that the top of each of them is a multiple of the line-height as set on the body. tcsst prints some typical test runner output to the console, and adds highlighting to elements that failed the test.

My imaginary workflow is to create a number of dummy documents that exercise the layout in various ways, and to run the tests against each. This could also be used to test responsive design, by running the tests in browser windows of different sizes, and it could all be part of a full-stack test suite, once I’ve implemented a way to get the results out of the browser and back into a calling process.

You can see what I’ve done so far on GitHub: tcsst.