Nerd

CloudFront Performance Numbers

Code on the Road

Yesterday, Amazon finally released the Content Delivery Network (CDN) they had been promising for several months. They're calling it CloudFront, and so far it seems to be living up to expectations.

It's dead simple to set up if you're already using S3 to store your content. Both Bucket Explorer and S3fox have already integrated CloudFront support, so you don't even need to write any new code. Just configure a few settings, switch the CNAME records in your DNS, and suddenly your content is serving a lot faster.

How much faster? Lots. Here are my numbers for serving a one pixel .gif file to my development machine here in the North of England (I've given URLs that are guaranteed to point to the right places, even after my CNAME changes propagate):

Amazon S3:
http://img.twiddla.com.s3.amazonaws.com/images/pixel.gif
300ms - 800ms latency, ~0s download time

CloudFront:
http://d2livl246cusvi.cloudfront.net/images/pixel.gif
46ms latency, ~0s download time

S3 performance is all over the map. As expected. Amazon never intended S3 to be used as a direct web host, so it's no surprise that it performs like a big dumb file storage system.

CloudFront, however, is amazingly well tuned. That 46ms time remained constant within 2ms every single time I loaded that file. In other words, CloudFront is so much faster and more consistent that there is simply no reason not to use it for all your S3 content hosting starting today.

The March of Access Control

John Resig

The web is changing. Historically it's been painfully easy to request resources from remote locations (such as stylesheets, scripts, images, and loading pages in iframes) - but this has brought along a whole world of security issues that browsers are continuing to try and resolve.

This openness has come to define what web development is all about: Dead simple sharing of resources and ability to get started. It's very likely that the lack of restrictions placed on these historical page elements will continue to plague browser developers for many years to come.

That doesn't mean that browsers have to make the same "mistakes" going forward.

This is where the new W3C Access Control specification comes into play.

All the new cross-domain-capable technologies that are coming to browsers will be requiring the use of Access Controls from the get-go, including:

• Cross-Domain XMLHttpRequest
• <video/> and <audio/> elements

The Access Control specification has been one of the most-rapidly-changing specifications that I've seen. I wrote a demo early this year and have had to update it at least twice since then in order to match the updated APIs - and it appears as if they may have even changed again.

Right now the specification requires that any resource that you wish to make accessible in a cross-domain manner must include an extra header specifying which site(s) are allowed to access it.

If you wish to allow any site to access your resource you would use:

Access-Control-Allow-Origin: *

and if you only wanted one other domain to access it you would use:

Access-Control-Allow-Origin: http://ejohn.org

This is important: It now means that site owners must make a conscious decision to enable cross-domain access of their resources (in contrast to images, stylesheets, and scripts which are always made available cross-domain with no way to disable it).

(There are a number of other headers specified by the Access Control specification, for more fine-grained access.)

There is going to be a lot of confusion and anger regarding this large, fundamental, change to the style of these upcoming APIs: They aren't like the web that we know and love!

The best write-up that I've seen, to date, was by Jonas Sicking of Mozilla on the Ogg Mailing lists. This one section, in particular, is particularly poignant:

Why not use the same policy as for <img>?

Yes, we could definitely do the same for <video> as we have for <img>. But it will come with the same downsides. It will mean that we will have to be much more cautious with how we develop the API for

There are already discussions about API features that we could not allow if we allowed cross-site video without Access-Control (or similar) protection. We would not be allowed to have callbacks for captions where the captions are handed to page javascript to be displayed in the page. This would allow an internet site to get captions from board presentation videos hosted on intranet sites, something that is obviously not acceptable.

We could say that the captions callback would work, but only if the video was loaded from same site, or if had the Access-Control-Allow-Origin:* header set. However this will likely result in random bugs like captions sometimes failing since the developer had perfect hearing and so didn't do a lot of testing with captions. In general accessibility is hard enough to get people to do correct that I'm reluctant to add features that work great as long as you don't take accessibility into account, but where you have to take extra steps to get accessibility to work.

Similar arguments goes for accessing the size of the video file (for example through progress events). We can not allow that to work for cross-site loads unless the site has opted in. This is because we likely won't know that it's actually a video that is being downloaded until after the first progress events have been fired. This means that you could use <video> to measure file sizes for arbitrary files that are otherwise protected by firewalls and/or logins.

If we always restrict usage of <video> to the cases when we know that the video is private data we will be much more free to develop APIs and functionality since we won't have to worry about protecting the data inside it, or deal with error conditions when someone tries to use sensitive APIs from a cross-site loaded video that didn't have Access-Control-Allow-Origin:*.

I recommend that you take the time to read his whole piece as it's worth it to gain a full understanding of the problems at play here (especially related to the <video/> tag).

One thing is clear: Security is being addressed center-stage in the new web APIs. This is going to be good as it'll prevent horrible security bugs going forward while, at the same time, change the landscape of web development in a very fundamental way. The web had its fun but now reality is starting to set in - it's time to get to work.

---

I'm reminded of the recent release of a crazy hack: transmitting data via URL encoded strings in stylesheets, named CSSHttpRequest. It's an insane technique (in the best possible sense of the word) and well outside the realm of most users. Even though the syntax and technique is different, the security/information-leak implications of this are every-bit as real as those presented by JSONP.

Theft of the Magi

xkcd.com

Anecdotes

Joel on Software

Michiko Kakutani reviews Malcolm Gladwell's latest book in the New York Times: “Much of what Mr. Gladwell has to say about superstars is little more than common sense: that talent alone is not enough to ensure success, that opportunity, hard work, timing and luck play important roles as well. The problem is that he then tries to extrapolate these observations into broader hypotheses about success. These hypotheses not only rely heavily on suggestion and innuendo, but they also pivot deceptively around various anecdotes and studies that are selective in the extreme: the reader has no idea how representative such examples are, or how reliable — or dated — any particular study might be.”

This review captures what's been driving me crazy over the last year... an unbelievable proliferation of anecdotes disguised as science, self-professed experts writing about things they actually know nothing about, and amusing stories disguised as metaphors for how the world works. Whether it's Thomas Friedman, who, it seems, cannot go a whole week without inventing a new fruit-based metaphor explaining everything about the entire modern world, all based on some random jibberish he misunderstood from a taxi driver in Kuala Lumpur, or Malcolm Gladwell with his weak theories on tipping points, crazy incorrect theories on first impressions, or utterly lunatic theories on experts, it all becomes insanely popular simply because the stories are fun and interesting and everybody wants to hear a good story. Spare me.

Friedman and Gladwell's outsized, flat-world success has lead to a huge number of wannabes. I was really looking forward to reading Simplexity, because it sounded like an interesting topic, until I settled down with it tonight and discovered that it was chock-full of all those amusing bedtime stories about the map of the cholera plague in London in 1854, which I've heard a million times, and then suddenly I noticed (shock!) that not only was the author a journalist, not a scientist, but he was actually an editor at Time Magazine, which has an editorial method in which editors write stories based on notes submitted by reporters (the reporters don't write their own stories), so it's practically designed to get everything wrong, to insure that, no matter how ignorant the reporters are on an issue, they'll find someone who knows even less to write the actual story. Panicking, I began to flip through the book at random. There's that story about Don Norman and complicated user interfaces. Here he is reading Nassim Taleb. I've heard all these anecdotes! Stop, already! I threw the book away in frustration.

This is the third one of the day. My business partner Jeff Atwood was busy extracting himself from the flamewars he started by writing an article on, of all things, NP-completeness, which is, actually, something that it's possible to know something about, because it's not a vague sociological hypotheticoncept like simplexiflatness or blinkoutliers, it's actually a real, important result from Computer Science, with a rigorous definition and lots of published papers, and poor Jeff got himself in something of a pickle by writing a book review when he hadn't read the book, and fortunately, he has comments on his blog, so his readers called him out on it.

Now, I am not one to throw stones. Heck, I practically invented the formula of "tell a funny story and then get all serious and show how this is amusing anecdote just goes to show that (one thing|the other) is a universal truth." And everybody is like, oh yes! how true! and they link to it with approval, and it zooms to the top of Slashdot. And six years later, a new king arises who did not know Joel, and he writes up another amusing anecdote, really, it's the same anecdote, and he uses it to prove the exact opposite, and everyone is like, oh yes! how true! and it zooms to the top of Reddit.

This is not the way to move science forward. On Sunday Dave Winer [partially] defined "great blogging" as "people talking about things they know about, not just expressing opinions about things they are not experts in (nothing wrong with that, of course)." Can we get some more of that, please? Thanks.

Not loving your job? Visit the Joel on Software Job Board: Great software jobs, great people.

Get In Focus

ongoing

Recently, the factor most limiting the quality of the pictures I take has been focus. When you shoot a few hundred pictures, a few will always constitute focus failures, but my fail ratio has been too high recently. I’ve thought about it and there are a few things wrong with my approach, but I thought I’d reassure myself that my camera and lenses were playing nice with each other.

The nice folks over at the Pentax-Discuss Mailing List recommended focustestchart.com which has Nikon in its title but the product works fine with any old camera. The idea is, you print out their PDF, you lay out the chart page really flat, you line your camera up, point it down 45º at the chart, and adjust your focus.

I have three everyday working prime lenses; the 40mm and 21mm “pancakes” from Pentax, and the Sigma 30mm f1.4.

If the Focus Test Chart is to be believed, both the 40mm and the 30mm needed adjustment, and the 21mm didn’t. Which may only be because the 21mm is at f3.2 and thus has more depth of field. Here are the “before” and “after” shots with the Sigma, forced to f1.4 for the minimum depth-of-field.

My Pentax K20 remembers adjustments on a per-lens basis, so these ones have been programmed in.

Will this actually improve my overall sharpness? Absolutely yes it will, because it establishes that my lenses and camera are about right, so the problem is clearly with the operator. I need to bear down on this.

Oh, and then there’s the pancake wide-angle, which, even opened out wide as it goes, has more depth of field than any sane person could want in any situation.

CentOS in the top500 supercomputers

Field Commander Wieers

Browsing through the top500 supercomputers list, I noticed that in the OS listing, 5 supercomputers are running specifically CentOS (1%) while 389 are running some sort of Linux (not specified).

From the Linux list undoubtedly more are using CentOS, but the remarkable fact is that this known 1% CentOS is the same amount as the 5 Windows supercomputers.

So if we assume from the 389 Linux supercomputers, more are using CentOS, CentOS outnumbers Windows for supercomputers. We simply don't know by what factor.

If only more organisations would be more specific to what exactly they are running.

More GitHub Gem Goodies

The GitHub Blog

You can now pass --learn to any GitHub gem command to see the git operations being run under the covers.

Hot off the presses:

$ gh fetch schacon --learn
Fetching schacon/master
== git fetch schacon master:refs/remotes/schacon/master
== git update-ref refs/heads/schacon/master refs/remotes/schacon/master
== git checkout schacon/master
warning: refname 'schacon/master' is ambiguous.
Switched to branch "schacon/master" 
$ gh fetch_all schacon --learn
== git fetch schacon
$ gh clone schacon/grit --learn
== git config --get github.user
== git clone git://github.com/schacon/grit.git
Initialized empty Git repository in /Users/chris/Projects/github-gem/grit/.git/
remote: Counting objects: 3369, done.
remote: Compressing objects: 100% (2107/2107), done.
remote: Total 3369 (delta 1229), reused 3355 (delta 1216)
Receiving objects: 100% (3369/3369), 1.80 MiB | 140 KiB/s, done.
Resolving deltas: 100% (1229/1229), done.

As always:

$ gem install defunkt-github

Want to contribute? Dive into the commands and join the long list of contributors.

GitHub and Eclipse

The GitHub Blog

Using Eclipse? Interested in GitHub integration? You may be in luck with EGit and this GitHub Guide by PEZ.

If you can help improve the guide, please do!

A Basic Emacs Setup for Clojure

Bill Clementson's Blog

A month ago, I wrote a post titled "Exploring Clojure (Lisp on the JVM) - Part 2 - Setup". In it, I listed the steps I went though to get Clojure setup on my machine. The steps I listed would normally work ok if you already use Emacs and SLIME and are used to customizing your .emacs file (Note: with the latest Clojure AOT changes, the setup instructions I listed in that post will need a little tweaking). However, my post wasn't meant to be a tutorial or a step-by-step set of instructions, it was just a list of things that I did to get going with Clojure. If you want/need a step-by-step tutorial for getting all the "key" components of a Clojure development environment setup, there is an excellent tutorial available. If, on the other hand, you want to avoid too many setup steps and want to get started with Clojure as quickly as possible, it is quite easy to get a working Emacs development environment setup for Clojure without using SLIME:

1. Download latest Clojure from svn:

   svn co https://clojure.svn.sourceforge.net/svnroot/clojure clojuresvn co https://clojure.svn.sourceforge.net/svnroot/clojure clojure

2. Download latest clojure-mode from git:

   git clone git://github.com/jochu/clojure-mode.git

3. Use ant to build clojure (just "cd" to the clojure directory and type "ant")
4. Put the following minimal setup in your .emacs file (adjusting the paths as necessary):

   (setq inferior-lisp-program "java -server -cp /path/to/clojure/trunk/clojure.jar clojure.lang.Repl")  
   (pushnew "/path/to/clojure-mode" load-path)       
   (require 'clojure-auto) 

5. Restart Emacs, load a clojure (clj) file in a buffer, press C-c C-z to get a repl and you've got a working development environment setup.

A slightly more sophisticated version of the .emacs file setup is in the Clojure Wiki. Later, after you've programmed in Clojure a bit and are ready for a better Lisp development experience, you can install SLIME. SLIME is a great Lisp editing package; but, if you've never used it for programming in Lisp before, then you can wait until you're ready to use it. And, it looks like Rich Hickey agrees with this too.

dj.god

The GitHub Blog

People have asked for our delayed_job god config.

Welp, here it is:

rails_root = "/data/github/current" 

20.times do |num|
  God.watch do |w|
    w.name     = "dj-#{num}" 
    w.group    = 'dj'
    w.interval = 30.seconds
    w.start    = "rake -f #{rails_root}/Rakefile production jobs:work" 

    w.uid = 'git'
    w.gid = 'git'

    # retart if memory gets too high
    w.transition(:up, :restart) do |on|
      on.condition(:memory_usage) do |c|
        c.above = 300.megabytes
        c.times = 2
      end
    end

    # determine the state on startup
    w.transition(:init, { true => :up, false => :start }) do |on|
      on.condition(:process_running) do |c|
        c.running = true
      end
    end

    # determine when process has finished starting
    w.transition([:start, :restart], :up) do |on|
      on.condition(:process_running) do |c|
        c.running = true
        c.interval = 5.seconds
      end

      # failsafe
      on.condition(:tries) do |c|
        c.times = 5
        c.transition = :start
        c.interval = 5.seconds
      end
    end

    # start if process is not running
    w.transition(:up, :start) do |on|
      on.condition(:process_running) do |c|
        c.running = false
      end
    end
  end
end

Enhanced Ubiquity Search

The GitHub Blog

Now you can search GitHub repositories or code using this new Ubiquity command.

Thanks bastos!

The Missing Near Future

KK Lifestream

Originally posted in The Technium

Science fiction is an entertaining way to worry about the present. It uses stories set in the future to confront issues of today. Even when it is oozing marvels which have not yet been invented, those futuristic things can only be related in the way the current audience perceives them. Just look at decades old science fiction to see how old-fashion-y it considers the inventions of today – computers and the like. That’s what makes us giggle about yesterday’s visions of tomorrow. In the past they get the new gizmo, but miss the new context. It’ll be the same with the most edgy science fiction today. In the future they will giggle. Regrettably the bias of the creation date is indelible.

The best scenarioists understand this. Here is contemporary sci-fi hero William Gibson on why much of his science fiction these days is placed in the here and now, or what he poetically calls the “ever-alien present”:

I took it for granted that the present moment is always infinitely stranger and more complex than any "future" I could imagine. My craft would be (for a while, anyway) one of importing steamingly weird fragments of the ever-alien present into "worlds" (as we say in science fiction) that purported to be "the future".

Of course not everyone is satisfied with the ever-alien present and craves some ever-alien future. Best would be the far future where true “otherness” lives. Where the beliefs and assumptions of today can really be tested. Hollywood, which has taken over the cultural center of science fiction,  prefers the cinematic far future, and so we continue to devour the far future sagas of Star Trek, Battlestar Galatica, Star Wars, Firefly, and so on.  But current science fiction of all types is leaving the near future a bit blank.

As an audience we can believe an alien present. It’s like today, only more so. Maybe an alternative version of today. We can also easily be persuaded to believe in a far future. We feel sure that someday, somehow they’ll have massive floating cities, or highways in the sky, instant food, and all the rest. We feel certain about this despite the fact that we can’t fund fast trains between our cities today, or permit genetically modified insect-resistant corn, or take any unified step toward large-scale 21-century developments. Even returning to the moon next decade seems far-fetched.

The near future – let’s peg it 2020 and beyond -- is a blank because there is almost no vision of a near-future that seems both desirable and plausible. Most stories, “worlds,” and scenarios of say the year 2050 are dystopian. Take your pick of nuclear self-annihilation, mortal pandemics, planetary floods, robotic overthrow, alien invasion, or fascist apocalypse. They are all very plausible, but not desirable.

The advantage of the far future is we don’t have to be told the story of how we arrived there, of how we passed through the near future. It’s far away enough that the creators can punt past it.  But the near future is such a conundrum that is it has disappeared from our culture.

Computer scientist and inventor Danny Hillis, born in 1956, noticed that when he was a child the future was ‘far away’ in the year 2000, but that as he grew older, the future remained rooted to the year 2000, as if newness could not move beyond that boundary. He describes it as feeling as if the future was “shrinking” year by year until in 1999 the future was only one year long.  Now that we’ve passed through 2000, the future has effectively disappeared – except for the far far future.

This disappearance has been made more real by key science fiction authors, futurists, and the brainy, nerdy folks who ordinarily keep busy churning out visions of tomorrow. A common belief in this circle is that things are moving so fast and weird that it is physically impossible to imagine the future of 2050 and beyond. Many of these futurists believe this discontinuity, called the singularity, is eminently desirable, because it is sure to lead to great intelligence, greater wealth, greater heath and immortality. But because it is forecast to come about via the shattering of what we understand humans to be now, many others will resist this future at all costs. And others believe the singularity future is not only undesirable, but implausible.

Either way, we are left with a blank for the near future. We have no story of progress that fits in the next century. There is no vision of 50 years hence that billions of people on earth would say, yes, that is what I want. Billions of people in the developing world know what they want tomorrow: clean water, free education, self-governance, cheap consumer goods, and hope for their kids. But beyond that, what? What do the billion in developed nations want? A clean environment and opportunities for meaningful work and ……?

We have great difficulty imagining progress in this century because in the last century we were educated about inherent complex side effects, by-products, and unintended consequences latent in every new thing.  We can’t see progress now because all we see are the costs.

It is unclear if the costs of technology are greater now because of their new complexity, or only more visible now because of their complexity. Probably both.

The conundrum is that no path, no vision of progress – technological, social, moral – will be plausible today if it does not include the complexity of costs, yet it will not be desirable if it does.

That makes our society blind. People assume progress even if they don't see it. They act as if progress is real, investing into future, starting things up, leaning into tomorrow as if it will be better than today – but there is no shared vision of what this is headed, or even where we’d like it to head. There’s no agreed way on measuring if in fact our actions are titled in the desirable direction – because there is no desired direction. Muddling through blind is the default scenario of the near future. We just kind of bumble along, taking one step after another with no larger goal. A few philosophers declare this is the post-modern stance. Living Without a Goal is all we can hope for, so we better get used to it.

The danger with this stance is that when there is no vision of progress or betterment to unify a society, the leaders will introduce fear to unify them.  The  BBC documentary series "The Power of Nightmares" argues that this is what recently happened in the US. When the hope of technology solving everything (he glorious days of Progress, with a capital P) petered out, it was replaced by the fear of communism as a way of unifying the country. When communism rotted from the inside and collapsed,  the fear of terrorism had to replace it. That exaggerated fear governed the past decade. But unless there is a plausible, desirable vision of betterment, one that billions can agree on, another fear will have to be found.

In that way I think there is a moral imperative to articulate our path towards something better. Not to leave it a vague post-modernist muddle. Not to shirk from the complexity and realities of costs. And not even to expect everyone to consent.

I don’t know if it is possible. It may be as the postmodernists insist: a vision relegated to the past. But I think we’ll behave better to each other, and towards future generations, if we can tease out a scenario of near-future progress for 8 billion humans and our uncountable natural co-inhabitants on this planet.

If you have a plausible desirable version of progress I'd like to hear it.

Playing the Blame Game

The GitHub Blog

One of the features I use a lot in git is ‘git blame’, which will show you who was the last person to change each line in any file in your project, and which commit it was.

This is most often used by me to avoid embarassment, as I have a ridiculously short term memory – there have been several dozen times in the last few years where I think “What moron added this code? It looks as if a slow monkey on barbituates got arbitrarily angry at the keyboard!”, only to find through my friend ‘git blame’ that that monkey was me. That’s helpful information to know before you go yelling at your co-workers.

So, we thought adding this face-saving functionality to GitHub might be a good idea. Enter the ‘blame’ link:

If you click that link from the file view of any of the files in a project, you’ll get something that looks thusly:

It’s basically the same view, with the code itself a bit truncated and a bunch of fun new information added. For each block of code that changed at once, it will show you the sha of the commit that last changed it, along with the author, date and message of that commit.

Even cooler, it will show a little ” » ” icon next to the sha – if you click this it will ‘drill down’ to show you the blame for the file from that commit instead. Meaning, if you wanted to know who changed that line before that gal, clicking on that icon will show you the next previous person, and so on. If that doens’t make sense, just give it a try – you can do this on any file in any of your projects. Go nuts!

(don’t really go nuts…)

We Are Typists First, Programmers Second

Coding Horror

Remember last week when I said coding was just writing?

I was wrong. As one commenter noted, it's even simpler than that.

[This] reminds me of a true "Dilbert moment" a few years ago, when my (obviously non-technical) boss commented that he never understood why it took months to develop software. "After all", he said, "it's just typing."

Like broken clocks, even pointy-haired managers are right once a day. Coding is just typing.

So if you want to become a great programmer, start by becoming a great typist. Just ask Steve Yegge.

I can't understand why professional programmers out there allow themselves to have a career without teaching themselves to type. It doesn't make any sense. It's like being, I dunno, an actor without knowing how to put your clothes on. It's showing up to the game unprepared. It's coming to a meeting without your slides. Going to class without your homework. Swimming in the Olympics wearing a pair of Eddie Bauer Adventurer Shorts.

Let's face it: it's lazy.

There's just no excuse for it. There are no excuses. I have a friend, John, who can only use one of his hands. He types 70 wpm. He invented his own technique for it. He's not making excuses; he's typing circles around people who are making excuses.

I had a brief email exchange with Steve back in March 2007, after I wrote Put Down The Mouse, where he laid that very same Reservoir Dogs quote on me. Steve's blog post was a very long time in coming. I hope Steve doesn't mind, but I'd like to pull two choice quotes directly from his email responses:

I was trying to figure out which is the most important computer science course a CS student could ever take, and eventually realized it's Typing 101.

The really great engineers I know, the ones who build great things, they can type.

Strong statements indeed. I concur. We are typists first, and programmers second. It's very difficult for me to take another programmer seriously when I see them using the hunt and peck typing techniques. Like Steve, I've seen this far too often.

First, a bit of honesty is in order. Unlike Steve, I am a completely self-taught typist. I didn't take any typing classes in high school. Before I wrote this blog post, I realized I should check to make sure I'm not a total hypocrite. So I went to the first search result for typing test and gave it a shot.

I am by no means the world's fastest typist, though I do play a mean game of Typing of the Dead. Let me emphasize that this isn't a typing contest. I just wanted to make sure I wasn't full of crap before I posted this. I know, there's a first time for everything. Maybe this'll be the start of a trend. Doubtful, but you never know.

Steve and I believe there is nothing more fundamental in programming than the ability to efficiently express yourself through typing. Note that I said "efficiently" not "perfectly". This is about reasonable competency at a core programming discipline.

Maybe you're not convinced that typing is a core programming discipline. I don't blame you, although I do reserve the right to wonder how you manage to program without using your keyboard.

Instead of answering directly, let me share one of my (many) personal foibles with you. At least four times a day, I walk into a room having no idea why I entered that room. I mean no idea whatsoever. It's as if I have somehow been teleported into that room by an alien civilization. Sadly, the truth is much less thrilling. Here's what happened: in the brief time it took for me to get up and move from point A to point B, I have totally forgetten whatever it was that motivated me to get up at all. Oh sure, I'll rack my brain for a bit, trying to remember what I needed to do in that room. Sometimes I remember, sometimes I don't. In the end, I usually end up making multiple trips back and forth, remembering something else I should have done while I was in that room after I've already left it.

It's all quite sad. Hopefully your brain has a more efficient task stack than mine. But I don't fault my brain -- I fault my body. It can't keep up. If I had arrived faster, I wouldn't have had time to forget.

What I'm trying to say is this: speed matters. When you're a fast, efficient typist, you spend less time between thinking that thought and expressing it in code. Which means, if you're me at least, that you might actually get at least some of your thoughts committed to screen before you completely lose your train of thought. Again.

Yes, you should think about what you're doing, obviously. Don't just type random gibberish as fast as you can on the screen, unless you're a Perl programmer. But all other things being equal -- and they never are -- the touch typist will have an advantage. The best way to become a touch typist is through typing, and lots of it. A little research and structured practice couldn't hurt either. Here are some links that might be of interest to the aspiring touch typist:

• Type Racer
• Typer Shark
• Dvorak, Keyboard Layout of Champions
• Colemak keyboard layout
• TyperA
• Das Keyboard with blank keys
• The Typing of the Dead (for PC)
• Put Down That Mouse. Seriously. It's a crutch.
• Typingmania (warning, Japanophiles only)

(But this is a meager and incomplete list. What tools do you recommend for becoming a better typist?)

There's precious little a programmer can do without touching the keyboard; it is the primary tool of our trade. I believe in practicing the fundamentals, and typing skills are as fundamental as it gets for programmers.

Hail to the typists!

[advertisement] Lighthouse — taking the suck out of issue tracking. Developer API. Email integration. Github integration. Used by thousands of developers and open source projects including Rails, MooTools, RSpec, and Sproutcore. Free for Open Source projects.

Multiple Post-Receive URLs

The GitHub Blog

You can now add multiple Post-Receive URLs for a single repository. Also, the Post-Receive settings have moved to the ‘Services’ tab (where they belong).

Recently Updated and Created Repositories

The GitHub Blog

You can now browse recently updated and recently created repositories for a specific language. Go nuts!

iPhone JavaScript Apps

John Resig

I've been watching, with interest, developers create new applications for the iPhone. Owning one myself - and being knowledgeable in JavaScript - I've been wondering what options there were for creating downloadable JavaScript applications or the iPhone. In doing some research I found a number of solutions, some simpler than others, some requiring more knowledge of Objective-C than others.

Before looking at the options I should note that given the requirements of the platform if you're really serious about getting in to iPhone development you should learn Objective-C. None of the options appear to provide the full range of functionality that a normally-written application would.

JiggyApp

This was an early entry to the JavaScript iPhone application market - arriving back in 2007. It requires a Jailbroken iPhone (likely running version 1.1 of the Operating System).

JiggyApp provides a full API for developing an application - apparently separate from most of the typical APIs. Arguably, though, the code ends up being relatively usable.

The above snippet was from the Getting Started with Jiggy page.

JSCocoa

JSCocoa is a full bridge that maps Cocoa development into JavaScript (instead of the typical Objective-C/Cocoa mapping). The result ends up working in both OS X and on the iPhone.

It's a pretty-clear port of Objective-C style and mannerisms but with a JavaScript syntax. Note some of the differences:

Objective-C/Cocoa:

JSCocoa:

Right now it seems like JSCocoa is more usable for developing OS X applications but the progress moving forward is certainly promising.

"Applications" in MobileSafari

While it's not, technically, a true iPhone application one option is to adapt your existing web applications to be used in a more application-centric manner.

Apple provides a number of tips for improving the style of your web application. The most important points of which are:

1. Providing a tray icon for the application (to be used when the user saves it).
2. Providing a full-screen view of the application (with no MobileSafari toolbars showing).

This will give the full appearance of a regular iPhone application (after using some more styling and setup from iui, or similar).

PhoneGap

The next step to making your iPhone web application more native-like is to tap into some of the underlying native APIs. One recent release that will help is that of PhoneGap.

PhoneGap is an application that exposes a few JavaScript APIs to pages running MobileSafari. Right now this includes Geolocation and access to the Accelerometer.

Geolocation:

Accelerometer:

They're also working on access to the camera, sound, and vibration tools of the phone.

WebTouch

The other day "Dr Nic" wrote up an article on how he had used a WebKit instance (along with HTML, CSS, and JavaScript) to render a portion of his iPhone application.

I contacted him about the project and wondered if he'd be willing to provide some of the code that backs it. Written up by his co-worker at Mocra, Anthony Mittaz, the result is called WebTouch.

Right now it's just a zip file of sample code but hopefully it'll be expanded at some point.

It's really simple and gives you a good entry point into the world of hybrid HTML/CSS/JavaScript/Objective-C/Cocoa development. If you've been interested in Objective-C this might make for a good starting point, as well.

Bonus

While this isn't something that you can actually use, I think it's pretty cool. This guy ported my Processing.js work to the iPhone, writing his own Canvas implementation using OpenGL ES hooking in to SpiderMonkey.

There are a lot of options available for the discerning JavaScript developer - the most promising of which is, I think, WebTouch. It's pretty obvious that in order to be able to build the best possible iPhone application you would have to know Objective-C. Having a clear path, paved with JavaScript, to that end result should be any JavaScript developer's goal.

MiniAjax.com For Sale

John Resig

Last fall I had the opportunity to acquire a great domain - one that was already quite popular within the Ajax realm: MiniAjax.com. It made a splash when it was released in early 2007 - and while it, literally, hasn't been updated since it first came out, it continues to draw a large audience.

It embodies a simple concept: Show nice screenshots and simple descriptions of eye-catching JavaScript/Ajax libraries.

That simple premise has drawn some phenomenal numbers:

• 20,900 adds on Delicious (One of the most popular sites on Delicious)
• 41,400 Monthly Visitors (Google Analytics access available upon request.)
• 4,500 - 5,000 Daily Feed Subscribers
• Google PageRank of 5
• Technorati Authority of 93
• At least 5 site submissions per month.

Any sort of update to the site would be sure to dramatically increase the, already substantial, traffic. There is nothing "powering" the site at the moment - it's just a static HTML landing page, some images, and a static RSS file.

Unfortunately, after owning the domain for just over a year and doing absolutely nothing with it, I've decided to pass it on. I've already received a couple offers by email but I'm open to more. My email address is: jeresig@gmail.com.

Something Like PyArg_ParseTuple For Ruby

hackety org

Calling into Ruby from C is great, but I’ve noticed that I spend a lot of time casting arguments coming into each function.

 rb_scan_args(argc, argv, "11", &port, &opts);
 if (rb_respond_to(port, rb_intern("to_str"))
   StringValue(port);
 else if (!rb_respond_to(port, rb_intern("read")))
   rb_raise(rb_eArgError, "a String or IO object only, please");
 if (TYPE(opts) != T_HASH && !NIL_P(opts))
   rb_raise(rb_eArgError, "options must be a hash");

Ruby is dynamically typed, but object types are a bit more reified in C. The TYPE macro can check an object to see if it’s a T_FIXNUM, T_HASH, T_STRING, T_ICLASS, etc. You can duck type all you want, but when you’re inside an extension, you’ll need to know the type before calling rb_str_cat or rb_hash_aref.

And StringValue does this, it’ll cast using to_str and then make sure you’ve actually a real T_STRING.

---

I’m disappointed with rb_scan_args. It’s wimpy. The function signatures it uses aren’t very expressive. It’s basically describing arity and that’s it.

One thing I like better in Python’s API, though, is the PyArg_ParseTuple function and its cousins.

 const char *file;
 const char *mode = "r";
 int bufsize = 0;
 ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);
 /* A string, and optionally another string and an integer */
 /* Possible Python calls:
    f('spam')
    f('spam', 'w')
    f('spam', 'wb', 100000) */

The function signature is more expressive here ("s|si"), indicating which types are allowed. You don’t have to check the types individually, nor do you need to throw individual exceptions.

---

Here’s an equivalent I’m working on for Ruby:

 rb_arg_list args;
 rb_parse_args(argc, argv, "s|h,-|h", &args);
 /* a string and optionally a hash OR an IO and an optional hash */

The rb_parse_args function returns the number of the match. If the first signature ("s|h") is matched, you get 1. If the second signature is matched, you get 2.

I’m planting this in a switch statement when I want to overload a method.

 rb_arg_list args;
 /* "k" means Class, "h" means Hash */
 switch (rb_parse_args(argc, argv, "kh,h,", &args))
 {
   case 1: /* "kh" - style(Link, :background => white) */ break;
   case 2: /* "h"  - style(:width => "100%") */ break;
   case 3: /* ""   - style() => {...hash of styles...} */ break;
 }

This is saving me some tedium. In typical shoddy form, my error messages blow. I am undisciplined’s middle name.

Ah, how good it feels to be inspired by Python. A bit good, a bit slimey. I’m like Joe Lieberman, guys.

Your computer might be at risk

Word Aligned

I dislike all three major OS — Windows, Mac, Linux.

— Peter Norvig, Stifflog interview

Although I write software for a living and write about writing software in my spare time, I don’t like computers much. I’m not afraid of them. I’m not completely incompetent at operating them. They just don’t behave the way I’d like. Here’s an example.

Hard drive failure

The family computer wouldn’t boot. Instead it came up with white text on a blue screen listing register contents in hex. A hardware check confirmed my fears: Error Code 7, hard drive failure.

This was irritating but not so very surprising: it’s an aging computer which had been running Windows XP progressively more and more slowly, and which was almost exclusively used by the kids for playing games on. On more than one occasion it had frozen and needed a hard reboot. What’s more saddening but also not so very surprising is that our domestic backup policy isn’t as good as it should be. There were some files on the dead drive we wanted to recover.

And even though Gail and I each have our own laptops, it soon became apparent how important a part of family life the defunct desktop was. Our children enjoy playing computer games. They like to watch Merlin on iPlayer. We needed their machine back.

Fortunately I had a laptop and internet access. After hunting through shelves and drawers I reckoned I could get my hands on most of the original install media. It didn’t take long to come up with a recovery strategy.

Recovery strategy

The plan was:

1. disconnect the dead drive
2. connect a new drive
3. install Windows on the new drive
4. reconnect the dead drive
5. boot Windows from the new drive
6. try and recover files from the dead drive

This article is not meant to be a tutorial on recovering from hard drive failures — if that’s what you’re after, the sanest source of information I found was this forum post by Patrick Keenan — but I do want to stress here that when you get to stage 5 you should not allow windows to run a disk check on the dead drive, which is what it defaults to doing.

Be ready to skip the check by pressing the Any Key!

“Free” file recovery software

The first few stages of this plan were tedious but straightforward. I nursed the machine through the Windows XP installation onto a new drive. Since the desktop was designed to accommodate a second hard drive, slotting in the dead drive alongside the new one was idiot-proof. One minor wrinkle: the system didn’t automatically register the new drive. I had to press F12 to interrupt the boot sequence, enter the BIOS and explicitly enable the drive. By now, I felt quite comfortable with the BIOS screens: this stuff works!

On reboot I skipped the default disk check, as mentioned. My hope was that the physical memory on the dead drive could still be read and that some lower level file reader utility could patch together enough of the file-system for me to recover what I needed. By the way, at this point the PC had no network access. Evidently Windows XP hadn’t installed a driver for the wireless PCI card. Fortunately the USB ports were functional, so I could download software on the laptop and transfer it to the desktop.

I wasn’t prepared to pay for any file recovery software (yet). The value of the lost files wasn’t that high, especially if I couldn’t get them back. Surely I could find some freeware? I took a couple of wrong turnings here and ended up feeling rather gullible. The top few search hits led me to software I could download and demo for free, which claimed they could do the job, but which I’d have to pay to use.

DiskInternals NTFS Reader turned out to be what I was looking for. I’m sure there are others. Happily I’d soon managed to locate and recover the files we wanted, and could disconnect the faulty drive. Phew!

Restoring the machine

Restoring the files was one thing. Next I needed to restore the machine itself to full functionality. The screen resolution wasn’t right. Screen repaints were jaggy. It couldn’t access the internet. There was no sound (apart from an annoying system beep). Etc. etc.

What’s going on here? I’m running a licensed copy of the world’s most popular operating system on hardware purchased from the world’s best known PC dealer, not some bleeding edge Linux distribution. Surely it should just work!

Hobbyists may delight in tinkering and fixing computers. Not me. I’m not clueless, though, and I did realise that what I needed to do was keep downloading and installing drivers until everything behaved. It would have been nice if the system had been a little more self aware, though. Why couldn’t it tell me why it wasn’t working? Why couldn’t it at least provide a hardware manifest in some standard format?

I won’t bore you with the details of what followed. You don’t need me to tell you about the repeated reboots, broken download links, quirky installers; like me, you’ll be immune to the screenfuls of THREATENING LEGAL MUMBO JUMBO YOU HAVE TO AGREE TO. But I do want to highlight a couple of egregious examples of what I’m talking about.

Netgear Wireless PCI card

I had the original install media for the wireless PCI card, and this got the machine connected to the internet. I didn’t just have to reboot to complete the installation, though, I also needed to physically remove the card and reseat it halfway through the process. Good job the PC chassis was already open. As if all this wasn’t bad enough, I also had to bypass a grim warning from Windows XP about destabilizing my system. Lovely!

Then I discovered wireless internet access only worked for system administrators, a level of privilege no one should really need to run at. Googling the problem suggested I was not alone. So I downloaded the latest drivers, uninstalled the original version, reinstalled the new (power off, remove the old card, power on, install software, power off, insert card, power on, finish installing the software) only to find the driver problems still had not been fixed!

To work around the problem I had to follow the install procedure for a third time, in this case not clicking through the recommended path, but rather allowing the Windows network connection manager to run the wireless service. Even then I had to stop the Netgear software from auto-starting.

Grrrr!!

Flash install problems

The kids like playing Adobe Flash games so I installed the Flash browser plugin. Well, I tried. Apparently I’d run out of disk space.

An alarming message, considering H:\WINDOWS\system32 was a directory on my brand new, unpartitioned and almost empty 500 GB hard drive! A little more googling revealed that the Flash installer actually required the presence of a C: drive. After mounting a USB flash drive and fiddling about with the control panel to remap drive labels, I managed to get Flash plugged in.

Unfunnily enough, I recently faced a similar but less tractable issue at work, trying to get the Flash plugin working on a Red Hat Linux box. Why should running Flash require me to upgrade libc!?

The antivirus racket

Checklist:

• Hardware happy? ✓
• Software happy? ✓
• Computer happy? ✘

Despite the clean install of properly licensed software on a new hard drive, Your computer might be at risk. Clicking the balloon to fix the problem directs me to the Microsoft antivirus partners page, where plenty of companies will be happy to help — at a price, that is.

This, I think, is the final straw. I’m no longer willing to play along with this racket. The antivirus software is itself a virus, a virus carried by my reconstructed computer from the moment I installed Windows; and clicking the threatening balloon will only make things worse. In the computer’s former life, I installed some free antivirus software which turned out to be free as in free beer, not free as in without cost. It soaked up resources, hammered the hard drive mercilessly, and was forever interrupting me, cajoling and wheedling me to upgrade to the professional edition.

You idiot!

As a software professional I should know better. I should know better than run Windows. I should know about security. I should know about backups. I had a lucky escape, and I’ll admit, I was lazy, but the truth is this kind of thing happens to computer users — both professional and amateur — all the time, and many have been burned far worse than me. I think the model of standalone computers with their own hard drives running their own operating system is all wrong, and bolting on an external hard-drive and some backup software doesn’t make it all right.

Alternatives

Right now, I’m not inspired to go out and buy another computer plus the additional hardware and software required to make the house a secure and safe place for computing. It’s a home, not a server farm!

Yes, if we went the Apple route and purchased hardware and software in beautiful (but expensive!) self-contained packages the driver problems ought to disappear and time machine would allow me to chase backwards and recover anything and everything. I’ll admit I’m tempted, but I have my concerns about vendor lock-in: if being tied to a software application is bad, being tied to a platform is worse. And yes, if we went the Ubuntu route I don’t think we’d be troubled by the antivirus protection racket but I suspect I’d still end up grubbing round for drivers, possibly with even less success, and I’m sure it wouldn’t be such a good games platform.

In an interview on stifflog a number of well known programmers answer questions about their favourite tools (operating systems, languages, editors etc.) I sympathise with Peter Norvig’s rather negative answer.

“I dislike all three major OS — Windows, Mac, Linux. I like Python and Lisp. Emacs.”

Personally, I prefer the model of computing as a service, a service you connect to from a variety of devices in a variety of locations. I don’t really care where exactly my data is — I just want (secure) access to it, wherever, whenever. I don’t want a house full of wires and hard drives. Backup should be transparent to me as a user, and it should be straightforward for service providers to implement: bytes are easy to replicate and transmit, to distribute, and there are economies of scale. (For what it’s worth, my personal backup strategy, based around travelling light, internet access, version control, cron and rysnc, does what it can to emulate this model, but it certainly isn’t for everyone.)

I don’t think this a Utopian view and I certainly don’t claim it’s original: computing has been gradually moving in this direction for some time now. Perhaps, though, we should refuse to accept the dystopia we’re currently living with.

The future of operating systems

You’ll have to look hard for an honest and open insider’s assessment of the state of Mac and Windows, but the Unix community have always been quick to acknowledge design limitations. In an interview on Slashdot, Rob Pike explains how weak the Unix design has become in the networked world we live in.

The major things we saw wrong with Unix […] back around 1985, all stemmed from the appearance of a network. As a stand-alone system, Unix was pretty good. But when you networked Unix machines together, you got a network of stand-alone systems instead of a seamless, integrated networked system. Instead of one big file system, one user community, one secure setup uniting your network of machines, you had a hodgepodge of workarounds to Unix’s fundamental design decision that each machine is self-sufficient.

We really are using a 1970s era operating system well past its sell-by date. We get a lot done, and we have fun, but let’s face it, the fundamental design of Unix is older than many of the readers of Slashdot, while lots of different, great ideas about computing and networks have been developed in the last 30 years. Using Unix is the computing equivalent of listening only to music by David Cassidy.

Earlier in the same interview, Rob Pike clears up a misconception. Although he co-authored a book on about Unix, he cannot take credit for its creation.

Ken Thompson and Dennis Ritchie created Unix and deserve all the credit, and more.

Which leads us from Slashdot to Google, the computing equivalent of listening to, say, the White Stripes. On the Ask a Google engineer forum, Jeff B from Woodstock GA asks a leading question.

I believe the future of operating systems is a “Cloud OS.” Do you agree?

Ken Thompson from Mountain View CA responds.

yes

Back in the present

Ironically, the remade computer is actually tolerable to use at present. It boots in seconds, has decent peripherals, and (at last!) carries no craplets. Having spent a few hours getting to know it, from BIOS, through driver, to control panel, I’m on better terms with it than ever before. Maybe it’s not such an old sad thing after all.