33 essays, edited by Andy Oram

I was wondering how to explain why I didn't like this book as much as I thought I would - possibly as much as I thought I ought to. Thirty-odd genius bitwranglers delving into their favourite bits of code - surely it must be packed with fabulous insights into the zen of programming. But something about it left me flat, and I was having trouble putting my finger on why. Fortunately Jeff Atwood's ever brilliant Coding Horror has saved me the bother, by explaining it perfectly.

Programmers love code. It is the culmination of their labours. It's not just a document or a diagram, but is a dynamic, living record of their conceptual jujutso. But to focus on the code itself, in a variety of languages, which few readers are likely to be familiar with all of, is a relatively superficial theme, and comes at the expense of the abstract designs that the code embodies, and of the features and capabilities of the languages themselves.

Rating 6/10: Contains some gems, but generally falls between two stools.

These folks will store samples of your pet's DNA, so that in the event of their death, you can replace them with a newer, younger clone.

Apart from the fact it's a gimmick ripoff (\$1,500 to store my pets DNA? That's quite a freezer they must have there, and the hypothetical cloning service 'may be offered by some other company at a future time') it seems to totally miss the point.

If you love your pet, it's surely for their personality. Your shared experiences and memories, having grown to know one another. I'm talking about their soul, or the state vector of their consciousness, whatever you want to call it. Growing a clone in their image is like creating a twin - it bears no relation to the animal you loved but for its superficial appearance, and even seems to me to be an insult to the memory of your former pet, that they could be replaced by the next one off the shelf. Certainly I wouldn't like to think my loved ones would be content to replace me with a look-alike.

Sometimes I wonder why we make it so hard on ourselves trying to make an honest buck when the rest of the world is one big scam, preying on the vulnerable and the unwary.

by Bertrand Russell (1946)

I've clearly been putting off this post for months, no doubt intimidated by my assumption that my review would have to be as weighty and well-considered as the tome itself. Screw that, a crap post is better than no post at all.

Overall, I enjoyed it greatly. It is a triumph in illustrating the breadth of the domain to a philosophical layman. It was as fascinating for me for the context in which different movements have arisen, for example the operation and mores of of ancient societies which differed greatly from our own, such as the Spartans.

In truth, my original acquisition of the book was partly motivated by a desire to justify my longstanding dismissal of philosophy as a meaningful discipline, and this narrow minded and self-fulfilling expectation was indeed confirmed to some extent.

The opening portion of the book, about ancient philosophy, made it all too easy to infer that the gems of ancient philosophy are very much rare pinnacles of achievement, set amidst a babble of incoherent theories and proclamations. The valuable ideas must be carefully searched out, extracted and refined, much as one might pan for gold on expansive shores of mud. However, this gratifying exercise in belittlement also brought with it an increment of my understanding as to why I feel this way.

It may be all very well for me to sit here nitpicking from atop the cumulative results of thousands of years of hindsight, but my shrill, gauche layman objections seem to be rarely well presented, and even less often indulged with a sensible rebuttal, so I'm glad that I read the book and gained some context in which to articulate them.

The raising of the intellect above the concerns of mere empiricism, as typified by the schools of Descartes and Plato, seems to be completely undermined by the apparently overlooked fact that there is no objective way of apprising the workings of 'pure reason.' The human mind, of course, is not capable of 'pure reason', if there even is such a thing, and is equally incapable of recognising its presence or absence in any given supposition. As a result, all exercises in deduction are extremely fallible, and this should come as no surprise to anyone who has ever forgotten or misjudged something. Just because the mind is not well understood, that does not make its machinations any less empirical than the evidence of the senses - what seems to us like logic is entirely subject to the vagueries of our implementation, regardless of whether or not our mind's implementation is embodied in our apparent physical brains or not.

To elevate the conclusions reached by this process above the concerns of mere empiricism is to put them beyond the reach of any form of feedback or discernment, a state of affairs which leads to entirely nonsensical stances standing uncontested for hundreds of years.

Because of this, I would argue that the teachings of thinkers such as Plato have in fact hindered the development of modern thought, not just by by blazing trails down several blind alleys, but more critically, by then systematically putting their own conclusions beyond the reach of criticism and refinement, effectively removing the intellectual tools that might have enabled pupils to retrace their steps, and map out the geography of thought more thoroughly.

The second portion of the book is the weakest, dealing with the only form of educated rumination occurring during the middle ages, which was religious in nature. It is perhaps unsurprising that a period such as this is notable for it's lack of original thought, barring individuals every few centuries such as Aquinas. As a result, large parts of this section degenerate into a litany of power struggles, assassinations and coronations, notable only for their tedium.

The final portion deals with modern philosophers, which regains the interest of the early sections, by virtue of describing the works of less religiously restricted protagonists, nurtured by the widening availability of education and communication, resulting in a glorious, chaotic competition of ideas. It was only during this section that I feel I gained an appreciation for the process itself, as opposed to the teachings of individual practitioners.

We, as humans, are rubbish at the act of thinking - we leap to conclusions, exhibit enormous biases, forget things, overlook things, make faulty correlations, are inconsistent, are influenced by personal gain and traumatic past experience, and worst of all: we are incapable of seeing these weaknesses - completely blind to our own incompetence. The means by which we evaluate our own conclusions are tied inextricably to the mechanism which derived them in the first place - we always think we are right.

Given this, it is no surprise that we can never agree on anything. We can never see each other's point of view. Whether tackling the problems of day-to-day life, or pondering the riddles of existence, a million different people will jump in a million different directions, no two quite the same. Most of those ideas turn out to be worthless. Idiocy that is detrimental or ineffectual. Only a few of them stick - and there is no way of knowing the merit or otherwise of any idea unless you have some criteria to judge them by. In day-to-day living, some ideas will allow the individual or group to prosper and be happy. In fields where objective measurement is possible - the sciences - then cumulative progress can be made, as successful ideas are shared and built upon.

However, outside of these domains, we may be captivated and entranced by the enthralling pictures philosophy presents to us, and our lives may be enriched as a result. But without any means of discerning the good ideas from the bad, we are are forever doomed to explore a multitude of alternative, competing, overlapping intellectual explorations, without any knowledge of which ones are meaningful, and which ones are purely fanciful. It may be great fun, but it's masturbation.

Rating: 8/10: The box with which to call me philistine is below:

In last week's 2D Graphics With pyglet and OpenGL, I used the pyglet library to produce some OpenGL triangles on the screen, from my rough-and-ready Python code. This week, I want to try to boost the throughput, to get some idea of how complex a scene we can realistically render from Python, while still maintaining a decent frame rate.

I was a little optimistic in my assessment of how fast last week's code was running. When I come to measure it carefully, I find that displaying just 85 triangles will bring the framerate down to a minimally acceptable 30fps. This is on my lappy - a Thinkpad T60, with a dual 1.6GHz cores, only one of which is busy, and an ATI Radeon Mobility X1400 running at 1680x1050. The framerate seems fairly independent of what size the triangles are, and of whether blend is enabled to make them translucent.

So what can we do to improve this? I suspect that an easy win would be to replace each entity's single triangle with a collection of triangles, specified by an array of vertices. To assemble the vertex list, we create the first vertex at (0, 0), and then lay all the following vertices in a ring around it.

I've shown vertex 6 lying adjacent to vertex 1, just to make them both visible, but in actuality they are coincident. Rendering these N vertices using glDrawArray() can produce N-2 triangles in the best case. All these vertices are shunted to the graphics card, translated, rotated, scaled and rendered in hardware, all without our code having to do any extra work, and hopefully without any significant performance penalty.

Starting with the code from last week, I modify it to generate the vertex list using the following new static member on class Entity. Note that I have coined the term shard to describe the individual triangles rendered by class Entity:

class Entity(object):

    numShards = 5
    vertsGl = None

    @staticmethod
    def _generateVerts():
        verts = [0.0, 0.0]
        for i in range(0, Entity.numShards+ 1):
            bearing = i * 2 * pi / Entity.numShards
            radius = (2 + cos(bearing)) / 2
            x, y = Position.CoordsFromPolar(radius, bearing)
            verts.append(x)
            verts.append(y)

        Entity.vertsGl = (GLfloat * len(verts))(*verts)

Entity._generateVerts()

The for-loop simply creates the list of vertex co-ordinates, as illustrated above. The cryptic-looking penultimate line converts that list into an array of GLfloats, as provided by ctypes, and stores that array on a class level attribute, Entity.vertsGl. The final line then calls this member function as soon as the class is defined, creating our vertex array at program startup. We also create a similar array of colors, which will be used to color each vertex, but since I want each fan drawn in a different set of colors, this is done in Entity.__init__(), and the resulting arrays are stored on the instance (not shown).

This vertex and color arrays can then be rendered as a triangle fan using the following Entity.draw() method:

def draw(self):
    glLoadIdentity()
    glTranslatef(self.pos.x, self.pos.y, 0)
    glRotatef(self.pos.rot, 0, 0, 1)
    glScalef(self.size, self.size, 1)

    glEnableClientState(GL_VERTEX_ARRAY)
    glEnableClientState(GL_COLOR_ARRAY)
    glVertexPointer(2, GL_FLOAT, 0, Entity.vertsGl)
    glColorPointer(4, GL_FLOAT, 0, self.colorsGl)

    glDrawArrays(GL_TRIANGLE_FAN, 0, len(self.vertsGl) // 2)

With other minor tweaks to give a new background color, running this with 3 shards per Entity produces quite a pleasing effect:

At 30fps, we can still manage 85 entities, and we're now rendering a fan of three shards for each one, so we've tripled our throughput to 225 triangles per frame. I suspect it can get better though. Let's try cranking up the number of shards per fan, while reducing the number of fans to maintain 30fps.

Shards per entity	Entities at 30fps	Triangles per frame
3	85	225
7	85	595
20	85	1,700
100	82	8,200
400	68	27,200
1,200	48	57,600
1,800	39	70,200
3,000	29	87,000
6,000	17	102,000
12,000	10	120,000
100,000	1	100,000

Above about 200 shards per fan, the shards start getting so thin that they produce moire effects, and above 10,000 there's some crazy white artifact starts happening in the middle of the fans. But nevertheless, the times taken to render these frames show a strong trend.

Fewer fans, each with more shards, results in much higher triangle throughput - up to 120,000 triangles per frame. Although it's exciting to see such high figures, I'd almost rather it wasn't the case. I'd prefer to create a game with more independent entities wandering around, regardless of how little graphical detail they could be adorned with. But there you have it, blame John Carmack. Anyhow, it's clear that we can deliver sufficient graphical grunt to put together some sort of game. Next time I hope to make a start on putting all these triangles to good use.

Update: For a 500% performance boost when running under Linux, invoke Python with the -O flag. I can now get 500 fans on screen, each with 100 triangles, at 30fps. See comments below.

On to Part 3 - Some Pretty Flowers...

Download the source

gameloop2.py.zip

So LC, Nixta and Aaron all like the Modu modular phone idea. A tiny cheap phone for evenings out, which slips into its super-blackberry exoskeleton for when you need to get hardcore about your comms.

Sounds great. I can't help but feel it's the first tiny step towards 'modular personal hardware' concept I've been banging on about for years. Think about what's good about this mobu phone, and turn it up to 11:

Your phone should be simply a transmitter and receiver. No screen. No keypad. No memory. It just makes phone calls, *nothing* else. Dirt cheap.

Your mp3 player likewise. No screen. No buttons. It just plays mp3s and produces an audio stream. Dirt cheap.

All those devices never come out of your pocket. If you want to interact with them, you carry ONE screen, ONE keypad control, or whatever. These connect to any of your carried devices, allowing you to control them.

If you're the sort of person who likes a nice screen, then you buy *one* nice screen. No need to also buy a nice screen on your phone and one for your mp3 player and one for your camera, etc etc etc. Same goes for any aspect of any component device, audio, control keypad, or storage, or whatever.

Storage is an illustrative example. Each device has almost no on board storage. Instead, they co-operate to share a single, central bank of storage, also in your pocket. So instead of carrying X gigabytes in each device (like now), you pool all that and carry 5X, shared between all your devices. So each device could still use X gigabytes each, if that was required. But now you also have the option to choose to allocate masses of storage to one single device, for example take four or five times more video than usual, and other devices can just get by with less memory for time being.

I'm tired of buying overpriced devices that bundle in a dozen mediochre implementations of stuff which all my other devices already do. It dilutes competition between hardware providers by forcing all devices to include a lowest common denominator *everything*. Christ, we *have* to put in a fancy graphical user interface and an mp3 player in every phone these days, so everyone puts in crappy ones, just barely adequate. And so when you choose a phone, you are stuck with all the other crap that your phone manufacturer decided to bundle with it. Unbundling this functionality would allow greater competition between device manufacturers. Users could pick and choose components that suit them.

Obviously, many users couldn't be bothered to mix and match, but for them, resellers could assemble pre-selected configurations. No biggie. Competition would still be enhanced, by the resellers choosing the best available component devices. And everything would cost 1/4 what it costs now.

It's the Unix philosophy applied to hardware. Small, sharp, incredibly powerful tools, that each do *one* thing incredibly well, and lean on each other to get things done. This gives users fantastic power to recombine them in manifold creative ways, unanticipated by the original hardware designers.

Welcome to my pipe dream. Everyone I've mentioned it to has called me an idjut. Your turn.

by Cory Doctorow

I just don't know about this one. I really wanted to like it.

spoilers

It starts out with an interesting, down-to-earth story about a guy establishing himself in a new neighbourhood, setting up an open wireless mesh net across the neighbourhood, back in a day when this was still a really cool idea. This tale is then peppered with initially disconcerting non-literal sounding references to his family, such as the cliffs and caves of his Father, the mountain.

Perhaps I embarked upon it too casually. It was certainly engaging, but by the last few pages I began to suspect that threads of the plot had eluded me. For example, I had entertained the idea that the fantasy elements of the story were actually the product of the main character's stated intention to write a book, which is otherwise only mentioned in passing here and there. However, the fantasy is woven extremely tightly together with the realistic aspects of the story, such that I can't discern any clear boundary to distinguish between the two. So I'm left wondering if parts of the book didn't actually happen, and if so, which parts? I should have anticipated someone like Mr Doctorow would have to be a terrific smarty-pants in the narrative department, and I doubtless should have paid more attention along the way.

Rating: 6/10 - One to revisit, maybe, if I like his other stuff.

Update 12th Feb: Coincidentally, I just read a short story by Cory, I Row-boat, which despite the awful titular pun, I really liked. However, I've read a bunch of discussion about Someone Comes to Town..., and I don't seem to have missed much that anyone else has been getting out of it. So while I'm looking forward to other stuff by him, I'm sticking by my personal low enjoyment of this one.

Directed by Francis Lawrence.

Summary This is going to be one of those reviews where someone reads the book and then sees the movie and doesn't like it very much.

Details I read Robinson Crusoe for the first time when I was about 30. It forms such a prominent thread within our cultural milieu that I expected it to be a pinnacle of literature, well-written and evocative. For me, however, it was nothing of the sort. It seemed to have been lazily dashed off by a nitwit aristocrat who'd never faced a single day of hardship in his life, with no thought at all given to the physical and psychological trials someone in Crusoe's predicament would suffer.

Only weeks later, I read Richard Matheson's novel I am Legend, and unexpectedly found it to be nigh on exactly the study of loneliness and despair that I had expected Crusoe would be. The portrayal of frantic days spent repairing the fortifications, alternating with horrific sleepless nights spent indoors, listening to them trying to get in, is as dense an atmosphere of desperation as I think I've ever been party to.

So it was with conflicting pangs of emotion that I first heard about the movie. In many ways the movie's strength is simply the inevitable reaction one has to the depiction of a desolate New York City. Will Smith's character is now a world-class scientist, bent on reversing the ravages of the zombie/vampire plague single-handed, working from his Hollywood-style basement laboratory. The novel's pivotal ending - from which the title is derived - is utterly transformed into something rather jolly and upbeat. Even the signs of him fraying around the edges under the strain have been sanitised - he talks to mannequins, rather than drinking himself into oblivion. It's Robinson Crusoe all over again.

Rating: 5/10 - not great.

pyglet is a cross-platform library that exposes Python bindings for OpenGL, and also provides a bunch of functionality layered on top of that, such as displaying text and images, mouse and keyboard events, and playing multimedia. I'd characterise it as a leaner alternative to PyGame.

I worked my way through pyglet's introductory example code, and was impressed enough to want to try it out with some of my own code, so I knocked together the following elementary 2D graphics demo.

Of particular note - it's a deliberate design goal of pyglet that it needs no other dependencies. After installing it, the following script 'just worked' on both my home Linux lappy and my work Windows desktop, and adopts sensible default behaviour across multiple monitors. Also, the resulting code is cleaner and less verbose than equivalent demos I've created in the past using PyGame.

App class (the controller)

The App(lication) class creates our other objects, and then runs the main animation loop. The window and clock modules are pyglet's - everything else will be our own classes:

• World class manages a collection of in-game entities. The world.tick() method updates the position or orientation of these entities.
• Camera class initialises OpenGL and defines projections that map from our in-world co-ordinates to pixels on-screen.
• Hud class defines text we draw on the screen overlaid on top of the world, such as a frames-per-second (fps) counter.

Note how in the main loop, we ask our camera class to set two different projections - one 'worldProjection', after which we draw in-game entities which are offset and rotated depending on the position of the camera, followed by a 'hudProjection', for drawing things that should be drawn in the style of a 'heads-up display', ie. always aligned with the screen, like text messages and frames-per-second (fps) counters.

class App(object):

    def __init__(self):
        self.world = World()
        self.win = window.Window(fullscreen=True, vsync=True)
        self.camera = Camera(self.win, zoom=100.0)
        self.hud = Hud(self.win)
        clock.set_fps_limit(60)

    def mainLoop(self):
        while not self.win.has_exit:
            self.win.dispatch_events()

            self.world.tick()

            self.camera.worldProjection()
            self.world.draw()

            self.camera.hudProjection()
            self.hud.draw()

            clock.tick()
            self.win.flip()

app = App()
app.mainLoop()

World class (the model)

The world class is just a container for a collection of in-game Entities. It uses pyglet's clock.schedule_interval() method to spawn a new Entity object at a random location every 0.25 seconds.

Every time world.tick() is called, we simply rotate each Entity by an amount dependent on its size.

Drawing the world merely clears the output buffer, resets the modelview matrix, and then asks each Entity to draw itself.

class World(object):

    def __init__(self):
        self.ents = {}
        self.nextEntId = 0
        clock.schedule_interval(self.spawn, 0.25)

    def spawnEntity(self, dt):
        size = uniform(1.0, 100.0)
        x = uniform(-100.0, 100.0)
        y = uniform(-100.0, 100.0)
        rot = uniform(0.0, 360.0)
        ent = Entity(self.nextEntId, size, x, y, rot)
        self.ents[ent.id] = ent
        self.nextEntId += 1
        return ent

    def tick(self):
        for ent in self.ents.values():
            ent.rot += 10.0 / ent.size

    def draw(self):
        glClear(GL_COLOR_BUFFER_BIT)
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        for ent in self.ents.values():
            ent.draw()

Entity class

Each entity knows its own location, orientation and size in world-space. It also knows how to draw itself, using a series of OpenGL calls. For now, I just draw a triangle for each entity, pointing along its orientation.

class Entity(object):

    def __init__(self, id, size, x, y, rot):
        self.id = id
        self.size = size
        self.x = x
        self.y = y
        self.rot = rot

    def draw(self):
        glLoadIdentity()
        glTranslatef(self.x, self.y, 0.0)
        glRotatef(self.rot, 0, 0, 1)
        glScalef(self.size, self.size, 1.0)
        glBegin(GL_TRIANGLES)
        glColor4f(1.0, 0.0, 0.0, 0.0)
        glVertex2f(0.0, 0.5)
        glColor4f(0.0, 0.0, 1.0, 1.0)
        glVertex2f(0.2, -0.5)
        glColor4f(0.0, 0.0, 1.0, 1.0)
        glVertex2f(-0.2, -0.5)
        glEnd()

Camera class (the view)

The camera class sets the OpenGL projections required to either draw in-game entities, or else HUD-style on-screen displays. In future enhancements, the camera's worldProjection mode will not just look at worldspace co-ordinates (0, 0), but will be able to roam around the world, and rotate.

Note that the widthRatio calculated in worldProjection() will do an integer division by default. To fix it, I imported real division (ie. from __future__ import division)

class Camera(object):

    def __init__(self, win, zoom=1.0):
        self.win = win
        self.zoom = zoom

    def worldProjection(self):
        glMatrixMode(GL_PROJECTION)
        glLoadIdentity()
        widthRatio = self.win.width / self.win.height
        gluOrtho2D(
            -self.zoom * widthRatio,
            self.zoom * widthRatio,
            -self.zoom,
            self.zoom)

    def hudProjection(self):
        glMatrixMode(GL_PROJECTION)
        glLoadIdentity()
        gluOrtho2D(0, self.win.width, 0, self.win.height)

Hud class (also part of the view)

The Hud class initialises the text string 'Hello, World!', and creates an fps counter. The draw() method renders both these to the screen. It is worth noting that pyglet handles text like this smartly, rasterising the Text object to a bitmap when it is first created, and then rapidly drawing that to the screen using a textured quad in the draw() method.

class Hud(object):

    def __init__(self, win):
        helv = font.load('Helvetica', win.width / 15.0)
        self.text = font.Text(
            helv,
            'Hello, World!',
            x=win.width / 2,
            y=win.height / 2,
            halign=font.Text.CENTER,
            valign=font.Text.CENTER,
            color=(1, 1, 1, 0.5),
        )
        self.fps = clock.ClockDisplay()

    def draw(self):
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        self.text.draw()
        self.fps.draw()

The Payoff

This is reasonably pleasing for a first stab. It runs at 60 frames per second, adding a new triangle to the screen every 0.25 seconds, and rotating them all gently. After about 100 triangles, it starts to slow down, but there are lots of things we can do to optimise it yet. In particular, I'm hoping that each triangle could be replaced by a complex geometry without any slowdow, by passing arrays of vertices to OpenGL, none of which need be touched by our code at all.

On to Part 2: Better Vertex Throughput...

Download the source

gameloop.py.zip

My lappy sketch lives! on Kamil's blog title. Har! That was an easy post. No thinking required.

by Bruce Sterling, read aloud by Cory Doctorow

Only the second audiobook I've bothered to listen to, and it seems to work very well, especially when commuting, since you can continue listening without interruption while leaving the tube and walking. (It's taken me 36 years to figure this out? Genius.)

Cory does a fine job of reading, and I'm grateful to him for bringing this important book to my attention. When he describes reading it on it's publication in 1994 as 'life changing', I can understand why. It describes the clash of cultures resulting from American law enforcement's attempts to crack down on the hackers, crackers and phreaks of the nascent cyberpunk underworld of the early '90s. Many of the underlying issues have even more relevance today. Can taking a copy of digital information be equated with stealing, given that the original owner still retains their original copy? Or is it more analogous to attempting to overhear a conversation which the speakers would rather wasn't overheard? While some hackers have real criminal intent, there are a significant proportion with a very strong ethic to do no harm, who view their tinkering as merely the exploration of an online frontier, filled with challenges and puzzles, and rewards of rich hordes of information - the dispersal of which is not just a god-given right, but actually a moral responsibility.

When law enforcement agencies went after this crowd with a heavy-handed and indiscriminate approach, teenagers still living with their parents ended up in court facing years in prison, colossal fines and legal fees, in some instances simply for simply republishing a document that was already in widespread circulation. Innocents had tens of thousands of dollars worth of computers seized and never returned, even though no charges were ever filed against them. The resulting backlash formed the beginnings of the Electronic Frontier Foundation, a philanthropic organization devoted to defending the rights of individuals in the online realm.

This whole tale is laced with many entertaining insights into the quirks and motives of the colorful individuals involved, and this makes the whole thing an enjoyable romp through a serious topic.

Rating: 8/10 - I only wish I'd read it ten years ago