Neogeography

Over the summer I've been keeping a geographic journal of some of our recent adventures:

It's an amusing journalling project for me, but as an exercise in neogeography (the distainful term Geographic Information System pros use to describe such upstart tools, popular with the great unwashed) I have to say it's pretty forlorn. The editing tool is thoroughly annoying to use, and the data model is none existant - you're simply crayoning one-off symbols on top of the map, with no ability to relate any symbol to any other, except by manually assigning them the same symbology. The ability to import KMZ is little consolation if all the resultant data is so unstructured. No doubt things will get better as GIS and neogeography borrow each other's best ideas, but for the time being this exercise left me disappointed.

I seem to recall that the editing tools in Google Earth are a smidgeon better, but still profoundly lacking. Anyone got any ideas what I should be using instead for personal 'for amusement' projects like this?

Writing C/C++ on Windows using g++ under Cygwin

I wrote this ages ago, for the benefit of a Windows user who wanted guidance getting set up with a free C/C++ compiler to learn the language. I recommended using the g++ compiler in a command-line environment under Cygwin, with the following guide to getting started. I recently found it lurking on my hard-drive, and figured it's not doing any good to anyone sitting there, so here it is, just in case.

Introduction

This article outlines one method of using the open source C/C++ compiler, Gcc to compile C/C++ programs on a Windows PC. It uses Cygwin, which emulates a Linux-like command-line environment, which is many times more useful and powerful than the DOS command line that comes out of the box with Windows.

Of course, if you don't like all this free-as-in-beer, free-as-in-speech, command-line hippy nonsense, you could always just splash out for Visual Studio instead. Such a well-designed IDE has many strengths - but so does the approach I am about to describe. Choose the tool appropriate to the task in hand.

Install Cygwin

Download and run setup.exe from http://www.cygwin.com/setup.exe

The root directory you specify to the installer, default C:\cygwin, will be your Unix root (/) directory under Cygwin. If this doesn't mean much to you, see the (currently hyperthetical) section on Unix filesystems below.

Install for all users, and leave default text file type as Unix, not DOS. There are more instructions for Cygwin setup online, but my intention is that this document should be sufficient.

Eventually you get to choose which packages to install. This window is resizable, even though it doesn't look like it. Maximise it. Also, select the 'Curr' radiobutton on the top right - so that we only view the current versions of each package. (as opposed to old or experimental versions.)

Expand the categories on the left by clicking their '+' signs. The 'current' column is empty, because you don't yet have anything installed. Under the 'new' column, you can click the word 'skip' to select which version of each package you wish to install. This installer understands the dependencies between packages, so selecting package A which requires package B will automatically select package B.

Everything under the 'Base' category is already selected, plus their dependencies. This is a minimal Cygwin install, to which we will now add one or two things.

Under the 'Devel' section, find the entry for 'gcc. C compiler upgrade helper.' Selecting the latest version of this package (3.3.3-2 as of 18 Aug 2004) will automatically select all the other parts of gcc that we need (gcc-core, g++, etc.) as well as the 'binutils' package.

Make sure the following packages are selected:

  • Devel/gcc - will automatically select some other packages
  • Devel/make
  • Doc/cygwin-doc

You'll also need a text editor. You can either select and install one of those listed in the Cygwin installer, or else use any Windows program. Notepad will work fine for the moment. Download Textpad for Windows if you want something a little more useful without being intimidating. Use something like Vi or Emacs if you are already familiar with them.

Click [Next] when you're done. Setup downloads and installs the selected packages.

Run Cygwin

Double click the Cygwin icon on your desktop. You should get an uninspiring DOS-like text terminal in a window. Type the command 'ls', which lists the files in the current directory. From now on I'll show commands you should type in monospace, preceded by a dollar sign, possibly followed by expected output of the command, like this:

$ ls
My Pictures My eBooks My Music

With any luck, you should recognise the files listed as those residing in your My Documents directory.

I don't know how much/little Unix or Linux you know, so I'm going to explain that as little as possible for now, and just press on with what commands you need to type to get a C program compiled.

Create a C file

Create a new directory within 'My Documents', where our C programs will live. You can use Linux commands if you know them, or use your Windows Explorer thing. I'm going to use Linux commands:

$ mkdir cprogs
$ cd cprogs

(The 'cd' changes directory into the new 'cprogs' directory. Note that the yellow '\~' above your dollar prompt has changed into a yellow '\~/cprogs', to indicate this. So now if we type 'ls', then it lists nothing - our new directory is empty.)

Next we want to create our c source file. If you installed vi or emacs as part of the cygwin setup, then you can use these here:

$ vi myprog.c

Otherwise, you can just use notepad, or any other windows text editor. You can invoke it from the command line if you wish.

$ notepad myprog.c

Enter and save the following minimal program:

#include <stdio.h>

int main(int argc, char ** argv)
{
    printf ("Hello world\n");
    return 0;
}

Notice that, while notepad is still open, you can no longer type anything in your Cygwin terminal window. Cygwin is waiting for notepad to finish before it lets you type anything else. Save and quit notepad (now you can type again) and then run it once more, appending an ampersand at the end of the command:

$ notepad myprog.c &

The ampersand tells cygwin to run notepad 'in the background', while allowing the cygwin terminal to continue accepting input. For example, while notepad is still open, you can now issue a command to compile your program.

Compiling

To compile the program, we invoke the gcc program that we installed as part of cygwin setup:

$ gcc myprog.c -o myprog

This tells the program gcc to compile myprog.c, and to call the output file myprog. If this produces error messages, check the program in Notepad, save again (Ctrl+S in Notepad) and then recompile. Note that you can re-issue the compile command by pressing the up arrow key, then return. If the compile command produces no error messages, nor any other kind of response, then the compilation has been successful. You can verify this:

$ ls myprog.c myprog.exe

(Note: on a real Unix/Linux box, the program would have no '.exe' extension. It would simply be called 'myprog'.)

And to run your new program:

$ ./myprog
Hello World

Wrapup

That's the basics. You've got yourself a kick-ass C compiler (or C++ compiler if you invoke it with 'g++' instead of 'gcc') along with a Linux-like environment to use it in. All for free, and without having to repartition your existing Windows install or suffer the frustration of a dual-boot system.

The Art of Unix

It should be noted that a real Unix/Linux development environment will build on the arrangement described above, often in the Unix style of 'many small tools talking to each other', rather than as a single monolithic application in the style of Visual Studio. This allows competing tools to be slotted into place, giving the developer the maximum freedom of choice. For example, it is trivial to replace Gcc with the Intel C++ compiler, Icc, resulting in faster execution speed at a cost of cross-platform compatibility. Such flexibility leaves the developer free to retain their favourite text editor, debugger, etc, regardless of the type of work or what project they are working on.

The command-line is often frowned upon by developers more familiar with a GUI way of working, who tend to assume that such an ancient interface represents a cruder way of working. In actual fact, the reverse is true. A GUI is good for providing guidance to novice users, but ultimately proves limited and inflexible. One can only use a GUI application for exactly the uses the original developer envisaged. With a well-designed command-line, the user is faced with a steeper learning curve, but is then free to combine programs in novel and constructive ways.

For example, suppose we had 100 C programs to compile (not so far-fetched if these are actually libraries or plug-ins.) In many IDE GUI environments, the user would have no choice but to spend all afternoon loading each program in turn and clicking the [compile] button. Knowing a little command line magic, however, makes it a ten-second job:

$ for SOURCE in *.c;
> do
>   gcc $SOURCE -o ${SOURCE%.c};
> done

Each method has it's strengths and should be used where they are most appropriate.

Acceptance testing a pyglet application

I've been trying to create a simple acceptance test for a pyglet application. A thorough suite of acceptance tests, verifying the correctness of all the shapes drawn to the screen by OpenGL, sounds like far more work than I want to do. But a couple of simple acceptance tests would be valuable, to check out basic things: that the application runs; opens a fullscreen window; responds correctly to some basic inputs and command-line options; and has an acceptable framerate. Especially if I could quickly run this basic smoke test on multiple operating systems.

I tried writing an acceptance test which ran the application-under-test on a new thread. This didn't work for me *at all* (perhaps because pyglet is not intended to be used with multiple threads), so for the time being I'd given up, and was proceeding without acceptance-level tests.

A couple of days ago I had the idea of a test that didn't involve threading. Instead, it takes a list of test conditions (as lambdas), and uses pyglet's own clock and scheduler to request a callback to a test function - try_condition() - on every frame:

    def try_condition(self, dt):
        if self.condition():
            self.next_condition()
        else:
            self.time += dt
            if self.time > self.timeout:
                self.fail("timeout on %s" % self.condition)

When try_condition() gets called by pyglet on the first frame, it evaluates the value of self.condition. If True, then that first part of the test has passed, and it gets the next condition off the list. If False, then this function simply returns, to let the application continue running. When try_condition() is called again on the next frame, it resumes where it left off, testing out the same condition again. After it has been trying the same condition for a long enough time, it deems that condition to have failed, and raises an assertion error.

Here is the rest of the class, which sets up the scheduled calls to try_condition().

from unittest import TestCase
from pyglet import app, clock

class AcceptanceTest(TestCase):

    timeout = 1.0

    def set_conditions(self, conditions):
        self.conditions = conditions
        self.next_condition()
        clock.schedule(lambda dt: self.try_condition(dt))

So self.conditions is a list of lambdas that will be provided by the acceptance test. Calling self.next_condition() merely plucks the next condition off the list, into self.condition. If there are no more conditions left, then the test has entirely passed and it requests the application to terminate, by setting the pyglet member window.has_exit to True.

    def next_condition(self):
        if len(self.conditions) > 0:
            self.condition = self.conditions.pop(0)
            self.time = 0.0
        else:
            self.terminate()

    def terminate(self):
        win = self.get_window()
        win.has_exit = True

    def get_window(self):
        windows = self.get_windows()
        if len(windows) == 1:
            return windows[0]
        return None

    def get_windows(self):
        return [w for w in app.windows]

In future, I should probably allow the acceptance test code to specify a different timeout value for each condition.

Anyhow, we can then write an acceptance test by inheriting from this AcceptanceTest class, providing an appropriate list of conditions, and then just calling the application's main() function. This function won't return until the application exits, either when one of the conditions times out and raises an assertion error, or else when all conditions have passed and the test framework sets window.has_exit.

from testutils.testcase import run_test
from acceptance_test import AcceptanceTest
from sole_scion import main

class AT001_app_opens_a_fullscreen_window(AcceptanceTest):

    def is_window_visible(self):
        win = self.get_window()
        return win and win.visible

    def is_window_fullscreen(self):
        win = self.get_window()
        return win and win.fullscreen

    def test_window(self):
        conditions = [
            self.is_window_visible,
            self.is_window_fullscreen,
        ]
        self.set_conditions(conditions)
        main()

if __name__ == "__main__":
    run_test()

The conditions don't simply have to be expressions to verify the application state. They could stimulate the application by raising mouse or keyboard events, etc, and then simply return True so that the test harness would move right on to the next condition.

Early days with this idea, but it seems to work, and thus far I'm very happy with it.

Micro ISV : From Vision to Reality

by Bob Walsh

Finally got around to plucking this off the shelf, and yes it's an interesting grab-bag of things to think about, and case studies of people who have taken the plunge. But there's not very much in here that's significantly unexpected. The chapter about business formalities says 'yes you should create a business plan', but has no guidance about how to go about that.

All in all then, a nice checklist to make sure you're not overlooking anything too obvious, but it didn't feel like the in-depth detail I was hoping for.

Rating: 6/10

Sole Scion project v0.1 : A 2D rigid-body Python game

screenshot Sole Scion
v1.0

Everyone today is abuzz with the 'release early, release often' sentiment, so I've decided to get in the spirit and upload the first iteration of 'Sole Scion', my summertime sabbatical game project.

Version 0.1 merely bounces some circles off an inclined plane (pictured.) I aim to transform this into a 2D graphical adventure game, in which I hope emergent fun will take place, once I populate the world with enough twisty corridors, uniquely-shaped objects that need slotting into the right places, and judicious instances of ornery fauna to contend with.

It's written in Python, using Chipmunk for rigid body Newtonian dynamics, with Python bindings using Pymunk. Rendering, gameloop, sounds, etc are handled by pyglet.

Sole Scion is hosted on Google Code, with Subversion read-access and a source code tarball, including the redistributables for the above dependencies, should anyone be curious. It has a New BSD License.

Feedback, thoughts and ideas, very welcome.

Update: I've only run it on Linux, but reportedly it also works on Windows. Thanks Xtian!

Update: It also reportedly works under PyPy, provided you install the 'functools' module. Thanks Maciej!

The Portable Jung

Carl G. Jung (Author), Joseph Campbell (Editor), R. F. C. Hull (Translator)

I didn't quite know what to make of Jung when KT first handed me this collection of his essays, many moons ago. Some sort of 19th-century lunatic Freudian psycho-analyst who was a dab hand with a ouija board and sang the praises of ritual sacrifices to the almighty triple-headed hermaphrodite goddesses, right? Something like that.

So it was good to discover that Dr. Jung isn't the wild-eyed loon that I'd written him off as. His attitude is entirely credible and scientific, and while he may invoke triple-headed hermaphrodite goddesses, he does so in an entirely respectable and deeply pertinent way.

His essays are prefaced by a short biography to set the works in a meaningful social and personal context. Of particular entertainment value is the ongoing deterioration of his relationship with Freud, and the corresponding divergence of their theories. This culminated in Freud's desperate plea for Jung's cooperation in ensuring that 'the sexual theory' be established as an unchallengeable axiom in the explanation of all human behaviours. This didn't strike Jung as being particularly scientific, and obviously speaks volumes about Freud - lifelong adulterer - as both a person and a theorist.

Of the essays themselves, some of them resonated more for me than others. I very much liked his explanation of the nature of the subconscious necessarily being the polar opposite to the conscious ego, since all perceptions which do not fit in with the beliefs or attitudes of the conscious mind are dismissed and repressed. This is a natural and normal reaction, and it helps us to maintain a consistent and useful world-view. However, those repressed memories are not erased - they are simply being ignored by the concious mind. The subconscious appears to have unfettered access to these repressed fragments. In situations where the conciousness exercises wilful determination, this leads to a stressful state of cognitive dissonance, in which the subconscious comprises a roiling mass of all the impressions and emotions that the concious mind has rejected. Hence, the tendancy for people to maintain a strident outward appearance which is completely at odds with, and arguably exacerbates, pent-up inner demons. For some reason, the examples that spring to my mind are priests molesting altar boys, or maybe catholic schoolgirls. Not sure if that's a reflection of modern organised religion or my own state of mind, oh well.

Equally interesting is Jung's theory of the collective unconscious - that the religions, myths and legends of the world are united by a handful of common themes, or archetypes, not because of cross-pollination of ideas, but because these themes are an expression of psychological commonalities between all human minds - seems credible and useful. I still have some reservation in accepting Jung's particular menagerie of archetypes as definitive. He may have distilled them from many years studying the world's religions and mythologies, but if you throw enough noise at a filter the result ends up being characteristic of the filter (ie. Jung himself) rather than the noise. However, the validity of the idea in general, and of some of fundamental archetypes, seems unarguable.

He carefully avoids any explicit judgement of whether any religions or myths have any basis in objective reality. Such judgements, he rightly notes, are beyond the ability of the scientific method to evaluate. However, he provides the potential to categorise them all as simply being phenomena of human psychology - ie. they are entirely in our head. This is a double-edge sword, however, for this seeming dismissal rebounds poetically upon us. What were once distant and faintly heard mythologies, are now given a deep and highly personal significance, as they are nothing less than the worldy manifestations of the innermost workings of our own minds, which each of us would fail to heed at our peril.

There are other sections which I didn't find so compelling, such as his concept of synchronicity, but overall it was thoroughly enjoyable and insightful.

Rating: 8/10

Evolved for suboptimal decision making?

I *should* be searching out and evaluating Python geometry libraries. But I keep getting these niggling impulses to start hammering out my own vector, polygon and quaternion classes. Possibly in C. Ohdear.

Why is it so much easier, or at least so much more appealing, to start beavering away on your own code, rather than to take the sensible approach of evaluating existing libraries? A couple of hours searching could save you days, weeks or months of work, and libraries written and used by several people will undoubtedly be of higher quality as well - it should be a no-brainer.

It's easy enough to conjure up subjective justifications for it on a personal level, but to my mind they rarely hold up to scrutiny. Reading and understanding other people's code is said to be harder than churning out some according to your own vision, but surely the depth of understanding required to use a module should surely be substantially less than it would take to write it? Likewise, curiosity, obsessiveness, a sense of personal achievement play a part, but these motives are deeply subjective, and cannot be used to argue the case that 'rolling your own' is an efficient or rational decision, merely an enjoyable (or compulsive) one. Expectations of quality also contribute - everyone thinks that their own code is better. But this is rarely really the case, and experienced practitioners recognise it.

On a deeper level, then, why does some part of us allow these inadequate justifications to hold sway? Even if rationally, we know it doesn't make sense, why does it feel so desirable? What I'm really asking is, why have we been evolved to want to build our own, even though it is so clearly the wrong thing to do?

What if we are tuned, as individuals, to prematurely settle on our own idiosyncratic approach? The net result would be that, for any given problem, a flurry of individuals would each set out in different directions, regardless of what results others might have already achieved. Viewed as a group activity, this starts to sound a little saner. Such a dispersion of individuals would rapidly and thoroughly explore all the corners of the problem space. Compared to the situation where we all just build cumulatively on what everyone else has done before, the chances of any one individual finding an optimal answer is low, but the chances of someone in the group finding it is greatly increased.

It's a survival-of-the-group trait, that each of us is evolved to pick a wrong-looking answer, and then expend ourselves beavering away to prove it. Oh dear indeed.