After Sinister Ducks, my videogame creation mentor suggested that I create a mod for an existing game, in order to distance myself a little from the programming aspects of creating a game, and instead spend a little time considering the gameplay and the art and the music from the perspective of the user. Sounds like useful advice.

So, the last couple of weeks I've been working through the tutorials for DromEd, the notoriously cranky level editor for vintage sneak-em-up Thief: The Dark Project. (Specifically for the sequel, Thief 2: The Metal Age, which has a slightly improved engine and editor.) I chose this for three reasons:

1. Released in 1998, Thief is old enough that the assets are simple low-fidelity geometry and bitmaps. These are easy enough for me to create and edit, plus if I intersperse existing game assets with my own shoddy creations, there won't be a tremendously jarring disparity in apparent quality.
2. Even though Thief is ten years old and the company that created it long gone, there's still a thriving community of amateur afficionados, churning out new missions at the rate of several per month, many of which are of exceedingly high quality - in some cases exceeding that of the original game. I'll be in good company, will have some meaningful feedback, and will have forums to turn to when I get into difficulties.
3. Last but not least, Thief is one of my favourite games of all time. The emphasis on sneaking around and avoiding confrontation suits my sensibilities. Your protagonist, Garrett, is a marvellous, mercenary character. Best of all, in Thief 1, it reveals unexpected depth halfway through - the player's expectations of a succession of simple heists takes a strange twist when the powerful storyline reveals itself.

Having finished the tutorials, this weekend I broke ground on creating my own level, or 'fan-mission' (FM), in the parlance:

I'm using the DromEd Toolkit, which is DromEd with some third-party patches and bugfixes applied to it. My first impressions with DromEd are that it's very clunky and ugly, and startlingly lacking in documentation. I've taken to dipping into the configuration files to see what keyboard commands exist to experiment with. There are a bewildering variety of binary patches to modify the executable in various exciting ways, and forum posts about it, although helpful and prolific, seem fragmentary and rife with broken links. I've still no idea whether I ought to be using Dromed Delux instead, nor where I should get that from. It's a glorious chaotic riot, and it's a little intimidating.

Still, having said that, the binary patches have all worked fine for me, and the more I use the editor, the more it's starting to grow on me.

I'm setting my FM in a museum. There's already an existing museum mission out there, but as opposed to its marble-halled austerity, I'm imagining this will be more like the cramped, cosy, wood-panneled chaotic collection of something like the wonderful Sir John Soane's Museum in London.

So progress as of week 1 looks like this: I've carved out some very basic geometry to form a stocky museum building. Here you can just about make out a hole in the brick facade that will form the front entrance. This is not the entrance that the player will likely be using:

This is all very crude thus far - with repeating textures on large surfaces, and plain uniform lighting. There are a complete set of mostly rectangular ground-floor rooms, with interconnecting doorways. The highlight of my modelling to date is this stairway leading up to the (otherwise nonexistant) next floor.

I'm not so happy with the wood texture I chose - I'll go back and look for something more uniform. But I am happy with the way the stairs flair out at the bottom. I realised in the process of creating this that this makes it possible to fit a flight of stairs into a smaller space than would otherwise be possible, by allowing the bottom few steps to gracefully project out into the corridor.

Having finished this last night, I then dreamed about geometric operations on three dimensional spaces, which I think is a good sign.

I left this as a comment somewhere else, in response to someone complaining about how:

"...musicians are asked to just get over the fact that no one pays for music... [it is] my hope that someday more people make the same realization that unless people to help finance records, you'll never get that fantastic song you can't stop listening to all summer."

Hey there. I really disagree that the onus is on the rest of society to solve all musicians' problems. People have been copying my work on the internet for decades now. I'm a computer programmer. We're in exactly the same boat as musicians - probably more so, and have been there for longer. But we manage to get by.

Some programmers attempt to stop copying of their work, but this is rarely successful. Others take a different tack. We've had to adapt. We acknowledge that our work will be routinely pirated on the internet, and we adopt our business model accordingly. We have to make and sell different sizes and types of tools than we were accustomed to making, and sell them through different channels. We have had to find other ways of charging for our work. My own company gives away our software for free. We embrace piracy as inevitable, and use it as a promotional tool.

This is difficult. It is counter-intuitive, grounded as we all are in the scarcity economy of the physical world. It requires us to change our attitudes and outlook and behaviour. Sometimes we fail. We are still figuring out the realities of our markets in a post-internet world. But goddammit we are trying and sometimes we try things and they work. The software business is more diverse and thriving than it ever has been - despite all our work being just as freely copyable (and freely copied) as music.

The financial hardship that musicians are feeling has always been felt by musicians. Copying music on the internet has not made this any worse than it always was. This is simply the cost of choosing to be a musician.

Musicians must adapt, just like programmers have. It saddens me to see so few musicians even try. Is it really true that computer programmers are more flexible and adaptable and imaginative than musicians. Who would have thunk it?

Embracing piracy as a promotional tool can allow bands to make more money on live shows than they have ever made selling records, but still, the majority of musicians sit around, complacent, thinking that the responsibility lies on the rest of society, to put things back the way they were before the internet.

No. I for one refuse. Adapt or die - if you can't make it, someone else will take your place. By definition, that someone will be more adaptable, imaginative, risk-taking and dynamic. Isn't that supposed to be what our creatives are good at?

People will doubtless think I'm being harsh - imposing my tyranny on struggling musicians. But I'm not. When I say "adapt or die", I'm not imposing my will. I'm merely reminding you of reality. If we were talking about a guitarist who couldn't play the guitar, we would all agree that they have put themselves in a bit of a paradoxical situation, and that they must either adapt (learn to play) or die (stop being a guitarist.)

This is how competition works. Competition might seem harsh, but it's best for everyone in the long run. The poor guitarists are eventually persuaded by market realities to stop being guitarists and try being something else. The good guitarists get rewarded, and the standard of music available to the world is increased as a result.

Instead of untalented guitarists, today we are talking about musicians who would like to get paid, but who don't want to expend any energy on figuring out how they will get paid. Can you see the paradoxical situation they have put themselves in? This attitude is entirely understandable and human - I also would enjoy locking myself in my room and making beautiful computer programs which nobody else need ever see. But if I expect to get paid for making computer programs, then I must interface with the reality of the world around me. It's no use me simply wishing that High St stores will start selling my software in shrinkwrapped boxes again - those days are long gone. Instead, I must figure out what programs people want, and produce them in a timely manner at high quality, promote them somehow, and figure out my angle on how I'm going to get people to pay.

It is not too much for us to ask musicians to also have to conform to reality like this. If they want to make money, and the means of making money from music are changing, then they will have to expend a little energy figuring out how they are going to get people to pay. If they don't want to think about that at all, then fine, they won't get paid. Adapt or die, we'll all be better off for it.

"God dammit. Why won't you just DO what I WANT you hopeless pile of crap!"

So goes the refrain. I think you know where I'm coming from. Yet again, I have ended up learning far more about crappy DOS scripting than I ever wanted to know.

So I'm writing a program to automate some small task on Windows. One of the jobs of this tool is to modify the current environment. But I don't know how a child process in Windows can modify the environment of it's parent (namely the command-line shell that invoked it.) Can it be done?

So I hack a ghastly workaround: Wrap the script in a .bat file. A .bat file is invoked from the command-line in the same process as the shell, so any change it makes to the environment are made to the environment of the invoking shell itself. This also has the advantage that the tool can now be invoked by typing 'toolname', just like on other platforms, as opposed to 'toolname.py' or even 'python toolname.py'. So I wrap my Python script 'toolname.py' with a new file, 'toolname.bat', living in the same directory:

:: first run our tool
python "%~dp0%~n0.py" %*

:: then make any changes to the environment
set THIS=THAT

The %\~dp0 and %\~n0 gobbledygook is a batch file way of referencing the same drive, path and filename (minus extension) as the current script, to which I add '.py' to run toolname.py. Easy enough.

There's a minor problem: The environment changes that need to be made depend on what goes on inside toolname.py. So I have that Python write a new batch file to the temp directory, containing all the 'set' commands which will replace the hardcoded 'set THIS=THAT' in the above script. Then we call that new temporary bat file from here:

:: first run our tool
python "%~dp0%~n0.py" %*

:: then make any changes to the environment
call %Temp%\%~n0-setvar.bat

Which is good enough. Presumably it will barf all over the place if run concurrently. But there's a more pressing problem. I need the exit value of this tool to be equal to the exit value from toolname.py. Currently, the exit value of this .bat script is always zero, because the 'call' command at the end is always successful.

One solution I've seen used is to remember the exit value from toolname.py, and then use the DOS exit command to propagate this value out to our caller:

:: first run our tool
python "%~dp0%~n0.py" %*
set EXITVAL=%ERRORLEVEL%

:: then make any changes to the environment
call %Temp%\%~n0-setvar.bat

exit %EXITVAL%

The problem with this is that 'exit' doesn't do what you think it does. It doesn't just stop interpreting the current script, rather it terminates the current interpreter, ie. the shell that is running the script. If you run this from a command-line, since Windows doesn't differentiate between a console and a shell, your window disappears. Sigh.

The exit command has a fix for this: It takes a switch '/B', that causes it to just end the current script, rather than killing the shell. But now, it ignores any %EXITVAL% parameter you try to feed it, so the exit value of your batch file is always zero.

This is what I get for developing software on Windows. Nothing ever works the way it ought to. It's as though everything were designed to oppose simple engineering idioms, like composing systems out of small, interchangeable parts.

So here's what I finally did. The exit value of running a batch script can be set without using the hopelessly brain-dead 'exit' command. It is equal to the exit value of the last process the script invokes. So instead of exit, simply find a process that will exit with the value you need, and invoke it as the final command in your batch script:

:: first run our tool
python "%~dp0%~n0.py"; %*
set EXITVAL=%ERRORLEVEL%

:: then make any changes to the environment
call %Temp%\%~n0-setvar.bat

:: and propagate the exit value to our invoker
python -c "import sys; sys.exit(%EXITVAL%)"

Bingo. I now have a Python process that can modify the environment of its invoking shell, and propagates the correct exit value out of the wrapping DOS script. [Short bow. Applause. Roses. etc.]

I know, I know. But I use Windows for 8 hours a day at work. So what do you do?

Here's the poster-boy for programmer-friendly fonts, your friend and mine, everyone loves him, Inconsolata:

I don't know about you, but to me that looks unutterably crap. Wonky and irregular. On the right of the orange line is what it looks like with ClearType turned off. Meh. Presumably I'm doing it wrong, somehow, but unless I figure out how, Inconsolata can fuck right off.

So then I fire up the trusty fallback, Consolas:

This is better, certainly, but the letters look cramped. For the number of visible lines in a window that size, the letters are awful small. Again, on the right of the line is ClearType turned off - which in this case distinctly worse.

So I continue peering at a succession of monospaced idiocy, eventually ending up on DejaVu Sans Mono:

This still isn't perfect, but it's the best I can find. Note that it bears out my impression of Consolas being cramped - even though the letters are substantially larger, we still manage to fit an extra line of text in. This time, turning ClearType off (right of the line) makes it a little crisper, and a little more wonky, but not much.

This isn't just in Vim. It looks like this in all applications. I've tried running the ClearType Tuning Powertool, to no avail. Admittedly, all these fonts start to look a damn sight better when I increase the size a few notches. But that's bugger all use really, isn't it? Maybe I should be digging out some mono fonts designed especially to be viewed at small sizes? Oooh, now that's actually not a bad idea: How about those Android fonts everyone's banging on about? Droid Sans Mono:

This isn't bad, but does contain all the old 'zero vs upper-case O' and 'one versus lower-case L' ambiguities. I think I'll stick with DejaVu Sans Mono.

Does Inconsolata work OK on Windows for everyone else?

Update: Gerry suggested trying out Proggy Fonts (thanks!). These are bitmap fonts (unless you want to do without extended characters) so they won't scale. Nevertheless, here's how some representative fonts from there look. First up, Proggy Clean slashed zero:

I love the crisp clarity of a pixel-perfect bitmap font with no scaling or anti-aliasing. On the downside, I have a feeling that the characters' shapes aren't quite as well-formed and beautiful as DejaVu. Not sure whether or not this would bug me. Next up, Proggy Opti:

Color me impressed. Opti still has the wonderful pixel-perfect clarity of Proggy Clean, and manages to fit four extra lines of text into the same sized window. Obviously in order to achieve this, the characters are smaller than Deja Vu, but this is no bad thing. I would have liked to scale Deja Vu down a tad from the size you see it above, but if I try, the letters start to become a little distorted and indistinct. I might well give Opti a spin for a few days, see how it wears on me. Thanks for the suggestion Gerry!

Update: Nowadays I use Dina, a monospaced bitmap font created by some obsessive with a penchant for microscopic fonts. It's absolutely perfect.

A few months ago, Glenn and Christian and Menno and myself created Sinister Ducks, a game with some quacking, for PyWeek 9, a contest to write a game in Python in a week. It plays a little like a simplified version of the arcade classic Joust - press fire to flap. When birds collide, the highest one wins, while the lowest one sheds feathers and plummets to the ground.

Since the competition, I've been polishing and refactoring for my own edification in odd hours here and there, and I'm pleased to declare that process finished. Behold, Sinister Ducks 1.0.2:

Things I added since the competition include:

• Fixed the game being so ridiculously easy that one could play it interminably
• There are now lives, and an actual Game Over screen and everything
• Completely reworked how the gameplay works in regard to attacking ducks and collecting feathers - our previous mechanic didn't work very well. The feathers now temporarily boost a score multiplier, displayed top-left. Attacking other birds yields more points for successful chained consecutive attacks, without collecting any feathers along the way. But you can only get feathers in the first place by attacking birds. It's your job to figure out the resulting best-scoring dynamic.
• Understand and enjoying the scoring dynamic is aided with little floating numbers showing the value of each bird you defeat.
• Enemy birds now come in larger waves as time goes on, and one or two of the sinister ducks are angry and fast.
• The game speed is now scaled by the time between refreshes, so it now runs perfectly fine on slow machines or VMs that can't make 60fps, or if your monitor has an unusual refresh rate, or even if you disabled vsync in your graphics drivers to yield hundreds of frames per second.
• Refactoring and OpenGL performance fixes (those sprites are texture-mapped quads, under the covers.) so that it's now massively faster than it should ever need to be. The refresh rate with vsync disabled on my own 5 year old lappy is about 400fps.

Repo at https://github.com/tartley/sinister-ducks/.

From there, you can download a Windows 'release' to download, unzip and double-click, without having to mess about installing Python first. Or a source release, which will run on Linux or Macs, but you'll have to download Python yourself, and possibly install the dependencies manually too, sorry.

This is a quick post in response to the Vimeo video post Getting Rid of White Halos in Transparent Textures by FlashBang Studios (creators of the awesome Offroad Velociraptor Safari).

Thanks to FlashBang for the video - the fix you describe, and the plugins you use to achieve it, are good to know about. If I'm interpreting the cause of the white haloes you describe correctly, then your fix applies to more than just Unity development.

Also, you might want to know that there is an alternative solution to this problem. This alternative solution requires small code changes, and is possibly slightly trickier to get your head around, but possibly has a small advantage in that it requires no ongoing effort by the artists (e.g. applying and undoing plugin actions to color transparent pixels) once in place. I don't use Unity, so this might not apply in that environment, but just in case, here goes:

The alternative solution is to save textures using light-opacity RGBA (a.k.a. pre-multiplied alpha), and then blend them in the engine using a more appropriate OpenGL blending mode. I learned about this from Brian Fisher's post on the pylget forums. I aim to paraphrase his explanation of it here in order to cement my own understanding of it.

To elaborate, as I understand it, the issue is caused by the bilinear interpolation between the RGBA of your transparent pixels, and your icon texture color. For example, for various transparencies of purple on top of a blue background:

color	purpose	r,g,b,a	color description
	icon	(1, 0, 1, 1)	fully opaque purple
	interpolation	(1, 0.5, 1, 0.5)	half-transparent whitish-purple
	background	(1, 1, 1, 0)	fully transparent white

The intermediate color isn't actually very close to white, but in contrast to the pure purple it abuts against, it's significantly paler, and causes the 'white looking halo' effect around the edge of the purple icons, as shown in the video.

Note that changing the color of the transparent pixels, from white to black for example, does not fix the problem - it just changes the color of the haloes from white to black.

The video shows how to fix this, using a technique that I infer is very common - change your transparent pixels to be the same color as whatever colors they abut against. The background pixels are now fully-transparent purple, giving us:

color	purpose	r,g,b,a	color description
	icon	(1, 0, 1, 1)	fully opaque purple
	interpolation	(1, 0, 1, 0.5)	half-transparent purple
	background	(1, 0, 1, 0)	fully transparent purple

So now the interpolation ends up being half-transparent pure purple. This eliminates the white halo artefact, as demonstrated in the video. The only minor downside is that the artist has to make sure the transparent pixels are saved with the correct RGB values, and the plugin demonstrated in the video helps to automate this.

The alternative solution, is to modify the RGBA values in the texture, by pre-multiplying the RGB values by the A value. ie:

R = R * A
G = G * A
B = B * A

(where all values range from 0.0 to 1.0)

This can either be done in the saved texture file (eg. 3d Studio Max writes TGA files this way.) Or, if you'd rather continue to use standard RGBA values in your saved texture files, then your application could do this transformation when loading image files from disk.

To use these alternate RGBA values in a texture, we have to display them on screen using a different OpenGL blending mode. We switch from using the traditional:

dest = src * alpha + dest * (1 - alpha)

to using this instead:

dest = src + dest * (1 - alpha)

Note how the difference between the two is that the src is no longer multiplied by the src alpha during blend. This is no longer required, because we have already done this multiplication in the RGBA of the src texture itself.

Storing RGBA values in this format is known as light-opacity RGBA, and it gives us a new interpretation of the meaning of the RGBA values. Previously, the RGB told us the color of the texture's pixel, but did not tell us how much this color would be used to affect the destination. We have to look at the alpha to determine that.

Now, using the new blending mode, we can interpret the RGB values to literally mean how much R and G and B is added to the destination. Similarly, the A value is now only used to diminish whatever existing color is currently at the destination. It is not used to scale the source RGB before applying it to the destination. The A value can now be interpreted to mean 'what proportion of the existing destination color is visible through the overwritten source color'.

This implies that for pixels to be fully transparent, ie to have no effect on the destination, they must have an RGB of 0. So we must always use (0, 0, 0, 0) to represent fully transparent pixels. Note how with the new interpretation of RGBA, these are not 'black' transparent pixels, they are literally 'no-color' transparent pixels. This obviously has a neat logical correposondance to what 'pure transparency' really means.

The pixels around the edge of the purple icon still get the same interpolation process, resulting in the same RGBA values as would have been calculated before, but now we have a different interpretation of what those values mean:

color	purpose	r,g,b,a	color description
	icon	(1, 0, 1, 1)	pure purple that fully overwrites the previous color
	interpolation	(0.5, 0, 0.5, 0.5)	half-bright purple mixed with 50% of the prev dest color
	background	(0, 0, 0, 0)	fully transparent (no color)

The interpolated color is now blended with the previous destination color using the new OpenGL blend mode:

dest = src + dest * (1 - alpha)

The existing destination color is scaled by (1 - alpha), ie, is reduced to half intensity, and then the RGB from our interpolated texture (0.5, 0, 0.5) is added to it. This results in a half-transparent purple - even though the transparent pixels in our texture had no purple at all in them. This results in a perfect blend between the transparent and pure purple colors. No haloes are visible.

Update: It crosses my mind to mention that the halo artefacts will be visible whenever textures are scaled, rotated, or positioned using sub-pixel co-ordinates. If you only ever display unrotated textures, at integer pixel co-ords, and at normal size, then you won't run into this problem. Since the icons on the radar on FlashBang's video don't seem to be scaled or rotated, maybe they could sidestep the problem by just displaying the icons at integer pixel co-ordinates? I don't really know anything about how their code works though, so I might be talking nonsense here.

An open letter I have sent to the ACM.

I have recently been considering joining the ACM, as the most eminent international professional body of software engineers.

On reflection I have decided not to join, because of two actions by the Association in recent weeks that appear strongly contrary to the best interests of its members, and indeed contrary to the interests of all software engineers and scholars of every discipline.

I am referring, firstly, to the shameful take-down requests issued to Ke-Sen Huang, and subsequent conditions imposed over his meticulously assembled pages of links to SIGGRAPH and other academic computer graphics papers. http://kesen.huang.googlepages.com

The second act is the recent official comment from the ACM on the White House's public feedback to expanding open access to scientific papers. This comment is to the effect that the ACM resists the move towards making scientific papers available to all. http://blog.ostp.gov/2009/12/10/policy-forum-on-public-access-to-federally-funded-research-implementation/comment-page-3/#comments

Both these actions bore the signature of the ACM's Bernard Rous, who has the unbelievable gall to suggest he opposes the creation of a repository of academic papers, because it would undermine the ACM's revenue from keeping such papers behind a paywall, and that this is a legitimate interest which must be 'balanced with' the interest of the ACM members desiring access to those papers.

Mr Rous perhaps needs reminding that the ACM's has *no such* legitimate interest here. The Association's entire existence is soley contingent upon the benefit that it provides to its members. These actions seem to me to be in flagrant violation of the ACM's own Code of Ethics (http://www.acm.org/about/code-of-ethics) and worse, it seems to be contrary to the ACM's entire charter and purpose.

I have no doubt the ACM performs many valuable services which incur substantial costs, and these costs need to be offset by exploring methods of generating revenue. But this is not an acceptable method. It costs nothing to make these papers available online - and the ACM has no business opposing the establishment of free learning resources that would compete with its own offerings. On the contrary, the ACM is not a profit-making organisation. Its charter to promote learning and education amongst its members, and this can only be best served by promoting and facilitating the free and open access to academic papers such as these.

It seems clear to me that if the ACM does not effect an immediate and total reversal of its policy on these matters, then the Association has changed from the illustrious body it once was, into a force for ill, pissing in the well of knowledge, to be actively opposed by professional, moral software engineers everywhere.

The background to this issue is most clearly expressed here: http://www.realtimerendering.com/blog/us-gov-requests-feedback-on-open-access-acm-gets-it-wrong-again/

I would urge others to boycott any further membership payments to the ACM until this is satisfactorily resolved.

Sincerely,

Jonathan Hartley
Software Engineer

So earlier tonight I turned up at the monthly 'surgery' of my MP, Frank Dobson (Labour), to present my objections to the Digital Economy Bill.

For the benefit of others also going to tackle Mr Dobson on this issue, (and anyone going to see some other MP) here's a quick write up of my experiences:

I arrived up half an hour before it started, and there were about eight parties ahead of me in line. My turn came half an hour after they kicked off. I had hoped to present my argument pretty much as I laid it out in my previous blog post. However, sadly I pretty much comprehensively failed to get my point across.

The 'I am not a spittle-flecked loon' introduction

As planned, I started out by saying that I fully support copyright law, and of the importance of finding ways to ensure that artists get renumerated for their work. The portions of the bill which describe the pursuit of illegal downloaders therefore have laudable and important goals, however I think that the costs and problems associated with preventing illegal downloads like this are substantial, and actually make this section of the bill counter-productive. Mr Dobson was an affable gent, and listened attentively to what I had to say.

Disconnection needlessly punishes more innocent people than guilty ones

The first objection I wanted to raise, because it seems to me to be the most clear-cut, is that the bill suggests disconnection from the internet for persistent downloaders. I pointed out that since most internet connections are shared between several people (families and housemates) disconnecting a single illegal downloader would also unfairly affect all the people who share their connection. Mr Dobson's opinion was that it would rarely come to actual disconnection - the intention of the Bill is very much that disconnection would be a rarely-used last resort. The earlier provisions in the bill (warning offenders for their first 'two strikes') are intended to stop almost everyone from illegally downloading, without any need for actual disconnections to take place.

Obviously I regard that as a grave misjudgement. But I had a lot of ground to cover, so I left this issue at that. I still feel that even if disconnection were only applied to a small number of people (which seems unlikely, from my perspective), it is still true that this would affect more innocents than guilty parties. There is no reason not to prefer more conventional punitive measures over disconnection.

Impossibility of distinguishing guilt from innocence

I moved onto my second point: When monitoring downloads, it is impossible to distinguish between guilt and innocence. If I download a song I own, from my home to my office, this behaviour may currently be legally murky, but it is clearly morally justifiable. To any external observer, this looks indistinguishable from illegal download. An observer cannot tell whether a particular download is illegal or not just by looking at the bits going down the wire - it requires more context than that. It requires the observer to know which CDs I have bought, and to what use I intend to put the proceeds (is it subject to fair use exemptions, etc?) Therefore, along with many illegal downloaders, the system proposed in the bill will also catch many innocent people in its net.

These innocents can appeal - there is a procedure in the bill for this - however they will find it impossible to demonstrate their innocence. If I have bought a CD and ripped it to my computer (which I have done with all my CDs, many years ago) then when I am accused of illegally downloading it, how am I supposed to demonstrate that I once bought that CD?

Mr Dobson listened with patience and good humour to this. However, it was clear from his responses that while he is a capable man who is working hard and with the best of intentions, the scenario I outlined was fairly foreign to his experience. He suggested that the bill is not aimed at catching people like me who are doing nothing wrong. It is aimed at people who are making a dishonest living by massive downloads which they are somehow making money from. I don't believe that any such people actually exist, never mind that the bill is aimed at them: The wording of the bill clearly encompasses individuals downloading music for their own listening pleasure, and deliberately so, but I did not go into this.

Mr Dobson felt that the appeals process would be sufficient for anyone such as my self to protest their innocence. I totally failed to communicate that demonstrating my innocence during such an appeals process would be very difficult. When I reiterated that I could not, for example, produce the receipts to all the music and movies I have ever bought in my entire lifetime, Mr Dobson reiterated that the bill was not aimed at innocent people like me, it would only be applied to illegal downloaders. I again said I don't see how the ISPs can distinguish between the two - for example I cannot produce the receipts nor the CDs that my music came from - I ripped them in 1998, and have since discarded them all. Mr Dobson said that perhaps when the bill become law (my emphasis), people will be obliged to keep their CDs "in a plastic bag in the garage", in preparation for just such a scenario. I pointed out that all of my music for the last few years has been purchased online - so never came on any CDs to begin with.

Mr Dobson didn't seem receptive to this argument. He was still in good spirits at this point, however, I totally failed to persuade him that difficulty in distinguishing between guilty and innocent downloaders was even a realistic possibility, never mind that it would actually be a serious problem.

I had hoped to move on from this point to demonstrate that this presumption of guilt that the bill contains will have a severe chilling effect on people's legitimate and morally justifiable uses of the internet, and this will hamper the development of business models, which will be presumptively quashed before ever getting a chance.

However, Mr Dobson suggested that I should send him the remainder of my points to him in a letter. I asked him if I could just spend 30 seconds skipping through the executive summary of the remainder of my argument, since I felt there was a logical chain of inferences to follow, which demonstrated some real problems with the bill. He suggested that I write it to him in a letter instead.

Probable ten or fifteen minutes had elapsed. We exchanged cordial pleasantries and headed out for a lovely curry down the street with my awesome wife.

This is pretty much a riff on what I posted yesterday, only this is my draft for how I intend to present it to my MP tomorrow. I have no idea how much time I'll have, nor how receptive he'll be. This may all be a big waste of time, but just in case, this is my plan, such as it is.

If you think any part of this is idiotic, or that I've misunderstood how the relevant bits of the Bill work, or that I'm belabouring some points at the expense of other, more important ones, then I'd really love to hear about it. The last thing I want is for some stupid part of my argument to undermine the other parts of it.

The Digital Economy Bill

http://www.publications.parliament.uk/pa/ld200910/ldbills/001/10001.i-ii.html

I want to clarify at the outset that I absolutely support the importance of copyright law, its enforcement, and give my full support to the expectation of artists to be remunerated in a market-driven manner. I do not want the following arguments to be confused with those which suggest that copyright should be abolished, or everything online should be free.

(This bit is my attempt to establish some credibility and rapport. In reality, I think copyright, although sound in principle, is sorely in need of substantial revision, namely it should be much shorter in duration, should have larger and more explicit exceptions, and should require pro-active registering and renewal by the rights holder. But those concerns are orthogonal to the Digital Economy Bill in its current form, and raising them here would only confuse the issue.)

As a computer programmer, I am directly affected by illegal downloads. Almost everything I have created professionally has subsequently been available online for illegal download - affecting my own income and the financial viability of my employers, such as my current small technology start-up. (This is not entirely true - some military projects I have worked on have not suffered any piracy, due to having, shall we say, extravagant hardware requirements)

However, even with that background, I still have strong objections to the sections of the bill 4 through 17, which attempt to redress the harm done to the creative industries by the elimination of illegal downloading.

I believe this pursuit of illegal downloaders comes with problems that are so profound as to make this portion of the bill counter-productive - and I believe that a superior alternative course of action does exist, that would be of substantially more benefit to artists.

The problems as I perceive them are:

1. Disconnection affects many innocents.

Regarding the Obligations to Limit Access described in section 11.

Almost every internet connection is used by more than one person. For example family members, or independent parties in a shared house.

Disconnection of persistent illegal downloaders will therefore impact all these other users of the shared connection. The innocent users disconnected by this legislation will greatly outnumber the illegal downloaders being targeted!

This similarly also applies to the other, lesser remedies described by section 11, of affecting service speed or access to particular materials.

This is flagrantly unfair - a whole household of innocent people are being punished because of the actions of one guilty person.

Infringing downloads may not even have originated from a resident of the house. They could easily have been made a dinner guest - very possibly without the resident's knowledge, and yet they are to be collectively punished for it. Are hosts expected to prevent this by searching every guest to their house, demanding guests hand over their phones and other devices?

This idea is equally problematic when considering publicly available internet connections, such as at libraries, airports, coffee-shops, etc. Are they to be disconnected when one of their customers infringes?

My conclusion: Disconnection and the other remedies described in section 11 are not, under any circumstances, appropriate courses of action, even for the most egregious offenders, and more conventional remedies should be considered instead.

2. Impossibility of distinguishing guilt from innocence

Several times a week, I routinely download many dozens of online media from my home collection, to wherever I happen to be at the time - the office, at a conference, travelling overseas. These are on the whole media I have previously bought - music or movies or computer programs or games. I have ripped these legally purchased items to my computer precisely so that I would have the freedom to move them geographically, and to format-shift them from one device to another.

To any external observer, these downloads are indistinguishable from an illegal download. Copying of the exact same songs is involved in each case. The external observer cannot know that I have already bought the music or movie or whatever in question.

If I were accused of illegal downloading under this law, which I apparently would be on a very regular basis, then would have no way of defending myself against these false accusations. I cannot produce proof of purchase for all of the media I have purchased over my whole lifetime.

If it came down to producing the physical media as evidence, such as CDs my songs came on, then of course many of my songs were purchased online, and the CDs I bought years ago were discarded as soon as I had ripped them, in celebration of my freedom from the constraints of physical media.

Similarly, my accuser, upon whom I believe the onus of proof should lie, cannot prove the negative (that I never bought the media in question.) So it is impossible to distinguish the guilty from the innocent, and

3. Chilling effect

The presumption of guilt inherent in the Bill will clearly have a chilling effect on my personal use of the internet as described above. The raison d'être of the internet is to facilitate the copying of information from one place to another, and yet my perfectly innocuous desire to use it for exactly that is, in this instance, prevented for fear of me being tarred with the same brush as illegal downloaders.

If this were the full extent of the chilling effect that the bill will have, then it is arguable that this is a regrettable but acceptable collateral damage in the struggle to recompense artists. However, this is not the only action that is chilled by the Bill's presumption of guilt.

TODO, this section is weak, and represents what I think is the most important problem. At the very least, more compelling examples are needed here

Artists exchanging material while collaborating on a project, or or someone downloading music that they themselves wrote, or business model that involves copying media from one place to another, all would be under risk from invoking the penalty of 'illegal downloads', and will be prevented from occurring.

Personal freedoms and business models of both the present and the future will be inhibited by the presumption of guilt that the pursuit of file-sharers requires.

This is, to my mind, the most serious of all the problems with the Bill's pursuit of illegal downloaders. Instead of stimulating the digital economy, these measures seem to me to stifle it, by pandering to the wishes of existing special interests, at the expense of personal freedoms and the emerging business models of the future.

4. No provision to clarify the legality of innocent use

It is disappointing that the bill does not take the opportunity to make actions which are both common and morally justifiable clearly and explicitly legal.

I am thinking here of actions such as my space-shifting of the files I own, as described above, and of format-shifting between different devices.

These actions do not in any way represent a valid 'lost sale', and have absolutely no negative effect on the content industry's business, other than countering unreasonable attempts to force consumers to purchase the same media several times over.

Removing the uncertainty about these actions would free individuals from the lurking threat of frivolous prosecution over harmless activities, and would foster innovation by giving business some guidelines about exactly what is allowed.

5. Intrusive state monitoring of everyone's online actions

I strongly object to the notion of the ISPs, acting as representatives of the State, monitoring every single action that I and my family and my friends carry out online. It is ripe for abuse, for the information falling into the wrong hands, and for leaving me under suspicion due to clerical or other errors during the collection or storage of the data. It is absolutely unacceptable to me.

6. Lack of substantive evidence that artists income has really dropped

As stated initially, the goal of ensuring artists are suitably remunerated for their creations is a laudable one, that I fully support.

Crucially though, the purported losses incurred by the music and movie industries are commonly derived from flawed studies, directly funded by the industries in question, and frequently make fundamental mistakes such as equating every download made with a lost sale. More substantive information than this is required about the size of the problem.

Indeed, some figures show that the music industry has in fact grown very healthily in the last few years. For example, these figures compiled and published by The Times on 12th November 2009, based on the BPI's own data:

http://labs.timesonline.co.uk/blog/2009/11/12/do-music-artists-do-better-in-a-world-with-illegal-file-sharing/

This suggests that since 2004, recorded music incomes in the UK have fallen by some £326M. Whether this is due to illegal downloading is not known - presumably it is partly due to that, and partly due to other factors, such as the music industry putting out fewer records in recent years. However, the same figures also show that over the same period, income from live music and merchandising have risen by the larger amount of £373M. The industry as a whole is around £47M better off, and as a bonus, the proportion of this income which is paid directly to artists has risen dramatically, since artists make more money from live shows than from recorded music sales.

This fits with my own personal experience - free downloads act as a tremendous promotional boost for bands. I have many friends who are able to try out far more music as a result of its availability online, and as a result are far more active in attending live music events, often of bands they would not otherwise have heard of, had they not discovered them online.

If this is true, then free downloads cause artists to lose money via recorded music, but also to gain more in recompense through other channels. In this case, do we really need heavy-handed legislation to protect their interests, if the legislation in question comes with all the other costs described here? If artists really are earning more than they were five years ago, do we need to introduce the chilling effects this bill will cause on the rest of the economy in order to try and help them out? What if the pursuit of illegal downloaders means that revenues in live music then drops back to their previous lower levels? Artists will have actually been financially hurt by this bill. This needs to be understood before taking such drastic actions as the bill proposes.

7. One alternative

TODO: Lay out the "central pool to remunerate artists in proportion to their number of downloads" idea, merely to make it clear that there are alternatives courses of action which could be used to tackle this situation, which seem to come with real incentives to stimulate the digital and creative economies, and with smaller, more soluble problems than Digital Economy Bill in its current form.

So my splendid talented intelligent handsome friend Robert (who is also my boss, of sorts) today shared this smashing post:

My \$62.47 Royalty Statement: How Major Labels Cook the Books with Digital Downloads

About one aspect of how artists get ripped off by the labels. It's not required reading, but you can if you like. I'll wait.

Hey. So it made me realise that a very possible fix to the dreadful abuse of artists by the labels is possible. Even better, the solution to this appalling situation is also the solution to the dire problems with the UK's Digital Economy Bill.

So the UK govt. are debating a new bill, part of which introduces measures to ruthlessly pursue illegal downloaders. There are some problems with this, namely that:

• someone (probably the ISPs) will have to monitor what everyone downloads, and report people downloading anything that trips their filters for 'copyrighted work'. Understandably, they don't want to do this.
• It will cost lots to implement - currently estimated at 100 million, but of course the ensuing arms race with encrypted downloads will inflate that. Has anyone compared this with the actual economic losses that it is aiming to fix? (not just the inflated ones self-reported by those with most to gain.)
• It is literally impossible to automatically detect breach of copyright accurately - it can't be done just by looking at the bits. There will be lots of false positives and false negatives.
• It will be easy to get around, using encryption or similar technologies to evade detection. It will not fix the problem it aims to fix.
• It requires representatives of the state to intrusively monitor every single thing we do online.
• The false positives will target and punish many innocent people, who have no way to demonstrate their innocence.
• Perhaps worst of all, its presumption of guilt will have a chilling effect on many legal and morally justifiable uses of the internet, including business models of the present and the future.
• They are talking about disconnection from the internet as a potential punitive measure for persistent downloaders. Most internet connections are shared by several people, and many of us use each other's connections all the time. Disconnecting one person for something that someone else downloaded is flagrantly wrong, and without wishing to be melodramatic, collective punishment like this is, strictly speaking, in breach of article 33 of the Geneva Convention. Plus it makes free wi-fi impossible to provide. Are we meant to be destroying the digital economy?

If your country doesn't have a Bill like this, rest assured, it soon will have.

Anyhow, it's clear to me that there's no use in railing against the Bill unless one has an alternative to offer. The debate in the House of Lords yesterday made it clear that the mood there is that millions are flouting the law with illegal downloads to rip off artists and Something Must Be Done.

So what's the alternative? The solution is the final chapter from Stanford Law Prof. Lawrence Lessig's book 'Free Culture', that describes how to fix the whole copyright mess in such a way that will make everybody happy. Artists will get paid in abundance, very fairly, in a market-driven way, and the payment will be direct to them. The labels will be eliminated from the loop. No state monitoring of what people download will be required. Everybody will be able to continue to download whatever they like from wherever they like, for free (at point of use), legally and with clear conscience. Best of all, the creation of new business models and grass-roots creativity such as songs and other content will be stimulated through the roof. Whichever country first introduced this would lead the world in a creative tidal wave that embraces the powers of the internet, rather than fighting against them.

I'm not going to write what the copyright fix is here. I have to go to work and I'm still in my pajamas. Oh alright, here's the quick quick version. We add a tax to every high bandwidth internet connection. The money goes in a pot. Then a central agency samples how many of each creative work is being downloaded. This is just a sample - it needn't be tied to any individual downloader, so it is cheap to do and unintrusive. Then we distribute the money in the pot to artists, in proportion to the number of downloads they had.

This quick version has some holes in it. Prof. Lessig ties it all up in his book. Yes it has some problems - taxes are never popular, and more expensive internet connections are not what we want. But these problems are small and soluble, compared to what is currently being discussed instead.

I know this is so starry-eyed idealistic that it has no chance of being seriously discussed. But I decided last night that I have to at least give it a try.

Readers from the UK - Do YOU have even the slightest inkling of interest in suggesting this to your own MP? Or at least pointing out the downsides of the current Bill? It would just be one evening out of your life. I'm visiting mine this weekend. Email me if you have any interest at all. I can lead you through the process, it's simple.