My Fonts Look Crap. I blame Windows and ATI.

I know, I know. But I use Windows for 8 hours a day at work. So what do you do?

Here's the poster-boy for programmer-friendly fonts, your friend and mine, everyone loves him, Inconsolata:

inconsolata-11

I don't know about you, but to me that looks unutterably crap. Wonky and irregular. On the right of the orange line is what it looks like with ClearType turned off. Meh. Presumably I'm doing it wrong, somehow, but unless I figure out how, Inconsolata can fuck right off.

So then I fire up the trusty fallback, Consolas:

consolas-10

This is better, certainly, but the letters look cramped. For the number of visible lines in a window that size, the letters are awful small. Again, on the right of the line is ClearType turned off - which in this case distinctly worse.

So I continue peering at a succession of monospaced idiocy, eventually ending up on DejaVu Sans Mono:

deja-vu-sans-mono-9

This still isn't perfect, but it's the best I can find. Note that it bears out my impression of Consolas being cramped - even though the letters are substantially larger, we still manage to fit an extra line of text in. This time, turning ClearType off (right of the line) makes it a little crisper, and a little more wonky, but not much.

This isn't just in Vim. It looks like this in all applications. I've tried running the ClearType Tuning Powertool, to no avail. Admittedly, all these fonts start to look a damn sight better when I increase the size a few notches. But that's bugger all use really, isn't it? Maybe I should be digging out some mono fonts designed especially to be viewed at small sizes? Oooh, now that's actually not a bad idea: How about those Android fonts everyone's banging on about? Droid Sans Mono:

droid-sans-mono-10

This isn't bad, but does contain all the old 'zero vs upper-case O' and 'one versus lower-case L' ambiguities. I think I'll stick with DejaVu Sans Mono.

Does Inconsolata work OK on Windows for everyone else?

Update: Gerry suggested trying out Proggy Fonts (thanks!). These are bitmap fonts (unless you want to do without extended characters) so they won't scale. Nevertheless, here's how some representative fonts from there look. First up, Proggy Clean slashed zero:

proggy-clean-sz font

I love the crisp clarity of a pixel-perfect bitmap font with no scaling or anti-aliasing. On the downside, I have a feeling that the characters' shapes aren't quite as well-formed and beautiful as DejaVu. Not sure whether or not this would bug me. Next up, Proggy Opti:

proggy-opti font

Color me impressed. Opti still has the wonderful pixel-perfect clarity of Proggy Clean, and manages to fit four extra lines of text into the same sized window. Obviously in order to achieve this, the characters are smaller than Deja Vu, but this is no bad thing. I would have liked to scale Deja Vu down a tad from the size you see it above, but if I try, the letters start to become a little distorted and indistinct. I might well give Opti a spin for a few days, see how it wears on me. Thanks for the suggestion Gerry!

Update: Nowadays I use Dina, a monospaced bitmap font created by some obsessive with a penchant for microscopic fonts. It's absolutely perfect.

Sinister Redux

A few months ago, Glenn and Christian and Menno and myself created Sinister Ducks, a game with some quacking, for PyWeek 9, a contest to write a game in Python in a week. It plays a little like a simplified version of the arcade classic Joust - press fire to flap. When birds collide, the highest one wins, while the lowest one sheds feathers and plummets to the ground.

Since the competition, I've been polishing and refactoring for my own edification in odd hours here and there, and I'm pleased to declare that process finished. Behold, Sinister Ducks 1.0.2:

Sinister Ducks 1.0.2

Things I added since the competition include:

  • Fixed the game being so ridiculously easy that one could play it interminably
  • There are now lives, and an actual Game Over screen and everything
  • Completely reworked how the gameplay works in regard to attacking ducks and collecting feathers - our previous mechanic didn't work very well. The feathers now temporarily boost a score multiplier, displayed top-left. Attacking other birds yields more points for successful chained consecutive attacks, without collecting any feathers along the way. But you can only get feathers in the first place by attacking birds. It's your job to figure out the resulting best-scoring dynamic.
  • Understand and enjoying the scoring dynamic is aided with little floating numbers showing the value of each bird you defeat.
  • Enemy birds now come in larger waves as time goes on, and one or two of the sinister ducks are angry and fast.
  • The game speed is now scaled by the time between refreshes, so it now runs perfectly fine on slow machines or VMs that can't make 60fps, or if your monitor has an unusual refresh rate, or even if you disabled vsync in your graphics drivers to yield hundreds of frames per second.
  • Refactoring and OpenGL performance fixes (those sprites are texture-mapped quads, under the covers.) so that it's now massively faster than it should ever need to be. The refresh rate with vsync disabled on my own 5 year old lappy is about 400fps.

Repo at https://github.com/tartley/sinister-ducks/.

From there, you can download a Windows 'release' to download, unzip and double-click, without having to mess about installing Python first. Or a source release, which will run on Linux or Macs, but you'll have to download Python yourself, and possibly install the dependencies manually too, sorry.

Use light-opacity RGBA to fix halo artifacts along edges of textures

This is a quick post in response to the Vimeo video post Getting Rid of White Halos in Transparent Textures by FlashBang Studios (creators of the awesome Offroad Velociraptor Safari).

Thanks to FlashBang for the video - the fix you describe, and the plugins you use to achieve it, are good to know about. If I'm interpreting the cause of the white haloes you describe correctly, then your fix applies to more than just Unity development.

Also, you might want to know that there is an alternative solution to this problem. This alternative solution requires small code changes, and is possibly slightly trickier to get your head around, but possibly has a small advantage in that it requires no ongoing effort by the artists (e.g. applying and undoing plugin actions to color transparent pixels) once in place. I don't use Unity, so this might not apply in that environment, but just in case, here goes:

The alternative solution is to save textures using light-opacity RGBA (a.k.a. pre-multiplied alpha), and then blend them in the engine using a more appropriate OpenGL blending mode. I learned about this from Brian Fisher's post on the pylget forums. I aim to paraphrase his explanation of it here in order to cement my own understanding of it.

To elaborate, as I understand it, the issue is caused by the bilinear interpolation between the RGBA of your transparent pixels, and your icon texture color. For example, for various transparencies of purple on top of a blue background:

| icon (1, 0, 1, 1) fully opaque purple | interpolation (1, 0.5, 1, 0.5) half-transparent whitish-purple | background (1, 1, 1, 0) fully transparent white

(Update: I think some CSS nuance has been lost in a website migration. On quick glance, this doesn't seem to look like I expect it to, any more.)

The intermediate color isn't actually very close to white, but in contrast to the pure purple it abuts against, it's significantly paler, and causes the 'white looking halo' effect around the edge of the purple icons, as shown in the video.

Note that changing the color of the transparent pixels, from white to black for example, does not fix the problem - it just changes the color of the haloes from white to black.

The video shows how to fix this, using a technique that I infer is very common - change your transparent pixels to be the same color as whatever colors they abut against. The background pixels are now fully-transparent purple, giving us:

| icon (1, 0, 1, 1) fully opaque purple | interpolation (1, 0, 1, 0.5) half-transparent whitish-purple | background (1, 0, 1, 0) fully transparent purple

So now the interpolation ends up being half-transparent pure purple. This eliminates the white halo artefact, as demonstrated in the video. The only minor downside is that the artist has to make sure the transparent pixels are saved with the correct RGB values, and the plugin demonstrated in the video helps to automate this.

The alternative solution, is to modify the RGBA values in the texture, by pre-multiplying the RGB values by the A value. ie:

R = R * A
G = G * A
B = B * A

(where all values range from 0.0 to 1.0)

This can either be done in the saved texture file (eg. 3d Studio Max writes TGA files this way.) Or, if you'd rather continue to use standard RGBA values in your saved texture files, then your application could do this transformation when loading image files from disk.

To use these alternate RGBA values in a texture, we have to display them on screen using a different OpenGL blending mode. We switch from using the traditional:

dest = src * alpha + dest * (1 - alpha)

to using this instead:

dest = src + dest * (1 - alpha)

Note how the difference between the two is that the src is no longer multiplied by the src alpha during blend. This is no longer required, because we have already done this multiplication in the RGBA of the src texture itself.

Storing RGBA values in this format is known as light-opacity RGBA, and it gives us a new interpretation of the meaning of the RGBA values. Previously, the RGB told us the color of the texture's pixel, but did not tell us how much this color would be used to affect the destination. We have to look at the alpha to determine that.

Now, using the new blending mode, we can interpret the RGB values to literally mean how much R and G and B is added to the destination. Similarly, the A value is now only used to diminish whatever existing color is currently at the destination. It is not used to scale the source RGB before applying it to the destination. The A value can now be interpreted to mean 'what proportion of the existing destination color is visible through the overwritten source color'.

This implies that for pixels to be fully transparent, ie to have no effect on the destination, they must have an RGB of 0. So we must always use (0, 0, 0, 0) to represent fully transparent pixels. Note how with the new interpretation of RGBA, these are not 'black' transparent pixels, they are literally 'no-color' transparent pixels. This obviously has a neat logical correposondance to what 'pure transparency' really means.

The pixels around the edge of the purple icon still get the same interpolation process, resulting in the same RGBA values as would have been calculated before, but now we have a different interpretation of what those values mean:

| icon (1, 0, 1, 1) - pure purple that fully overwrites the previous color | interpolation (0.5, 0, 0.5, 0.5) - half-bright purple mixed with 50% of the prev dest color | background (0, 0, 0, 0) - fully transparent (no color)

The interpolated color is now blended with the previous destination color using the new OpenGL blend mode:

dest = src + dest * (1 - alpha)

The existing destination color is scaled by (1 - alpha), ie, is reduced to half intensity, and then the RGB from our interpolated texture (0.5, 0, 0.5) is added to it. This results in a half-transparent purple - even though the transparent pixels in our texture had no purple at all in them. This results in a perfect blend between the transparent and pure purple colors. No haloes are visible.

Update: It crosses my mind to mention that the halo artefacts will be visible whenever textures are scaled, rotated, or positioned using sub-pixel co-ordinates. If you only ever display unrotated textures, at integer pixel co-ords, and at normal size, then you won't run into this problem. Since the icons on the radar on FlashBang's video don't seem to be scaled or rotated, maybe they could sidestep the problem by just displaying the icons at integer pixel co-ordinates? I don't really know anything about how their code works though, so I might be talking nonsense here.

The ACM is pissing in the well

An open letter I have sent to the ACM.

I have recently been considering joining the ACM, as the most eminent international professional body of software engineers.

On reflection I have decided not to join, because of two actions by the Association in recent weeks that appear strongly contrary to the best interests of its members, and indeed contrary to the interests of all software engineers and scholars of every discipline.

I am referring, firstly, to the shameful take-down requests issued to Ke-Sen Huang, and subsequent conditions imposed over his meticulously assembled pages of links to SIGGRAPH and other academic computer graphics papers. http://kesen.huang.googlepages.com

The second act is the recent official comment from the ACM on the White House's public feedback to expanding open access to scientific papers. This comment is to the effect that the ACM resists the move towards making scientific papers available to all. http://blog.ostp.gov/2009/12/10/policy-forum-on-public-access-to-federally-funded-research-implementation/comment-page-3/#comments

Both these actions bore the signature of the ACM's Bernard Rous, who has the unbelievable gall to suggest he opposes the creation of a repository of academic papers, because it would undermine the ACM's revenue from keeping such papers behind a paywall, and that this is a legitimate interest which must be 'balanced with' the interest of the ACM members desiring access to those papers.

Mr Rous perhaps needs reminding that the ACM's has *no such* legitimate interest here. The Association's entire existence is soley contingent upon the benefit that it provides to its members. These actions seem to me to be in flagrant violation of the ACM's own Code of Ethics (http://www.acm.org/about/code-of-ethics) and worse, it seems to be contrary to the ACM's entire charter and purpose.

I have no doubt the ACM performs many valuable services which incur substantial costs, and these costs need to be offset by exploring methods of generating revenue. But this is not an acceptable method. It costs nothing to make these papers available online - and the ACM has no business opposing the establishment of free learning resources that would compete with its own offerings. On the contrary, the ACM is not a profit-making organisation. Its charter to promote learning and education amongst its members, and this can only be best served by promoting and facilitating the free and open access to academic papers such as these.

It seems clear to me that if the ACM does not effect an immediate and total reversal of its policy on these matters, then the Association has changed from the illustrious body it once was, into a force for ill, pissing in the well of knowledge, to be actively opposed by professional, moral software engineers everywhere.

The background to this issue is most clearly expressed here: http://www.realtimerendering.com/blog/us-gov-requests-feedback-on-open-access-acm-gets-it-wrong-again/

I would urge others to boycott any further membership payments to the ACM until this is satisfactorily resolved.

Sincerely,

Jonathan Hartley
Software Engineer

Result of MP visit - I suck

So earlier tonight I turned up at the monthly 'surgery' of my MP, Frank Dobson (Labour), to present my objections to the Digital Economy Bill.

For the benefit of others also going to tackle Mr Dobson on this issue, (and anyone going to see some other MP) here's a quick write up of my experiences:

I arrived up half an hour before it started, and there were about eight parties ahead of me in line. My turn came half an hour after they kicked off. I had hoped to present my argument pretty much as I laid it out in my previous blog post. However, sadly I pretty much comprehensively failed to get my point across.

The 'I am not a spittle-flecked loon' introduction

As planned, I started out by saying that I fully support copyright law, and of the importance of finding ways to ensure that artists get renumerated for their work. The portions of the bill which describe the pursuit of illegal downloaders therefore have laudable and important goals, however I think that the costs and problems associated with preventing illegal downloads like this are substantial, and actually make this section of the bill counter-productive. Mr Dobson was an affable gent, and listened attentively to what I had to say.

Disconnection needlessly punishes more innocent people than guilty ones

The first objection I wanted to raise, because it seems to me to be the most clear-cut, is that the bill suggests disconnection from the internet for persistent downloaders. I pointed out that since most internet connections are shared between several people (families and housemates) disconnecting a single illegal downloader would also unfairly affect all the people who share their connection. Mr Dobson's opinion was that it would rarely come to actual disconnection - the intention of the Bill is very much that disconnection would be a rarely-used last resort. The earlier provisions in the bill (warning offenders for their first 'two strikes') are intended to stop almost everyone from illegally downloading, without any need for actual disconnections to take place.

Obviously I regard that as a grave misjudgement. But I had a lot of ground to cover, so I left this issue at that. I still feel that even if disconnection were only applied to a small number of people (which seems unlikely, from my perspective), it is still true that this would affect more innocents than guilty parties. There is no reason not to prefer more conventional punitive measures over disconnection.

Impossibility of distinguishing guilt from innocence

I moved onto my second point: When monitoring downloads, it is impossible to distinguish between guilt and innocence. If I download a song I own, from my home to my office, this behaviour may currently be legally murky, but it is clearly morally justifiable. To any external observer, this looks indistinguishable from illegal download. An observer cannot tell whether a particular download is illegal or not just by looking at the bits going down the wire - it requires more context than that. It requires the observer to know which CDs I have bought, and to what use I intend to put the proceeds (is it subject to fair use exemptions, etc?) Therefore, along with many illegal downloaders, the system proposed in the bill will also catch many innocent people in its net.

These innocents can appeal - there is a procedure in the bill for this - however they will find it impossible to demonstrate their innocence. If I have bought a CD and ripped it to my computer (which I have done with all my CDs, many years ago) then when I am accused of illegally downloading it, how am I supposed to demonstrate that I once bought that CD?

Mr Dobson listened with patience and good humour to this. However, it was clear from his responses that while he is a capable man who is working hard and with the best of intentions, the scenario I outlined was fairly foreign to his experience. He suggested that the bill is not aimed at catching people like me who are doing nothing wrong. It is aimed at people who are making a dishonest living by massive downloads which they are somehow making money from. I don't believe that any such people actually exist, never mind that the bill is aimed at them: The wording of the bill clearly encompasses individuals downloading music for their own listening pleasure, and deliberately so, but I did not go into this.

Mr Dobson felt that the appeals process would be sufficient for anyone such as my self to protest their innocence. I totally failed to communicate that demonstrating my innocence during such an appeals process would be very difficult. When I reiterated that I could not, for example, produce the receipts to all the music and movies I have ever bought in my entire lifetime, Mr Dobson reiterated that the bill was not aimed at innocent people like me, it would only be applied to illegal downloaders. I again said I don't see how the ISPs can distinguish between the two - for example I cannot produce the receipts nor the CDs that my music came from - I ripped them in 1998, and have since discarded them all. Mr Dobson said that perhaps when the bill become law (my emphasis), people will be obliged to keep their CDs "in a plastic bag in the garage", in preparation for just such a scenario. I pointed out that all of my music for the last few years has been purchased online - so never came on any CDs to begin with.

Mr Dobson didn't seem receptive to this argument. He was still in good spirits at this point, however, I totally failed to persuade him that difficulty in distinguishing between guilty and innocent downloaders was even a realistic possibility, never mind that it would actually be a serious problem.

I had hoped to move on from this point to demonstrate that this presumption of guilt that the bill contains will have a severe chilling effect on people's legitimate and morally justifiable uses of the internet, and this will hamper the development of business models, which will be presumptively quashed before ever getting a chance.

However, Mr Dobson suggested that I should send him the remainder of my points to him in a letter. I asked him if I could just spend 30 seconds skipping through the executive summary of the remainder of my argument, since I felt there was a logical chain of inferences to follow, which demonstrated some real problems with the bill. He suggested that I write it to him in a letter instead.

Probable ten or fifteen minutes had elapsed. We exchanged cordial pleasantries and headed out for a lovely curry down the street with my awesome wife.

My objections to the Digital Economy Bill

This is pretty much a riff on what I posted yesterday, only this is my draft for how I intend to present it to my MP tomorrow. I have no idea how much time I'll have, nor how receptive he'll be. This may all be a big waste of time, but just in case, this is my plan, such as it is.

If you think any part of this is idiotic, or that I've misunderstood how the relevant bits of the Bill work, or that I'm belabouring some points at the expense of other, more important ones, then I'd really love to hear about it. The last thing I want is for some stupid part of my argument to undermine the other parts of it.

The Digital Economy Bill

http://www.publications.parliament.uk/pa/ld200910/ldbills/001/10001.i-ii.html

I want to clarify at the outset that I absolutely support the importance of copyright law, its enforcement, and give my full support to the expectation of artists to be remunerated in a market-driven manner. I do not want the following arguments to be confused with those which suggest that copyright should be abolished, or everything online should be free.

(This bit is my attempt to establish some credibility and rapport. In reality, I think copyright, although sound in principle, is sorely in need of substantial revision, namely it should be much shorter in duration, should have larger and more explicit exceptions, and should require pro-active registering and renewal by the rights holder. But those concerns are orthogonal to the Digital Economy Bill in its current form, and raising them here would only confuse the issue.)

As a computer programmer, I am directly affected by illegal downloads. Almost everything I have created professionally has subsequently been available online for illegal download - affecting my own income and the financial viability of my employers, such as my current small technology start-up. (This is not entirely true - some military projects I have worked on have not suffered any piracy, due to having, shall we say, extravagant hardware requirements)

However, even with that background, I still have strong objections to the sections of the bill 4 through 17, which attempt to redress the harm done to the creative industries by the elimination of illegal downloading.

I believe this pursuit of illegal downloaders comes with problems that are so profound as to make this portion of the bill counter-productive - and I believe that a superior alternative course of action does exist, that would be of substantially more benefit to artists.

The problems as I perceive them are:

1. Disconnection affects many innocents.

Regarding the Obligations to Limit Access described in section 11.

Almost every internet connection is used by more than one person. For example family members, or independent parties in a shared house.

Disconnection of persistent illegal downloaders will therefore impact all these other users of the shared connection. The innocent users disconnected by this legislation will greatly outnumber the illegal downloaders being targeted!

This similarly also applies to the other, lesser remedies described by section 11, of affecting service speed or access to particular materials.

This is flagrantly unfair - a whole household of innocent people are being punished because of the actions of one guilty person.

Infringing downloads may not even have originated from a resident of the house. They could easily have been made a dinner guest - very possibly without the resident's knowledge, and yet they are to be collectively punished for it. Are hosts expected to prevent this by searching every guest to their house, demanding guests hand over their phones and other devices?

This idea is equally problematic when considering publicly available internet connections, such as at libraries, airports, coffee-shops, etc. Are they to be disconnected when one of their customers infringes?

My conclusion: Disconnection and the other remedies described in section 11 are not, under any circumstances, appropriate courses of action, even for the most egregious offenders, and more conventional remedies should be considered instead.

2. Impossibility of distinguishing guilt from innocence

Several times a week, I routinely download many dozens of online media from my home collection, to wherever I happen to be at the time - the office, at a conference, travelling overseas. These are on the whole media I have previously bought - music or movies or computer programs or games. I have ripped these legally purchased items to my computer precisely so that I would have the freedom to move them geographically, and to format-shift them from one device to another.

To any external observer, these downloads are indistinguishable from an illegal download. Copying of the exact same songs is involved in each case. The external observer cannot know that I have already bought the music or movie or whatever in question.

If I were accused of illegal downloading under this law, which I apparently would be on a very regular basis, then would have no way of defending myself against these false accusations. I cannot produce proof of purchase for all of the media I have purchased over my whole lifetime.

If it came down to producing the physical media as evidence, such as CDs my songs came on, then of course many of my songs were purchased online, and the CDs I bought years ago were discarded as soon as I had ripped them, in celebration of my freedom from the constraints of physical media.

Similarly, my accuser, upon whom I believe the onus of proof should lie, cannot prove the negative (that I never bought the media in question.) So it is impossible to distinguish the guilty from the innocent, and

3. Chilling effect

The presumption of guilt inherent in the Bill will clearly have a chilling effect on my personal use of the internet as described above. The raison d'être of the internet is to facilitate the copying of information from one place to another, and yet my perfectly innocuous desire to use it for exactly that is, in this instance, prevented for fear of me being tarred with the same brush as illegal downloaders.

If this were the full extent of the chilling effect that the bill will have, then it is arguable that this is a regrettable but acceptable collateral damage in the struggle to recompense artists. However, this is not the only action that is chilled by the Bill's presumption of guilt.

TODO, this section is weak, and represents what I think is the most important problem. At the very least, more compelling examples are needed here

Artists exchanging material while collaborating on a project, or or someone downloading music that they themselves wrote, or business model that involves copying media from one place to another, all would be under risk from invoking the penalty of 'illegal downloads', and will be prevented from occurring.

Personal freedoms and business models of both the present and the future will be inhibited by the presumption of guilt that the pursuit of file-sharers requires.

This is, to my mind, the most serious of all the problems with the Bill's pursuit of illegal downloaders. Instead of stimulating the digital economy, these measures seem to me to stifle it, by pandering to the wishes of existing special interests, at the expense of personal freedoms and the emerging business models of the future.

4. No provision to clarify the legality of innocent use

It is disappointing that the bill does not take the opportunity to make actions which are both common and morally justifiable clearly and explicitly legal.

I am thinking here of actions such as my space-shifting of the files I own, as described above, and of format-shifting between different devices.

These actions do not in any way represent a valid 'lost sale', and have absolutely no negative effect on the content industry's business, other than countering unreasonable attempts to force consumers to purchase the same media several times over.

Removing the uncertainty about these actions would free individuals from the lurking threat of frivolous prosecution over harmless activities, and would foster innovation by giving business some guidelines about exactly what is allowed.

5. Intrusive state monitoring of everyone's online actions

I strongly object to the notion of the ISPs, acting as representatives of the State, monitoring every single action that I and my family and my friends carry out online. It is ripe for abuse, for the information falling into the wrong hands, and for leaving me under suspicion due to clerical or other errors during the collection or storage of the data. It is absolutely unacceptable to me.

6. Lack of substantive evidence that artists income has really dropped

As stated initially, the goal of ensuring artists are suitably remunerated for their creations is a laudable one, that I fully support.

Crucially though, the purported losses incurred by the music and movie industries are commonly derived from flawed studies, directly funded by the industries in question, and frequently make fundamental mistakes such as equating every download made with a lost sale. More substantive information than this is required about the size of the problem.

Indeed, some figures show that the music industry has in fact grown very healthily in the last few years. For example, these figures compiled and published by The Times on 12th November 2009, based on the BPI's own data:

http://labs.timesonline.co.uk/blog/2009/11/12/do-music-artists-do-better-in-a-world-with-illegal-file-sharing/

This suggests that since 2004, recorded music incomes in the UK have fallen by some £326M. Whether this is due to illegal downloading is not known - presumably it is partly due to that, and partly due to other factors, such as the music industry putting out fewer records in recent years. However, the same figures also show that over the same period, income from live music and merchandising have risen by the larger amount of £373M. The industry as a whole is around £47M better off, and as a bonus, the proportion of this income which is paid directly to artists has risen dramatically, since artists make more money from live shows than from recorded music sales.

This fits with my own personal experience - free downloads act as a tremendous promotional boost for bands. I have many friends who are able to try out far more music as a result of its availability online, and as a result are far more active in attending live music events, often of bands they would not otherwise have heard of, had they not discovered them online.

If this is true, then free downloads cause artists to lose money via recorded music, but also to gain more in recompense through other channels. In this case, do we really need heavy-handed legislation to protect their interests, if the legislation in question comes with all the other costs described here? If artists really are earning more than they were five years ago, do we need to introduce the chilling effects this bill will cause on the rest of the economy in order to try and help them out? What if the pursuit of illegal downloaders means that revenues in live music then drops back to their previous lower levels? Artists will have actually been financially hurt by this bill. This needs to be understood before taking such drastic actions as the bill proposes.

7. One alternative

TODO: Lay out the "central pool to remunerate artists in proportion to their number of downloads" idea, merely to make it clear that there are alternatives courses of action which could be used to tackle this situation, which seem to come with real incentives to stimulate the digital and creative economies, and with smaller, more soluble problems than Digital Economy Bill in its current form.

[SOLVED]UK Digital Economy Bill

So my splendid talented intelligent handsome friend Robert (who is also my boss, of sorts) today shared this smashing post:

My \$62.47 Royalty Statement: How Major Labels Cook the Books with Digital Downloads

About one aspect of how artists get ripped off by the labels. It's not required reading, but you can if you like. I'll wait.

Hey. So it made me realise that a very possible fix to the dreadful abuse of artists by the labels is possible. Even better, the solution to this appalling situation is also the solution to the dire problems with the UK's Digital Economy Bill.

So the UK govt. are debating a new bill, part of which introduces measures to ruthlessly pursue illegal downloaders. There are some problems with this, namely that:

  • someone (probably the ISPs) will have to monitor what everyone downloads, and report people downloading anything that trips their filters for 'copyrighted work'. Understandably, they don't want to do this.
  • It will cost lots to implement - currently estimated at 100 million, but of course the ensuing arms race with encrypted downloads will inflate that. Has anyone compared this with the actual economic losses that it is aiming to fix? (not just the inflated ones self-reported by those with most to gain.)
  • It is literally impossible to automatically detect breach of copyright accurately - it can't be done just by looking at the bits. There will be lots of false positives and false negatives.
  • It will be easy to get around, using encryption or similar technologies to evade detection. It will not fix the problem it aims to fix.
  • It requires representatives of the state to intrusively monitor every single thing we do online.
  • The false positives will target and punish many innocent people, who have no way to demonstrate their innocence.
  • Perhaps worst of all, its presumption of guilt will have a chilling effect on many legal and morally justifiable uses of the internet, including business models of the present and the future.
  • They are talking about disconnection from the internet as a potential punitive measure for persistent downloaders. Most internet connections are shared by several people, and many of us use each other's connections all the time. Disconnecting one person for something that someone else downloaded is flagrantly wrong, and without wishing to be melodramatic, collective punishment like this is, strictly speaking, in breach of article 33 of the Geneva Convention. Plus it makes free wi-fi impossible to provide. Are we meant to be destroying the digital economy?

If your country doesn't have a Bill like this, rest assured, it soon will have.

Anyhow, it's clear to me that there's no use in railing against the Bill unless one has an alternative to offer. The debate in the House of Lords yesterday made it clear that the mood there is that millions are flouting the law with illegal downloads to rip off artists and Something Must Be Done.

So what's the alternative? The solution is the final chapter from Stanford Law Prof. Lawrence Lessig's book 'Free Culture', that describes how to fix the whole copyright mess in such a way that will make everybody happy. Artists will get paid in abundance, very fairly, in a market-driven way, and the payment will be direct to them. The labels will be eliminated from the loop. No state monitoring of what people download will be required. Everybody will be able to continue to download whatever they like from wherever they like, for free (at point of use), legally and with clear conscience. Best of all, the creation of new business models and grass-roots creativity such as songs and other content will be stimulated through the roof. Whichever country first introduced this would lead the world in a creative tidal wave that embraces the powers of the internet, rather than fighting against them.

I'm not going to write what the copyright fix is here. I have to go to work and I'm still in my pajamas. Oh alright, here's the quick quick version. We add a tax to every high bandwidth internet connection. The money goes in a pot. Then a central agency samples how many of each creative work is being downloaded. This is just a sample - it needn't be tied to any individual downloader, so it is cheap to do and unintrusive. Then we distribute the money in the pot to artists, in proportion to the number of downloads they had.

This quick version has some holes in it. Prof. Lessig ties it all up in his book. Yes it has some problems - taxes are never popular, and more expensive internet connections are not what we want. But these problems are small and soluble, compared to what is currently being discussed instead.

I know this is so starry-eyed idealistic that it has no chance of being seriously discussed. But I decided last night that I have to at least give it a try.

Readers from the UK - Do YOU have even the slightest inkling of interest in suggesting this to your own MP? Or at least pointing out the downsides of the current Bill? It would just be one evening out of your life. I'm visiting mine this weekend. Email me if you have any interest at all. I can lead you through the process, it's simple.

Use of Asserts in Unit-Tested Code

I pretty much abandoned the use of asserts in production code once I was introduced to test driven development. Amongst their failings, assert statements are only suited to testing very localised conditions, such as within a particular function - it's not clear to me how they can be used to simply verify the behaviour of a number of objects working in concert with one another.

Some recent papers have touched on this issue. For example 'Assessing the Relationship between Software Assertions and Code Quality' by Gunnar Kudrjavets, Nachiappan Nagappan and Thomas Ball of Microsoft: "We observe from our case study that with an increase in the assertion density in a file there is a statistically significant decrease in fault density. Further, the usage of software assertions in these components found a large percentage of the faults in the bug database."

However, both assertion density and low defect rates also correlate with higher levels of developer experience. So, nothing definitive is proven, but the study has got me thinking. One of the recurring themes in the Coders at Work book that I enjoyed recently, is that several of the expert practitioners interviewed in the book mentioned how useful they found thinking about code in terms of its invariants. The use of assertions does seem to lend itself to testing invariants like this - some quantity which can be calculated at the start of a function call, and then asserted to be unchanged by the end of it.

So can assertions be useful even in code that is well unit-tested? Are there any rules we can use to choose which things might be more productively tested using such asserts, rather than in unit tests?

I don't know. This is as far as I've got. Thoughts, speculations and anecdotes welcome.

Set your Prompt

It's a tiny detail, but the default command-line prompt (on both Windows and Unix) drives me nuts.

mswin-command-prompt1

See how my cursor is way over on the right. So every command you ever type gets immediately split over two lines.

This is governed by the PROMPT environment variable. It's the Windows equivalent of the Unix PS1 variable. You can set this using the environment variable editing GUI on the System Properties dialog. (I'll describe a better way to manage environment variables in a later post.)

set Prompt=$P$_$G$S

The dollar codes are expanded into:

  • \$P - present working directory
  • \$_ - newline
  • \$G - greater than
  • \$S - space

mswin-command-prompt2

There, isn't that better? There are a bunch of other special dollar codes, that are described here. They only work within this one environment variable, you can't use them anywhere else.

Comparing methods of deploying Python applications

I had a cry last month about deploying a Python application to end-users being more fiddly and difficult than I expected it to be. I'm talking about py2exe, Freeze, and their ilk.

As a quick follow-up, I recently posted to comp.lang.python about a spreadsheet I've made, comparing comparing the solutions I've found thus far:

http://spreadsheets.google.com/pub?key=tZ42hjaRunvkObFq0bKxVdg&output=html

Each column represents a method of distributing a Python application to an end-user without them having to worry about what Python is or whether they have an appropriate version of it installed. 'Bundle' represents manually bundling an interpreter with your application. 'Bootstrap' represents a sort of fanciful vision of manually creating a compiled installer which downloads and installs an interpreter if required, before running your Python application on it. The other columns represent the various splendid projects like py2exe which perform the bundling process for you.

Each row represents a reason to choose between them. By my criteria, which are no doubt idiosyncratic, cx_freeze or maybe PyInstaller are starting to look like good contenders for future projects.

Obviously feedback about any wrong-headedness on my part is appreciated.