My objections to the Digital Economy Bill

This is pretty much a riff on what I posted yesterday, only this is my draft for how I intend to present it to my MP tomorrow. I have no idea how much time I'll have, nor how receptive he'll be. This may all be a big waste of time, but just in case, this is my plan, such as it is.

If you think any part of this is idiotic, or that I've misunderstood how the relevant bits of the Bill work, or that I'm belabouring some points at the expense of other, more important ones, then I'd really love to hear about it. The last thing I want is for some stupid part of my argument to undermine the other parts of it.

The Digital Economy Bill

http://www.publications.parliament.uk/pa/ld200910/ldbills/001/10001.i-ii.html

I want to clarify at the outset that I absolutely support the importance of copyright law, its enforcement, and give my full support to the expectation of artists to be remunerated in a market-driven manner. I do not want the following arguments to be confused with those which suggest that copyright should be abolished, or everything online should be free.

(This bit is my attempt to establish some credibility and rapport. In reality, I think copyright, although sound in principle, is sorely in need of substantial revision, namely it should be much shorter in duration, should have larger and more explicit exceptions, and should require pro-active registering and renewal by the rights holder. But those concerns are orthogonal to the Digital Economy Bill in its current form, and raising them here would only confuse the issue.)

As a computer programmer, I am directly affected by illegal downloads. Almost everything I have created professionally has subsequently been available online for illegal download - affecting my own income and the financial viability of my employers, such as my current small technology start-up. (This is not entirely true - some military projects I have worked on have not suffered any piracy, due to having, shall we say, extravagant hardware requirements)

However, even with that background, I still have strong objections to the sections of the bill 4 through 17, which attempt to redress the harm done to the creative industries by the elimination of illegal downloading.

I believe this pursuit of illegal downloaders comes with problems that are so profound as to make this portion of the bill counter-productive - and I believe that a superior alternative course of action does exist, that would be of substantially more benefit to artists.

The problems as I perceive them are:

1. Disconnection affects many innocents.

Regarding the Obligations to Limit Access described in section 11.

Almost every internet connection is used by more than one person. For example family members, or independent parties in a shared house.

Disconnection of persistent illegal downloaders will therefore impact all these other users of the shared connection. The innocent users disconnected by this legislation will greatly outnumber the illegal downloaders being targeted!

This similarly also applies to the other, lesser remedies described by section 11, of affecting service speed or access to particular materials.

This is flagrantly unfair - a whole household of innocent people are being punished because of the actions of one guilty person.

Infringing downloads may not even have originated from a resident of the house. They could easily have been made a dinner guest - very possibly without the resident's knowledge, and yet they are to be collectively punished for it. Are hosts expected to prevent this by searching every guest to their house, demanding guests hand over their phones and other devices?

This idea is equally problematic when considering publicly available internet connections, such as at libraries, airports, coffee-shops, etc. Are they to be disconnected when one of their customers infringes?

My conclusion: Disconnection and the other remedies described in section 11 are not, under any circumstances, appropriate courses of action, even for the most egregious offenders, and more conventional remedies should be considered instead.

2. Impossibility of distinguishing guilt from innocence

Several times a week, I routinely download many dozens of online media from my home collection, to wherever I happen to be at the time - the office, at a conference, travelling overseas. These are on the whole media I have previously bought - music or movies or computer programs or games. I have ripped these legally purchased items to my computer precisely so that I would have the freedom to move them geographically, and to format-shift them from one device to another.

To any external observer, these downloads are indistinguishable from an illegal download. Copying of the exact same songs is involved in each case. The external observer cannot know that I have already bought the music or movie or whatever in question.

If I were accused of illegal downloading under this law, which I apparently would be on a very regular basis, then would have no way of defending myself against these false accusations. I cannot produce proof of purchase for all of the media I have purchased over my whole lifetime.

If it came down to producing the physical media as evidence, such as CDs my songs came on, then of course many of my songs were purchased online, and the CDs I bought years ago were discarded as soon as I had ripped them, in celebration of my freedom from the constraints of physical media.

Similarly, my accuser, upon whom I believe the onus of proof should lie, cannot prove the negative (that I never bought the media in question.) So it is impossible to distinguish the guilty from the innocent, and

3. Chilling effect

The presumption of guilt inherent in the Bill will clearly have a chilling effect on my personal use of the internet as described above. The raison d'être of the internet is to facilitate the copying of information from one place to another, and yet my perfectly innocuous desire to use it for exactly that is, in this instance, prevented for fear of me being tarred with the same brush as illegal downloaders.

If this were the full extent of the chilling effect that the bill will have, then it is arguable that this is a regrettable but acceptable collateral damage in the struggle to recompense artists. However, this is not the only action that is chilled by the Bill's presumption of guilt.

TODO, this section is weak, and represents what I think is the most important problem. At the very least, more compelling examples are needed here

Artists exchanging material while collaborating on a project, or or someone downloading music that they themselves wrote, or business model that involves copying media from one place to another, all would be under risk from invoking the penalty of 'illegal downloads', and will be prevented from occurring.

Personal freedoms and business models of both the present and the future will be inhibited by the presumption of guilt that the pursuit of file-sharers requires.

This is, to my mind, the most serious of all the problems with the Bill's pursuit of illegal downloaders. Instead of stimulating the digital economy, these measures seem to me to stifle it, by pandering to the wishes of existing special interests, at the expense of personal freedoms and the emerging business models of the future.

4. No provision to clarify the legality of innocent use

It is disappointing that the bill does not take the opportunity to make actions which are both common and morally justifiable clearly and explicitly legal.

I am thinking here of actions such as my space-shifting of the files I own, as described above, and of format-shifting between different devices.

These actions do not in any way represent a valid 'lost sale', and have absolutely no negative effect on the content industry's business, other than countering unreasonable attempts to force consumers to purchase the same media several times over.

Removing the uncertainty about these actions would free individuals from the lurking threat of frivolous prosecution over harmless activities, and would foster innovation by giving business some guidelines about exactly what is allowed.

5. Intrusive state monitoring of everyone's online actions

I strongly object to the notion of the ISPs, acting as representatives of the State, monitoring every single action that I and my family and my friends carry out online. It is ripe for abuse, for the information falling into the wrong hands, and for leaving me under suspicion due to clerical or other errors during the collection or storage of the data. It is absolutely unacceptable to me.

6. Lack of substantive evidence that artists income has really dropped

As stated initially, the goal of ensuring artists are suitably remunerated for their creations is a laudable one, that I fully support.

Crucially though, the purported losses incurred by the music and movie industries are commonly derived from flawed studies, directly funded by the industries in question, and frequently make fundamental mistakes such as equating every download made with a lost sale. More substantive information than this is required about the size of the problem.

Indeed, some figures show that the music industry has in fact grown very healthily in the last few years. For example, these figures compiled and published by The Times on 12th November 2009, based on the BPI's own data:

http://labs.timesonline.co.uk/blog/2009/11/12/do-music-artists-do-better-in-a-world-with-illegal-file-sharing/

This suggests that since 2004, recorded music incomes in the UK have fallen by some £326M. Whether this is due to illegal downloading is not known - presumably it is partly due to that, and partly due to other factors, such as the music industry putting out fewer records in recent years. However, the same figures also show that over the same period, income from live music and merchandising have risen by the larger amount of £373M. The industry as a whole is around £47M better off, and as a bonus, the proportion of this income which is paid directly to artists has risen dramatically, since artists make more money from live shows than from recorded music sales.

This fits with my own personal experience - free downloads act as a tremendous promotional boost for bands. I have many friends who are able to try out far more music as a result of its availability online, and as a result are far more active in attending live music events, often of bands they would not otherwise have heard of, had they not discovered them online.

If this is true, then free downloads cause artists to lose money via recorded music, but also to gain more in recompense through other channels. In this case, do we really need heavy-handed legislation to protect their interests, if the legislation in question comes with all the other costs described here? If artists really are earning more than they were five years ago, do we need to introduce the chilling effects this bill will cause on the rest of the economy in order to try and help them out? What if the pursuit of illegal downloaders means that revenues in live music then drops back to their previous lower levels? Artists will have actually been financially hurt by this bill. This needs to be understood before taking such drastic actions as the bill proposes.

7. One alternative

TODO: Lay out the "central pool to remunerate artists in proportion to their number of downloads" idea, merely to make it clear that there are alternatives courses of action which could be used to tackle this situation, which seem to come with real incentives to stimulate the digital and creative economies, and with smaller, more soluble problems than Digital Economy Bill in its current form.

[SOLVED]UK Digital Economy Bill

So my splendid talented intelligent handsome friend Robert (who is also my boss, of sorts) today shared this smashing post:

My \$62.47 Royalty Statement: How Major Labels Cook the Books with Digital Downloads

About one aspect of how artists get ripped off by the labels. It's not required reading, but you can if you like. I'll wait.

Hey. So it made me realise that a very possible fix to the dreadful abuse of artists by the labels is possible. Even better, the solution to this appalling situation is also the solution to the dire problems with the UK's Digital Economy Bill.

So the UK govt. are debating a new bill, part of which introduces measures to ruthlessly pursue illegal downloaders. There are some problems with this, namely that:

  • someone (probably the ISPs) will have to monitor what everyone downloads, and report people downloading anything that trips their filters for 'copyrighted work'. Understandably, they don't want to do this.
  • It will cost lots to implement - currently estimated at 100 million, but of course the ensuing arms race with encrypted downloads will inflate that. Has anyone compared this with the actual economic losses that it is aiming to fix? (not just the inflated ones self-reported by those with most to gain.)
  • It is literally impossible to automatically detect breach of copyright accurately - it can't be done just by looking at the bits. There will be lots of false positives and false negatives.
  • It will be easy to get around, using encryption or similar technologies to evade detection. It will not fix the problem it aims to fix.
  • It requires representatives of the state to intrusively monitor every single thing we do online.
  • The false positives will target and punish many innocent people, who have no way to demonstrate their innocence.
  • Perhaps worst of all, its presumption of guilt will have a chilling effect on many legal and morally justifiable uses of the internet, including business models of the present and the future.
  • They are talking about disconnection from the internet as a potential punitive measure for persistent downloaders. Most internet connections are shared by several people, and many of us use each other's connections all the time. Disconnecting one person for something that someone else downloaded is flagrantly wrong, and without wishing to be melodramatic, collective punishment like this is, strictly speaking, in breach of article 33 of the Geneva Convention. Plus it makes free wi-fi impossible to provide. Are we meant to be destroying the digital economy?

If your country doesn't have a Bill like this, rest assured, it soon will have.

Anyhow, it's clear to me that there's no use in railing against the Bill unless one has an alternative to offer. The debate in the House of Lords yesterday made it clear that the mood there is that millions are flouting the law with illegal downloads to rip off artists and Something Must Be Done.

So what's the alternative? The solution is the final chapter from Stanford Law Prof. Lawrence Lessig's book 'Free Culture', that describes how to fix the whole copyright mess in such a way that will make everybody happy. Artists will get paid in abundance, very fairly, in a market-driven way, and the payment will be direct to them. The labels will be eliminated from the loop. No state monitoring of what people download will be required. Everybody will be able to continue to download whatever they like from wherever they like, for free (at point of use), legally and with clear conscience. Best of all, the creation of new business models and grass-roots creativity such as songs and other content will be stimulated through the roof. Whichever country first introduced this would lead the world in a creative tidal wave that embraces the powers of the internet, rather than fighting against them.

I'm not going to write what the copyright fix is here. I have to go to work and I'm still in my pajamas. Oh alright, here's the quick quick version. We add a tax to every high bandwidth internet connection. The money goes in a pot. Then a central agency samples how many of each creative work is being downloaded. This is just a sample - it needn't be tied to any individual downloader, so it is cheap to do and unintrusive. Then we distribute the money in the pot to artists, in proportion to the number of downloads they had.

This quick version has some holes in it. Prof. Lessig ties it all up in his book. Yes it has some problems - taxes are never popular, and more expensive internet connections are not what we want. But these problems are small and soluble, compared to what is currently being discussed instead.

I know this is so starry-eyed idealistic that it has no chance of being seriously discussed. But I decided last night that I have to at least give it a try.

Readers from the UK - Do YOU have even the slightest inkling of interest in suggesting this to your own MP? Or at least pointing out the downsides of the current Bill? It would just be one evening out of your life. I'm visiting mine this weekend. Email me if you have any interest at all. I can lead you through the process, it's simple.

Use of Asserts in Unit-Tested Code

I pretty much abandoned the use of asserts in production code once I was introduced to test driven development. Amongst their failings, assert statements are only suited to testing very localised conditions, such as within a particular function - it's not clear to me how they can be used to simply verify the behaviour of a number of objects working in concert with one another.

Some recent papers have touched on this issue. For example 'Assessing the Relationship between Software Assertions and Code Quality' by Gunnar Kudrjavets, Nachiappan Nagappan and Thomas Ball of Microsoft: "We observe from our case study that with an increase in the assertion density in a file there is a statistically significant decrease in fault density. Further, the usage of software assertions in these components found a large percentage of the faults in the bug database."

However, both assertion density and low defect rates also correlate with higher levels of developer experience. So, nothing definitive is proven, but the study has got me thinking. One of the recurring themes in the Coders at Work book that I enjoyed recently, is that several of the expert practitioners interviewed in the book mentioned how useful they found thinking about code in terms of its invariants. The use of assertions does seem to lend itself to testing invariants like this - some quantity which can be calculated at the start of a function call, and then asserted to be unchanged by the end of it.

So can assertions be useful even in code that is well unit-tested? Are there any rules we can use to choose which things might be more productively tested using such asserts, rather than in unit tests?

I don't know. This is as far as I've got. Thoughts, speculations and anecdotes welcome.

Set your Prompt

It's a tiny detail, but the default command-line prompt (on both Windows and Unix) drives me nuts.

mswin-command-prompt1

See how my cursor is way over on the right. So every command you ever type gets immediately split over two lines.

This is governed by the PROMPT environment variable. It's the Windows equivalent of the Unix PS1 variable. You can set this using the environment variable editing GUI on the System Properties dialog. (I'll describe a better way to manage environment variables in a later post.)

set Prompt=$P$_$G$S

The dollar codes are expanded into:

  • \$P - present working directory
  • \$_ - newline
  • \$G - greater than
  • \$S - space

mswin-command-prompt2

There, isn't that better? There are a bunch of other special dollar codes, that are described here. They only work within this one environment variable, you can't use them anywhere else.

Comparing methods of deploying Python applications

I had a cry last month about deploying a Python application to end-users being more fiddly and difficult than I expected it to be. I'm talking about py2exe, Freeze, and their ilk.

As a quick follow-up, I recently posted to comp.lang.python about a spreadsheet I've made, comparing comparing the solutions I've found thus far:

http://spreadsheets.google.com/pub?key=tZ42hjaRunvkObFq0bKxVdg&output=html

Each column represents a method of distributing a Python application to an end-user without them having to worry about what Python is or whether they have an appropriate version of it installed. 'Bundle' represents manually bundling an interpreter with your application. 'Bootstrap' represents a sort of fanciful vision of manually creating a compiled installer which downloads and installs an interpreter if required, before running your Python application on it. The other columns represent the various splendid projects like py2exe which perform the bundling process for you.

Each row represents a reason to choose between them. By my criteria, which are no doubt idiosyncratic, cx_freeze or maybe PyInstaller are starting to look like good contenders for future projects.

Obviously feedback about any wrong-headedness on my part is appreciated.

PyChoose: Switch between installed versions of Python

You can switch between different installed versions of Python by simply prepending to your PATH. However, this goes wrong in some scenarios, such as when a tool like 'virtualenv' is installed in one Python version, but not in another. Pretty soon you will be executing your project with one Python version, but referencing the site-packages of another. When switching, other versions of Python need to be stripped from your PATH.

As a solution, I present a new Python module, pychoose. For the moment this is Windows only:

>python -V
Python 2.6.4
>easy_install pychoose
...
>pychoose 24
(Py24) >python -V
Python 2.4.4
(Py24) >exit
>python -V
Python 2.6.4

It prepends to the PATH as above, but it also filters other Python versions out the PATH. It modifies the PROMPT to show the user they are operating in a modified environment. It works by spawning a new shell with the modified environment. To return to your default Python interpreter, type 'exit'.

PyPI page: http://pypi.python.org/pypi/pychoose

Subversion repository: http://code.google.com/p/pychoose

Preview the module source code.

It seems to work for simple scenarios, but there are a bunch of known problems with it (see the PyPI page above), which I'd like to fix if people think this is useful. I just whipped this up and don't really understand the issues surrounding this. In particular is spawning a new shell the best way to modify the current console's environment? Is a less-instrusive alternative to generate .bat file which can be executed to change the current shell's environment?

Feedback & suggestions very welcome.

A Layman's Philosophical Musings

A quick response to to polymath Chris DeLeon's thoughtful post exhorting the virtues of Philosophy.

A position of ignorance

Right off the bat I should qualify all this by saying that I don't know anything about Philosophy. I've never studied it. It seems at times to be an alien mismatch to the sensibilities of my left-brained mind, and worse, one that claims to be about logic and reasoning, topics that ostensibly should be right up my alley. I read Bertrand Russel's History of Western Philosophy last year, and although that was a fabulous, educational romp through the field, my complaints about it still stand. Now I'm reading a second book, Think : A Compelling Introduction to Philosophy by Simon Blackburn.

The Process

The process of thinking about things is a valuable and worthwhile endeavour. Some people enjoy it, some people don't. Amongst those who enjoy it, it becomes an end unto itself, and thought and discussions typically stray towards the more abstract, the more meta, the more foundational. Anyone who has gleefully participated in a pub discussion about religion, death, reality, consciousness, government, or any of a million other topics has participated in this process. It is joyful, sometimes enlightening, entertaining, and refines the individual's ability to meaningfully reason on all kinds of topics, from the abstract to the prosaic. In this sense, and especially from the perspective of the individual's active participation in the activity, I'm all in favour of philosophy. Presumably this is why I have been drawn back to read this second book.

The Content

If anything, I have had more objections to Think than I had to Russel's book. At first, I thought that my issue was simply that the discussed historical philosophers produced curiously shoddy work. Their premises seem shaky. Their arguments full of holes at best. Their conclusions entirely suspect. While there are some genuinely insightful ideas and arguments, they are often mutually contradictory, and the signal to noise is terrible.

Chris points out the importance of reading a decent translation of the original sources to gain enough insight to benefit from the arguments, and in this I defer to his experience. There certainly are some historical philosophers who seem to make a lot more sense to me than the others. I had not expected to have to pick and choose based so much upon my own personal preferences, so in some way my frustration here is perhaps indicative of an expectation mismatch.

As for Think, as I continue to read, it has slowly dawned on me that perhaps this impression is actually because the author himself is an appalling philosopher (sorry Simon!) Every few pages he makes what seems to me like a terrible logical blunder. If this were correct, that would presumably make his interpretation of historical philosophers equally suspect, and perhaps this is why they so often seem simply ludicrous to me.

I realise it is arrogant and bonkers for me to assume I can outclass the author, a man apparently of considerable knowledge and experience, at his own game. But this is how it seems to me, so what am I to make of that?

It seems to me that there are only three possibilities:

  1. Professor Blackburn's thinking is entirely defective. I am sure this cannot be the case. He is a professor of his subject, his book, while quite understandably not comparable to Russel, has garnered great reviews on Amazon and in the press, for whatever that's worth. As a layman I shall accord him the deference that is entirely his due.
  2. My own thinking is entirely defective. This is, of course, entirely possible, however distasteful it may be to me. How would I know? In defence of my sanity, I note that I seem to get by, more or less, in everyday life. Or at least, I don't seem to be any more confounded by the state of the world than the people around me.
  3. Perhaps, then, the problem lies not solely with myself, but is shared by us all. Perhaps human reasoning as expressed in the English language is generally not sufficiently precise, well-defined or powerful to support consistent philosophical arguments of the type that are being put forward. More generally, we aren't remotely as good at thinking as we think we are. Pretty much I'm saying that (1) and (2) are both true, but that both the professor and I can derive some comfort from the idea that the same applies to pretty much everybody else.

The Method

There is, of course, a method for dealing with the unreliability of our thoughts and perceptions - the scientific method, in its many forms and manifestations. I have friends who claim that the scientific method came out of philosophy, and while I have some reservations about the truth of this in the day-to-day reality of individual practitioners, of course in an abstract sense I can see that this lineage is real.

Application of the scientific method would be the normal way out of the morass of imprecise thoughts that, to date, characterise the majority of philosophy for me. Some aspects of ancient philosophy have proven tractable in this regard. These are the subjects that have subsequently calved off from philosophy to form whole new fields and sciences.

Of course, this has the curious effect of diluting the perceived value of philosophy. As soon as the application of the scientific method lets thinkers actually gain any traction and start to make real progress on their topic, we immediately snip that subject off from the realm of philosophy and give it a different name. Astronomy and cosmology, biology and psychology, chemistry, physics, and many more. Philosophy proper, then, consists of the left over bits - the topics upon which the scientific method has not proven applicable, and hence upon which no meaningful progress can be made.

Chris, and discussion with friends, has helped me see that in this dismissal, I was overlooking the value of philosophy as an incubator for other fields. This is of course critical to the development of human civilisation, and of interest to historians of the respective fields.

What it isn't

What I'm left with is the impression that my minor experiences of philosophy would have been much improved if my expectations had been very different at the outset.

My layman's impression of philosophy from the outside was that it claimed to be the study of tremendously insightful reasoning done by some of the most powerful minds in history. This expectation was naturally bound to lead me to be disappointed when my initial foray led me to discover that many philosophers, particularly ancient ones, distorted as they are by translation and interpretation, cultural mismatch, and lack of my modern knowledge of the world, appear to my sensibilities to be pretty poor. I hope to improve on this situation by taking on board Chris' advice about choosing particular philosophers who resonate for me personally, together with the idea of studying particular translations of original works, rather than quick summaries.

In addition, I had an expectation that all this intense thought might lead to some more definitive and objective conclusions. However ancient philosophers acted without the benefit of the scientific method, and modern philosophy is pruned of topics to which the scientific method is applicable. As a result, there is much bad mixed in with the good. Even the good parts cannot be trusted as a guide to any sorts of truth, since choosing 'the good parts' seems to be a subjective process.

This seems inevitable, since without the benefit of the scientific method, there is no utility function to be used as feedback, by which ideas could be judged and one school of thought meaningfully compared against another. There can be no definitive winners or losers, and without competition, ideas cannot improve by evolution. Instead, each contributor adds their own voice to the mutually contradictory babble, and it is left to the reader to sift through for the parts of value.

The lack of an objective means to prune the discussion of its less worthy branches mean that many philosophers are deified, their writings sacrosanct and studied for generations, even in cases where they were clearly absolutely wrong. (Plato's thoughts about the shapes of atoms, for example.) I accept that study of historical and ancient philosophers is valid for the perspective it gives on the process of philosophy, and as a study of the history of philosophy itself. However, I am appalled that so much of their actual, specific writings are studied and discussed in so much detail, in the same way I would be mortified by the prospect of scholars spending centuries reviewing the specifics of what me and my friends said in our philosophical discussions in the pub. There may be some value in there, but I came to this expecting a précis of the best minds in the world, not a mishmash of sometimes incoherent ideas.

The Upshot

I've done a lot of bitching, but don't take it as a wholesale rejection. In addition to the things I complain about, there are also a bunch of really great, stimulating ideas. I just didn't expect to have to do so much searching and interpretation to find them. Rather, interpret the above as the painful transitions in my understanding of what philosophy is, which parts of it are likely to be of value to me, and what I should expect from it. Thanks to Chris for his significant contributions towards my continuing education on this.

Programming on Windows : Use Console

Developing software on Microsoft Windows is a bloody awful experience compared to the boyish flamboyance of Macs or the mad scientist thrill-a-minute of wrestling Linux into shape. But fear no more, you no longer need to hide behind that fullscreen IDE, pretending that the rest of your OS doesn't exist. Help is at hand.

I often feel that with just a dozen small modifications, the UI of Microsoft Windows could be perfectly usable for me. However, successive releases of Windows seem to steadfastly ignore the minor changes I'd love to see. If anything, they move relentlessly in the opposite direction with every release.

This is the first in a series of tweaks to work around some of these deficiencies, to create something approaching a sane software development environment on Windows. If you are a *nix or command-line jockey, this might make your visits to Windows-land more comfortable. If you're a Visual Studio devotee, this might add some alternative tools to your repertoire.

My idea is that each post will be small enough that you can implement the change it suggests immediately, right now, as you're reading it.

The Console

The console, by which I mean the black DOS 'Command Prompt' window, is the pivot around which the rest of your software development activities rotate. Or at least it could be, if it was any good. The built-in Command Prompt, however, is absolute garbage. Replace it with the poorly-named 'Console' project from SourceForge:

https://sourceforge.net/projects/console/

Once installed, you could fiddle around with the settings endlessly, or just copy my console.xml config to your home directory:

console.xml (config for Console)console.xml

after which Console should look something like:

Console: It doesn't look like much

Console provides a few superficial benefits that prove to be indispensable in everyday use:

  • Cut and paste is no longer an unimaginable pain in the ass. What a concept. My config automatically copies selected text to the clipboard, so you don't even need to hit Ctrl-C. Because of this, I've reverted the action of Ctrl-C to its more venerable usage of 'kill the running command'.
  • Selecting text with the mouse now works sanely, although sadly you can't double/triple click to select words or lines.
  • Multiple tabs. My config defines keys: Ctrl-T (new), Ctrl-W (close), and Ctrl-tab (next).
  • PageUp and PageDown scroll, without needing to move your hand to the mouse. My config actually uses Shift-PageUp and Shift-PageDown, so that it's like old XTerms that my fingers apparently still remember.
  • We can choose any font. I like Deja Vu Sans Monospace, no doubt you like something totally different.
  • We can resize the window however we like.
  • Transparency. I don't like it, some people do. (Update: Redacted. I've been converted. Now I love it.)

The old built-in Command-Prompt will still pop up if you double-click a .bat file - I don't know how to change this magic built-in association. Otherwise though, you can now just use the new Console for everything you do.

Alright. We've taken our first step. Next up, we'll see some of the uses to which we can put our new toy.

Making a wix installer run a program by default

Guest post by my least favourite colleague, Tom.

Normally - but not always - people want to run a program just after the install it - you can help then to do this by using an opt-out system with a checkbox checked by default

To create the checkbox and control whether it is set you must use the following properties in your Product section:

Id="WIXUI_EXITDIALOGOPTIONALCHECKBOXTEXT"
Value="Launch the cool program"
Id="WIXUI_EXITDIALOGOPTIONALCHECKBOX"
Value="1"

The value of WIXUI_EXITDIALOGOPTIONALCHECKBOX controls the checkboxes checkedness.

You then need a custom action to do some launching:

Id="LaunchApplication"
BinaryKey="WixCA"
DllEntry="WixShellExec"
Impersonate="yes"

and some UI wiring in the UI element under the Product element to get you custom action called.

Control="Finish"
Event="DoAction"
Value="LaunchApplication">WIXUI\_EXITDIALOGOPTIONALCHECKBOX = 1 and
NOT Installed

Huzzah - you now may have a working installer.

Python Deployment Sucks

I don't want to whinge just for the sake of whinging. But I think it's healthy to assess something I currently find difficult, that I might better understand where the deficiencies lie and how they might be addressed.

Deploying a Python program to fellow programmers is easy - just send them the script, they can install the interpreter if they haven't already, and figure it out from there. Use of PyPI helps with this.

But distributing Python as source code is only suitable when distributing to other programmers. For distributing to end-users, it is totally inappropriate. They have no idea what Python is nor whether it is installed, and they should never need to know. While it is arguably possible to lead a user through the process of installing Python so that they can run your script, it is an absolutely ghastly, terrifying experience for them, and they will never willingly use your software again. They want an icon they can double-click on which just runs, period. Considering the potential complications such as requiring side-by-side installations of multiple versions of Python, I don't see this as a workable solution for any serious software to be used by non-programmers.

Thus we have the projects to convert Python source into stand-alone executables. I've used py2exe and bbfreeze, and both do a brilliant jobs at what they do. But there are problems with the use of such tools.

With a compiled language, you can simply give the binary to someone with the right operating system, and they can double click it to run it. Better still, this process of compiling and linking the executable for an end-user is the exact same process that you perform all the time for yourself while you are developing the software. There is no additional overhead required to create an executable for end-users (presuming the simplest case of end-users on the same operating system that you are developing on.)

With Python this is not true. The whole 'create a distributable binary' step is an additional headache over and above whatever effort you've already gone to to make the program run in your own development environment. Because of this, in a very pragmatic sense, deployment of Python programs requires a whole extra set of work that deployment of compiled programs does not.

Presumably this is true for all interpreted languages. I've never worked seriously with one before. I should go read about how Ruby and more venerable languages approach and solve these problems.

This situation would not be quite so bad if there was a clear way to proceed, but for me, my first few binary distributable projects have been a real ramshackle hack through the docs of py2exe and the various projects which compete with it. Lots has to be figured out to choose a tool, and then to get it working with your project. This has to be done over for each OS you choose to deploy on. For me, it has been a real time-sink, and I'm still not really happy with the results.

The resulting executables are bulked out by including the interpreter with each one. This is not a suitable technique for including a bunch of lightweight command-line executables to augment your shell -something which, otherwise, you would think Python should be absolutely ideal for. I guess in some circumstances you could work around this somewhat by using an svn-style 'one executable, many commands' interface, although this isn't always suitable.

I'd prefer it if, instead of bundling an interpreter, py2exe looked to see if a suitable version of Python was already installed, and if not, downloaded and installed it side-by-side with any existing installations and then used it to execute your program.

I can't figure out how to tweak the output from py2exe such that my executable file isn't buried in a menagerie of various other .dll, .zip and .exe files. An end-user would have no idea what to click on. I want to make it easy for my users, by creating an executable by itself, with a subdirectory of binary dependencies (and a different subdirectory of data.) I could sidestep this by creating an installer that creates a shortcut to the relevant executable - but again, that would be yet another day of needless make-work, which has to be undertaken over again for each OS you plan to deploy on.

For large projects, these sorts of problems are surmountable - they have enough man hours to soak it up. But for small projects and one-off scripts, problems like this burn up a substantial proportion of time. Writing a hundred-line script to solve your friend's particular problem and emailing him the binary is awkward to say the least. Entries to the PyWeek 'game in a week' contest are substantially impacted - producing those binaries burns up hours and hours, when you have precious few to spare, and very few of the presumably 'average developers' taking part managed to create binaries that just worked for everyone.

In Python's favour, many of the wrinkles and complications that arise during the process are due to cross-platform issues. Getting hold of binary dependencies for other operating systems, stuff like that. The only reason this seems harder in Python than it is in other languages, is that when using other languages you often don't even attempt to deploy across multiple operating systems. The apparent difficulty of doing this in Python is in actual fact just an illusion caused by being able to attempt it in the first place.

Doubtless some of my other perceived problems lie in my own misunderstandings. Nobody else seems to struggle with this as much as I do. Comments are welcome.

Brought to you from the a-crap-post-is-better-than-no-post-at-all dept.