tartley.com (Posts about python)

Code in Place

Jonathan Hartley — Mon, 07 Jul 2025 14:03:35 GMT

Code in Place 2025 has ended. I participated as a Section Leader, tutoring a small group of beginner programmers through learning basic coding in Python. We covered fundamental programming - variables and loops and conditionals and data-structures, and used them to cajole small on-screen robots into moving around and performing tasks, and to draw some graphics with lines and filled shapes.

The course was open to all, so there were many working members of the public alongside the students. While the course is a redrafting of the famous Stanford CS106A course content, these were not generally computer science students, but instead were studying some other subject, and wanted to add a little bit of programming as an extra skill on the side.

The interesting thing about Code in Place is that it aims to investigate scaling the teaching experience, without falling prey to the usual failure modes of Massive Open-Access Online Courses (MOOCs), of poor feedback on student work and high drop-out rates. They did this by recruiting 4,000 teaching volunteers like me to act as Section Leaders. We guided the 40,000 students through the material in small classes of ~10 students, and provided 1:1 guidance and question answering in forums.

The application for section leader involved some basic competency questions, and an interesting task to record a video of me teaching some basic programming concepts. This all helped me to understand that I need to scale down my expectations of what is "self-evident", reminding me that beginners need to be taught from the beginning.

While AI was used, it was in a thoughtful, advisory capacity. For students, providing explinations of terse technical errors in their own code, and giving feedback on successfully completed tasks, to improve their solutions.

Once the classes started, I heard complaints in the teachers' forums that other classes suffered from some absenteeism, or students simply not doing the work. But my students were absolutely fabulous, reliably showing up on time, doing all the exercises, and engaging honestly and curiously in the classes. I think that students were sorted into age groups that tend to be not too dissimilar to their Section Leader, so perhaps I had an older-than-average cohort, which might partly explain how I got so lucky.

Regardless of how it happened, it was a thrillingly rewarding experience, seeing people rapidly grow their abilities, and become confident where they had originally been fearful. I'm already looking forward to next year's course!

Integer Division With Recurring Decimals

Jonathan Hartley — Mon, 03 Mar 2025 18:42:50 GMT

I've been doing some programming tests and puzzles while job hunting lately. One quick challenge was quite nice, reminding me a bit of Project Euler questions, and I nerd sniped myself into doing a 2nd pass at it here.

Question

Produce a Python function which takes two integers, numerator and denominator, and returns the result of their division as a decimal fraction in a string. E.g:

divide(1, 4) -> "0.25"

If the decimal places contain an infinite recurring pattern of digits, then enclose the recurring digits in parentheses. E.g:

divide(1, 3) -> "0.(3)"
divide(1, 7) -> "0.(142857)"

Wrong approaches

Evaluating the division using normal floats is going to trip you up in several ways with the limited precision.

For one, a large enough denominator might have a recurring sequence which is longer than the number of decimal places you have available (more on this later), which makes it impossible to detect recurring sequences by examining the division's decimal digits.

Worse, the inherent imprecision of floating point, e.g. if a simple division like 10/3 comes back as 3.3333333333333335, then examining the trailing digits of this looking for recurring digits is going to be problematic.

Using the decimal module does markedly improve precision and control. But infinitely repeating sequences are still going to return results like Decimal(20) / Decimal(3) -> Decimal('6.666666666666666666666666667'), which is going to trip us up.

We can sidestep all these complexities if we see that the question is asking us to perform this division ourselves, longhand. We are going back to elementary school! Wheee!

Better

Let's just do basic division first, ignoring infinite or recurring digits:

def divide(numerator: int, denominator:int) -> str:
    # Accumulate parts of our result here
    results = []
    while True:
        int_part = str(numerator // denominator)
        remainder = numerator % denominator
        numerator = remainder * 10
        results.append(int_part)

        # If there is no remainder, we are done
        if remainder == 0:
            break

        # Add a decimal point after our first integer part
        if len(results) == 1:
            results.append(".")

    return ''.join(results)

The only confusing parts of this are that int_part might contain several digits on the first iteration, but is only ever one decimal digit thereafter. Plus we have to be careful to get the ordering right for our checks to exit the loop, versus appending the decimal point to the output, to avoid weird looking outputs like divide(6, 2) -> "3.".

Trying this out:

>>> divide(1, 4)
'0.25'

It works! But we haven't yet handled infinite decimals, they result in an infinite loop:

>>> divide(1, 3) # Hangs!

Recurring digits

Because we're dividing integers, we cannot get infinitely varying decimal places. If we have an infinite number of decimal places, it must be because of a cycle of one or more recurring digits. To detect such a cycle we have to notice a couple of things.

First, simply seeing a digit in the output which we have seen before is not enough. Looking at the three assignments at the start of the above while-loop, which capture our state:

int_part = str(numerator // denominator)
remainder = numerator % denominator
numerator = remainder * 10

Here, int_part gets the value of each successive decimal digit. However if it takes on the same value as in a previous iteration, the accompanying remainder might be different, and it is the remainder which is used to generate the numerator for the next iteration, and hence generate the sequence of digits after this.

So, as we already knew from common sense, two iterations with the same int_part may go on to produce different sequences of subsequent digits. However, The value of remainder is the only thing which determines the inputs to our next iteration:

int_part depends on numerator and on denominator (which is constant)
numerator depends on remainder.

Hence, two iterations might produce different digits, but produce the same remainder, and from that point onwards, they will be in lockstep. If we find two such iterations, then we have detected an infinite recurring cycle of digits.

So, before the loop begins, initialize a dict:

# Remainders seen to date, mapped to their position in the result
remainders = {}

Then inside the loop, after everything else, use our new dict to detect if we have seen the current remainder before:

# If we have seen this remainder before, we are now in exactly the
# same state as then, so we have found a recurring digit sequence.
if remainder in remainders:
    # We have found a cycle of decimal digits! Insert parens into our results,
    # from the last seen position of this remainder, up to the current digit.
    last_pos = remainders[remainder]
    results = (
        results[:last_pos] +
        ["("] +
        results[last_pos:] +
        [")"]
    )
    break
# Remember the position at which we saw this remainder
remainders[remainder] = len(results)

Trying this out:

>>> divide(1, 3)
0.(3)
>>> divide(1, 7)
0.(142857)

OMG it works!

Defensive coding

We're putatively done, but the grumpy old dev in me is uncomfortable leaving that while True in there. By deduction, we always must eventually hit the if <condition>: break to escape from it, so ostensibly it's fine. But if I have a bug in the code or my reasoning, then it might lead to an infinite loop, in some scenario I'm not thinking of. Can we limit the number of iterations in some other, foolproof way? Turns out we can.

We've seen already that a repeated value of remainder means we can break from the loop. Also, notice that remainder, given by:

remainder = numerator % denominator

can only take values from 0 to denominator - 1. So it can have denominator possible values, and this is the maximum number of iterations we will ever need.

Hence, we can safely replace the while(True) with:

for _ in range(denominator):
    ...

Splendid! Much less anxiety-inducing

The source is on github.

SVG Trees using recursive Python functions

Jonathan Hartley — Fri, 28 Feb 2025 17:10:27 GMT

Inspired by a woodland hike under the first blue skies we've seen this year, I got home and showed the kiddo how to draw an SVG tree with recursive functions in Python.

At first the generated shape looked kinda lumpy and uninspiring, but it did demonstrate the principle. We were thinking of calling it a day, but I did a little bit of tweaking on parameters to control how each branch differs in length and direction from its parent. Suddenly, the generated shape really came alive, and started to look a lot more like the trees we'd seen on our hike that afternoon.

This image uses a recursion depth of 18, yielding 2^18 twigs, i.e. 250,000, which generates a 100MB SVG file. This takes about ten seconds to generate, and another ten to display in an SVG viewer. Alternatively, I can convert the SVG to a lossy webp, as displayed here, which is only 280kB and displays instantly.

Pushing the generation to greater recursion depth makes my SVG viewer and conversion tools start to stutter and barf. Presumably I could be smarter about the SVG I generate -- maybe generating the outline of the tree as points on fewer, more complex polygons, instead of a polygon for each branch segment? No matter, the artifact is the thing here, and it's done now.

Source is at https://github.com/tartley/tree-art.

Structured Pattern Matching in Python

Jonathan Hartley — Sat, 29 Jul 2023 23:15:26 GMT

I read through descriptions of structured pattern matching when it was added in Python 3.10 a couple of years ago, and have studiously avoided it ever since. It seemed like a language feature that's amazingly useful in one or two places, like writing a parser, say, and is a horrifically over-complicated mis-step just about everywhere else.

Update: A day after writing this I see that Guido van Rossum wrote exactly that, a parser, to showcase the feature. I'm guessing he writes a lot of parsers. I definitely don't write enough of them to think this language feature is worth the extra complexity it brings.

Regardless, I really ought to remember how it works, so this is my attempt to make the details stick, by writing about it.

If you're not me, you really ought to be reading about it from the source instead:

PEP 643: Specification.
PEP 635: Motivation and rationale.
PEP 636: A tutorial.

Basic structure

match EXPRESSION:
    case PATTERN1:
        ...
    case PATTERN2:
        ...
    case _:
        ...

This evaluates the match EXPRESSION, then tries to match it against each case PATTERN, executing the body of the first case that matches, falling back to the optional final _ default case. (match and case are not keywords, except in the context of a match...case block, so you can continue using them as variable names elsewhere.)

But what are PATTERNs, and how are they tested for a match?

Patterns

Patterns can be any of the following. As becomes increasingly obvious down the list, the real power of this feature comes from composing each of these patterns with the others. For complicated patterns, parentheses can be used to indicate order of operations.

Literals

Like other languages' traditional switch statement:

match mycommand:
    case 'start':
        ...
    case 'stop':
        ...
    case _:
        raise CommandNotFoundError(mycommand)

Such literal case patterns may be strings (including raw and byte-strings, but not f-strings), numbers, booleans or None.

Such cases are compared with equality:

match 123:
    case 123.0:
        # matches!

except for booleans and None, which are compared using is:

class Any:
    def __eq__(self, _):
        return True

myfalse = Any()

match myfalse:
    case False:
        # Doesn't match, even though myfalse == False
        assert False

Variable names

We can replace a literal with a variable name, to capture the value of the match expression.

match command:
    case 'start':
        ...
    case 'stop':
        ...
    case unknown:
        # New variable 'unknown' is assigned the value of command

The 'default' case pattern _ is just a special case variable name which binds no name.

Beware the common error of using "constants" as the case pattern:

NOT_FOUND = 404

match error:
    case NOT_FOUND: # bad
        handle_404()

The above case is intended to test for error == NOT_FOUND, but instead assigns the variable NOT_FOUND = error. The best defense is to always include a default catch-all case at the end, which causes the above NOT_FOUND case to produce a SyntaxError:

NOT_FOUND = 404

match error:
    case NOT_FOUND:
        handle_404()
    case _:
        pass

SyntaxError: name capture 'NOT_FOUND' makes remaining patterns unreachable

To use a 'constant' in a case pattern like this, qualify it with a dotted name, such as by using an enum.Enum:

match error
    case errors.NOT_FOUND:
        # correctly matches

Sequences

Using a list-like or tuple-like syntax, matches must have the right number of items. Like Python's existing iterable unpacking feature. Use * to match the rest of a sequence. Included variable names are set if a case matches by all other criteria.

match command:
    case ('start', name):
        # New variable name=command[1]
    case ('stop', name):
        # New variable name=command[1]
    case ('stop', name, delay):
        # New variables name=command[1], delay=command[2]
    case ('stop', name, delay, *extra):
        # New variables name=command[1], delay=command[2] & extra=command[3:]
    case _:
        raise BadCommand(command)

Mappings

Using a dict-like syntax. The match expression must must contain a corresponding mapping, and can contain other keys, too. Use ** to match the rest of a mapping.

match config:
    case {'host': hostname}:
        # 'config' must contain key 'host'. New variable hostname=config['host']
    case {'port': portnumber}:
        # 'config' must contain key 'port'. New variable portnumber=config['port']
        # Remember we only use the first matching case.
        # If 'config' contains 'host', then this 'port' case will not match.
    case {'scheme': scheme, **extras}:
        # new variables 'scheme' and 'extras' are assigned.

Case patterns may contain more than one key-value pair. The match expression must contain all of them to match.

    case {
        'host': hostname,
        'port': portnumber,
    }:
        ...

Objects and their attributes

Using class syntax, the value must match an isinstance check with the given class:

match event:
    case Click():
        # handle click
        ...
    case KeyPress():
        # handle key press
        ...

Beware the common error of omitting the parentheses:

match myval:
    case Click: # bad
        # handle clicks

The above case is intended to test for isinstance(myval, Click), but instead creates a new var, Click = myval. The best defence against this error is to always include a default catch-all at the end, which makes the Click catch-all produce an error by making subsequent patterns unreachable.

Attribute values for the class can be given, which must also match.

match event:
    case KeyPress(key_name='q', release=False):
        game.quit()
    case KeyPress():
        handle_keypress(event)

Values can also be passed as positional args to the class-like case syntax:

    case KeyPress('q', True)
        ...

If the class is a namedtuple or dataclass, then positional args to a class-like case pattern can automatically be handled using the unambiguous ordering of its attributes:

@dataclass
class Dog:
    name: str
    color: str

d = Dog('dash', 'golden')

match d:
    case Dog('dash', 'golden'):
        # matches

But for regular classes, the ordering of the class attributes is ambiguous. To fix this, add a __match_args__ attribute on the class, a tuple which specifies which class attributes, in which order, can be specified in a case pattern:

class KeyPress:
    __match_args__ = ('key_name', 'release')

event = KeyPress(key_name='q', release=False)

match event:
    case KeyPress('q', False):
        # matches!

As you might expect, the literal positional args can be replaced with variable names to capture attribute values instead:

match event:
    case KeyPress(k, r): # names unimportant, order matters
        handle_keypress(k, r)

Positional sub-patterns behave slightly differently for builtins bool, bytearray, bytes, dict, float, frozenset, int, list, set, str, and tuple. A positional value is matched by equality against the match expression itself, rather than an attribute on it:

match 123:
    case int(123):
        # matches
    case int(123.0):
        # would also match if it wasn't shadowed

Similarly, a positional variable is assigned the value of the match expression itself, not an attribute on that value:

match 123:
   case int(value):
        ...

assert value == 123

The values passed as keyword or positional args to class-like case patterns can be more than just literals or variable names. In fact they can use any of the listed pattern types. For example, they could be a nested instance of this class-like syntax:

class Location:
    def __init__(self, x, y):
        self.x = x
        self.y = y

class Car:
    def __init__(self, location):
        self.location = location

mycar = Car(Location(11, 22))

match mycar:
    case Car(location=(Location(x=x, y=y))):
        # matches, and captures 'x' and 'y'

assert x == 11
assert y == 22

Combine patterns using `|`

To match either one pattern or another:

    case 1 | True | 'true' | 'on' | 'yes':
        # matches any of those values

Capture sub-patterns using `as`

We've seen how we can either match against a value, or capture the value using a variable name. We can do both using as:

    case 'a' | 'b' as ab:
        # matches either value, captures what the value actually was

This might not be much use when capturing the whole match expression like that. If the match expression is just a variable, then we could instead simply refer to that variable. But using as can be useful when the match expression is lengthy or has side-effects:

match events.get_next():
    case KeyDown() as key_event:
        ...

or to capture just a component of the whole expression. Contrived example:

    case ('a' | 'b' as ab, 'c'):
        # matchs ['a', 'c'] or ['b', 'c'], and captures the first letter in 'ab'

An `if` guard clause

Add arbitrary conditions to the match:

    case int(i) if i < 100:
        # matches integers less than 100

Or, alternatively:

    case int() as i if i < 100:
        # matches integers less than 100

Complications

This feature seems rife with complexity. The flexible syntax of case patterns forms a new mini-language, embedded within Python. It has many similarities to Python, but also many initially unintuitive differences.

For example, a class-like case pattern such as case Click():. Anywhere else in the language, the expression like Click(...) would create an instance of the Click class. In a case statement, it instead is doing things like isinstance and hasattr checks.

Similarly, including variable names doesn't return the variable value as in ordinary Python. Instead it binds a value as that name. This is the source of the annoying gotcha described above, that bare "constants" like NOT_FOUND behave very unexpectedly when used as case expressions.

There are a few places in real-world code where structured pattern matching will produce nicer code than the equivalent using nested elifs. But equally, there are a lot of places where the elifs are a more natural match. Developers now get to choose which they're going to use, and then later disagree with each other about it, or simply change their mind, and end up converting code from one to the other.

If this was a simple feature, with low overheads, then I'd forgive its inclusion in the language, accepting the costs in return for the marginal and unevenly distributed benefits.

But it's really not simple. In addition to Python programmers all having to do an exercise like this post just to add it to their mental toolbox, it needs maintenance effort, not just in CPython but in other implementations too, and needs handling by tools such as syntax highlighters, type checkers. It really doesn't seem like a net win to me, unless you're writing way more parsers than the average programmer, which no doubt the champions of this feature are.

TIL: Format Python Snippets with Black.

Jonathan Hartley — Tue, 09 Jun 2020 19:36:58 GMT

Black, the opinionated Python code formatter, can easily be invoked from your editor to reformat a whole file. For example, from Vim:

" Black(Python) format the whole file
nnoremap <leader>b :1,$!black -q -<CR>

But often you'd like to reformat just a section of the file, while leaving everything else intact. In principle, it's easy to tell Vim to just send the current visual selection:

" Black(Python) format the visual selection
xnoremap <Leader>b :!black -q -<CR>

(Note that both the above Vim configuration snippets map the same key sequence -- leader (commonly comma) followed by lower case b. These can be defined simultaneously, because the second one uses 'xnoremap', meaning it is used only while a visual selection exists, while the first uses 'nnoremap', so is used all other times.)

But if the given code starts with an indent on the first line, for example if it comes from lines in the middle of a function, then this won't work. Black parses the given code into a Python abstract syntax tree (AST), and a leading indent is a syntax error - it's just not valid Python.

I filed a hopeful issue with Black, suggesting they could handle this case, but it was a long shot and hasn't gained much enthusiasm.

So, I present a tiny Python3 wrapper, enblacken, which:

Unindents the given code such that the first line has no indent.
Passes the result to Black.
Reindents Black's output, by the same amount as the original unindent.

See enblacken on github

Rhythmbox plugin: "Announce"

Jonathan Hartley — Mon, 16 May 2016 01:34:48 GMT

I use the Linux music player "Rhythmbox". This morning I wrote a plugin for it, called "Announce":

https://github.com/tartley/rhythmbox-plugin-announce

Every time a new song starts to play, it announces the title using speech synthesis. I like it when I'm listening to some new music I'm not familiar with, but am away from the computer. Then I can still know which track is which.

If the album or artist names are different from the previous track, then it includes those in the announcement, too.

PyRochesterMN

Jonathan Hartley — Thu, 21 Jan 2016 03:01:27 GMT

Announcing the world's newest Python User Group, PyRochesterMN, based in Rochester, Minnesota, USA.

Update: We've had a couple of name & hosting changes since:

~~http://www.meetup.com/PyRochesterMN~~

~~http://www.meetup.com/RochesTech~~

https://www.reddit.com/r/rochestertech/

Thoughts on Nylas' "How We Deploy Code"

Jonathan Hartley — Wed, 22 Jul 2015 00:06:02 GMT

Some thoughts on Nylas' post "How We Deploy Code."

The goals of making deployment consistent, reliable and fast are very laudable, and the conclusion involving creating Debian packages is just great. But in the spirit of geek nitpicking, I can't help but think the justifications given are misguided, and overlook a simpler solution.

>> pip does not offer a "revert deploy" strategy

Yes it does. Throw away the virtualenv and create a new one using the requirements.txt from an earlier release. This might be slow, but you can both speed it up (see below), and you can keep old versioned virtualenvs around, as a cache of the output of this slow step, so that reverts (the time when you really want deploys to go quickly) require only a couple of symlinks.

Update: Oooh, and I just had an idea. You could version virtualenvs using a hash of the requirements, so that deploys which do not change dependencies can share the same virtualenv. I've never tried that - it might work?!?!

>> Installing dependencies with pip can make deploys painfully slow

This is true. But it's not the final word on the matter.

Firstly, don't grab the dependencies you're installing from PyPI. Have a local cache of them. That speeds up the install tremendously, not just because no download takes place, but also because no trawling of PyPI and author sites for possible alternate versions takes place. Some people use a local PyPI instance, but I like simply using a directory full of packages. Point pip at it using 'pip install --no-index --find-links=packages -r requirements'. The 'packages' directory could be checked into your project's repo, so that once you've checked a project out, you have everything you need to deploy to local VMs, even with no network connection at all. I wrote about this a while ago.

>> Building your code separately on each host will cause consistency issues

So don't install dependencies using source packages, use wheels instead. Then any slow or unreliable build step is done once, when you create the packages directory, while deployment is now reliable and requires no development tools on the production server such as compilers, headers, etc.

Update: As a bonus, this will again substantially speed up the creation of your virtualenv when deploying.

>> Deploys will fail if the PyPI or your git server are down

PyPI outages (or a package author deleting their package, which happens routinely) are not a problem if you're deploying from a local source of packages.

I agree that application code deployment shouldn't be done using a 'git pull'. Accessibility to GitHub shouldn't be a single point of failure, and the fewer development tools needed on production servers the better. So export the code from git into a tar file when you cut a release, and then push it out using scp at deploy time.

Having said all that, there are still advantages to having your whole app and its dependencies handled by a single mechanism like Debian packages, rather than more bug-prone ad-hoc scripts and Ansible config. So I'm not at all against the final conclusions of the Nylas article. (Hugs to you folks at Nylas!)

Dashed this out in ten minutes between daycare pickup and dinner being ready. Apologies for the inevitable shoddyness.

Chaining a sequence of generators

Jonathan Hartley — Tue, 15 Oct 2013 20:09:54 GMT

I often gravitate towards solutions using a series of chained generators, in the style of David Beazley's 'Generator Tricks for Systems Programmers.'

This results in the outer level of my code calling one generator after another, terminating in something that consumes the rows, pulling data one row at a time through each of the generators:

inputRows = read()
parsedRows = parse(inputRows)
processedRows = process(parsedRows)
outputRows = format_(processedRows)
output(outputRows)

where each called function except the last is actually a generator, e.g:

def parse(rows):
    for row in rows:
        yield int(row)

This is great. But my itch is that the top level code above is a bit wordy, given that what it does is so simple. The reader has to check each temporary variable quite carefully to be sure it's doing the right thing.

Fowler's 'Refactoring' describes circumstances when it's good to remove intermediate variables, which results in:

output( format_( process( parse( read() ) ) ) )

This is certainly less wordy, and expresses what's happening very directly, but it annoys some of my colleagues that the called functions are listed in reverse order from what one might intuitively expect.

I've had this idea in my head to create a decorator for generators which allows one to chain them in an intuitive order, possibly using some unconventional notation such as:

read() | parse | process | format_ | output

where 'parse', et al, are now decorated with '@chainable' or somesuch, which returns an instance of a class that stores the wrapped generator, and overrides __or__ to do its magic. Maybe 'read' doesn't need to be invoked manually there at the start of the chain. I haven't really thought this through.

Luckily, before embarking on that, I realised today I've been over-complicating the whole thing. There's no need for decorators, nor for the cute '|' syntax. I just need a plain old function:

def link(source, *transforms):
    args = source
    for transform in transforms:
        args = transform(args)
    return args

Update: This code has been improved thanks to suggestions in the comments from Daniel Pope (eliminate the 'first' variable) and Xtian (take an iterable rather than a callable for the source.)

This assumes the first item passed to link is an iterable, and each subsequent item is a generator that takes the result of the item before.

If the final item in the sequence passed to 'link' is a generator, then this returns a generator which is the composite of all the ones passed in:

for item in link(read(), parse, process, format_):
    print item

Or if the final item passed to 'link' is a regular function, which consumes the preceding generators, then calling 'link' will invoke the generators, i.e. the following is the same as the above 'for' loop:

link(read(), parse, process, format_, output)

There's some rough edges, such as determining what to do if different generators require other args. Presumably 'partial' could help here. But in general, 'link' only needs to be written once, and I'm liking it.

pip install : Lightspeed and Bulletproof

Jonathan Hartley — Fri, 08 Mar 2013 15:29:09 GMT

I saw a post about speeding up the Python packaging command "pip install", by specifying more responsive mirrors for querying and downloading packages. For my situation, a better tactic is this.

Step one: Download all your project's dependencies into a local 'packages' dir, but don't install them yet:

mkdir packages
pip install --download=packages -r requirements.txt

Step two, install from the 'packages' dir:

pip install --no-index --find-links=packages -r requirements.txt

(The above syntax works on pip 1.3, released yesterday. Docs for older versions of pip claim to support this, but in practice, for pip 1.2, I've had to use "--find-links=file://$PWD/packages")

Step 2 works even if PyPI is unreachable. It works even if some of your dependencies are self-hosted by the authors, and that website is unreachable. It works even if the version you have pinned of one of your dependencies has been deleted by the author (some packages do this routinely after security updates.) It works even if you have no network connection at all. In short, it makes creation of your virtualenv bulletproof.

As a nice side effect, it runs really fast, because it isn't downloading the packages across the internet, nor is it attempting to scan a remote index to check for matching or newer versions of each package. This is much quicker than just using a Pip download cache, especially for large projects with many dependencies which only change occasionally.

At Rangespan, we check the 'packages' directory into source control, so that once you've checked out a project's repo, you have everything you need to deploy locally and run, even if you have no network. You might choose to treat 'packages' as ephemeral.

It was pointed out to me recently by @jezdez, Pip maintainer, this usage pattern has now been explicitly called out in the documentation, which was substantially reorganised and improved with the recent 1.3 release.