Python Packaging

In his great article Hate, Hate, Hate Everywhere, Armin Ronacher recently explained the software history of python packaging leading up to its current state, which is kludgy.

When I install python, to get it "ready" I have to install four tools whose functionality should come with python: setuptools, pip, and virtualenv (and for development, virtualenvwrapper). The first two overlap in direct functionality (but not completely), and the last two overlap in targeted features and audience (but not completely).

This kind of superset/subset thought pattern carries through to the act of publishing a python module, and it's not easy. There are so many different platforms that unless you go to great lengths, your software is going to be incompatible somewhere. [*] Lots of hoops and incompatible instructions; time spent on google trying to figure out mandatory files called MANIFEST.in, where mistakes fail silently. This is where the hate comes from.

On the other hand, you eventually get to the good parts:

  • Pypi hosts and serves everything, no bandwidth charges or update restrictions. They even enforce good versioning practices.
  • Sphinx generates nice looking documentation and has excellent code discovery and publishing. [†]
  • Readthedocs takes your sphinx-compatible logic and builds/hosts your documentation for free, pulling it right out of your public repository. Remembers old documentation versions. You can have it trigger rebuilds on commit. Even has decent error messages.

They all just work.

It's easy when you're fighting to publish to lose sight of the good parts. Overall, it's not that bad! Yes, it's unnecessary overhead and occasionally frustrating. However, the polish in the ecosystem is focused in the correct places: the users of your software get a great experience.

$ pip install indeed
...
$ indeed fyeah
> "fyeah indeed"

The python packaging system is far better than the alternative: download, configure, adjust library paths, etc. You get to deliver something great without having to write the entire stack yourself. You get to have a great deploy and documentation system for free.

While it's not What more could you want?, it's certainly pretty good and free.

[*]The packages I've published recently probably don't work in a number of settings. It's part of the deal. And in my defense, I can't imagine that people want to filter mass quantities of gene names on their smartphone.
[†]Incidently, I enjoy RST so much that I now live here. It's just what I've always wanted.