Byteplay3 uploaded to pypi

The PyPi registration process turned out to be remarkably easy. I had to register as a user, something I'd never done despite hundreds of uses of pypi over the years. Then it was just python setup.py register and that was it.

It took a little while to show up in a search at PyPi, but here it is now. I do not understand why the descriptive paragraphs (which the PyPi "edit package" page clearly says, can use reStructured Text markup) are not formatted right. The rST markup for the example and the headings is just being ignored. Meh. Don't care.

What I did to Noam's code

Heh heh quite a lot, really. For one thing, I commented the bejeezus out of it. That was how I went about understanding it (and it took some understanding, lemme tell ya). I read it (and read it and read it) and then I commented what it was doing. Initially I peppered it with comments like #TODO what the bleep is this for? and the like, then gradually replaced all those with meaningful comments.

Prior to and during this work, I was watching Philip Guo's lectures on Python internals. Although these are based on Python 2.7, he leads you through the modules of CPython that are involved with compiling and executing code. So part of the experience of making byteplay work in Python 3 was discovering what about CPython internals had changed from 2.7 to 3.4. The changes are mostly cosmetic, for instance the names of attributes of the function object are different. A few bytecodes have been added and deleted. But the concepts are pretty consistent.

Function Signatures

Some significant changes involve the code object and its handling of the function signature. I'm not sure if the *args and **kwargs features are Python 3 innovations? But the Python 3 code object has properties to deal with them that are not in Python 2.

If the signature of a function has some arguments, then *args, then another argument:

def foo( a, b=False, *args, z=True ):

then the argument z is a keyword-only argument. It cannot be entered as a positional value; it must be specified with z=:

res = foo( 1, True, 2,3,4, z='z' )

The code object attribute kwonlyargcount is the count of such arguments. If the function has such arguments, it is nonzero. This attribute wasn't present in Python 2, and I had to add it to the Code object that is the centerpiece of byteplay.

Byteplay2 did know about the varargs attribute of a code object, a flag bit to say if it has a *args in its signature. (So, that's from Python 2.) However, Python 3 has added a varkwargs flag. That also I had to incorporate into the Code class.

All of these affect the interpretation of other parts of the code object. One crucial thing in a code object is a tuple, varnames, composed of the names (strings) of all local variables. The sequence of names in this tuple is:

• Names of normal arguments, ones that precede a *args or **kwargs item, in their declared order. From the example above, a and b.
• Names of keyword-only arguments, if any. In the example, z
• Name of a *args argument, if any (it doesn't have to be args).
• Name of a **kwargs argument, if any.
• Names of other local variables.

The varnames tuple is rather important because bytecodes that reference a local, such as STORE_FAST, have as their argument an index to this tuple. But the composition of the tuple is not drop-dead simple. The Code object method from_code() takes one apart and stores the pieces; and the to_code() method assembles the pieces to recreate a code object. And that had to change a bit for Python 3.

And now I've reminded myself, that my unit test suite lacks a test of function signatures with *args and **kwargs. I think I shall go and do one now...