Finally! All platforms bundled.

Thursday, my Elance contractor delivered hunspell for both 32- and 64-bit Windows and Python 3.4. That dropped in and worked fine, and PPQT ran nicely from the command line. So then I could begin working with PyInstaller on Windows 7. Yesterday and today I discovered and circumvented four bugs in it, all unique to running under Windows, or the combination of Windows and Python 3.

First off there were two of the "hook" files that used a wrong path to import the PyInstaller windows "utils" module. Clearly at some recent point that module was moved within the PyInstaller folder, and somebody forgot to update all the hook files.

Next, it ran but the bundled app couldn't start, "module SIP not found". Now, every PyQt5 module needs SIP (the C++ shim that PyQt uses to cross from Python to the Qt binaries), and every one of the several PyQt5 "hook" modules that was being called named "SIP" as a hidden-import. Why wasn't it being bundled? I did not resolve this question, but I did circumvent the problem simply enough: I just added --hidden-import=sip to the PyInstaller invocation line. That was all it took to make the bundled app run, and wasn't that a lovely sight?

While investigating that, I tried to use the pyi-archive_viewer script that is included with PyInstaller. It lets you examine a bundled app to see what was actually included in it. Or it should; but I quickly found that it couldn't execute one of its basic functions, because it was trying to compare a user input string against a class member that was in bytes format. In Python 2, that worked. In Python 3 it doesn't, because Python 3 requires a clear distinction between strings of bytes and strings of characters, which are Unicode. It's one of the most common issues when converting from Python 2 to Python 3, and this comparison had been overlooked. I reported it and applied a quick one-line source change to get around it.

Once I patched that point in the archive viewer, it immediately turned up another error: when it tried to open a sub-archive it threw a run-time error exception because some "magic number" that it used as a signature didn't match. I traced this far enough to see that the magic number calculation had a three-level if statement, in principle saying "if this is Python 2, do it this way; elif this is Python 3 and the version is less than or equal to 3.3, do it that way; else it's Python 3.4 or above and do it this other way." I'm pretty confident I'm the first person to try this code on Windows and Python 3.4, so I just opened an Issue pointing to that code. Having gotten around the missing SIP problem, I no longer needed the archive viewer so I moved on.

One more step, then. I could bundle to a folder; but could I bundle to a single file .exe? Preferably one using my cute little Marvin icon? So I ran PyInstaller with that option—and it threw an exception. Oh, pooh. The exception was another very typical Python 3 compatibility problem, "str type does not support buffer protocol". This error gets thrown whenever you try to feed a string type to a file that has been opened with the "b" raw-bytes mode. In Python 2 you could do that because both str and byte types were aliases for a C char *. In Python 3, bytes still means that, but str means "16- or 32-bit Unicode characters" and you can't just feed them into a bytes file. You have to tell Python how to encode the characters into a byte-stream, for example by coding bytes(str_var.encode('UTF-8')).

I traced the error to a call to the win32api module. PyInstaller was trying to update a portion of the Windows "manifest" (whatever that is) using the UpdateResource Win API call. The win32api module is another open-source project; PyInstaller is just using it. And that module was accepting a string type as an argument to this UpdateResource method, and then (it appears) trying to feed that string into a file opened as bytes, and causing an exception. The bug is in that module. But I circumvented it by changing the code of PyInstaller so that it passed the string encoded to bytes.

And with that monkey-patch thrown on, it ran and produced a lovely single-file PPQT2.exe file with cute little Marvin icon.!

So now I have successfully bundled the app for all three platforms and put them up on my Public dropbox folder. There is nothing but nerves standing between me and announcing the availability of the alpha test publically. I will probably wait until Monday to do that.