Success with a Nuitka crash; Stymied by stderr

The Nuitka compile of Cobro died looking for the Tk and Tcl framework folders in the wrong place, because otool reports incorrect paths. I put symlinks in /Library/Frameworks pointing to the folders in /System/Library/Frameworks, and ran the compile again. It completed, and produced a 9MB executable. I tried executing it with the --help option, and it ran and responded as expected. Of course this minimal test only gets as far into the __main__ code as the call to parseargs.

Alas, when I ran it properly, it immediately crashed with a segfault. Somewhere in the construction of the numerous UI objects it dies. I sent a rather dejected note with the crash report to Kay Hayen, Nuitka's author. He quickly replied pointing out that one of the final calls in the stack was to "pickle". He said Nuitka uses pickle, and perhaps that is somehow conflicting with PyQt's use of pickle.

PyQt does pull in pickle among the couple of hundred other modules Nuitka lists. But just then I couldn't see a connection. This was Friday afternoon.

Around 4 or 5am today, Saturday, I was half-awake and thinking about this (obsessing about software at 4am is not uncommon for me; well, there are worse things to obsess about when awake in the wee hours...) and it occurred to me that one place PyQt might use pickle is in converting between Python data types and Qt datatypes. And that the program crashes about the point where it would be fetching values from the QSettings. Most of what it pulls from settings is character strings, but it also keeps the SHA-1 hash signature of each comic in settings. These are byte types in Python, but PyQt would have to convert them to QByteArray types. Maybe it uses pickle for this?

So after breakfast I trotted to my computer and composed a ten-line test case: make the QApplication, make the QSettings, push a byte string into the settings and fetch it back.

Nuitka compile. Completed. Run the resulting testcase.exe (Nuitka gives its output binary the .exe suffix even in Mac or Linux).

Boom! Instant segfault, with precisely the same stack trace ending in pickle that the big program produced!

I tell you, I was chuffed. A lovely little test case that crashes in exactly the expected way, first time out of the box. Niiiiice! Shot that off to Kay who I expect will be as pleased as I.

Not quite such a success was my attempt to capture and stifle the stderr output of Cobro. One can quite easily redefine stderr in Python. You just assign something that behaves like a file to sys.stderr. That's all. I did this, assigning a stringIO object to it, and ran the program. Out comes the "can't find DjVu" and "can't find QuickTime" plug-in messages on the console. Nothing captured in the stringIO object.

So the code that emits at least those messages has its own copy of the program's stderr handle. It doesn't go through the one kept by the sys module.

One way to do this would be to have a two-level program. The "cobro" app would be a shell that used the subprocess module to launch the "real" cobro app in a subprocess. Then for sure the shell could capture all the stderr output of the subprocess. Well, probably for sure. Who knows what those CGContextSaveGState messages are capable of?

I really do not want to complicate the program in this way but I might try it just to see if it worked.