Assisting the Upgradement

I got burned by one of what turned out to be quite a list of small incompatibilities between PyQt4 and PyQt5. Just fooling around I tried upgrading one of Mark Summerfield's utilities to PyQt5. It contained the following code:

        path = QFileDialog.getOpenFileName(self,
                "Make PyQt - Set Tool Path", label.text())
        if path:
            label.setText(QDir.toNativeSeparators(path))

Pretty obviously Mark expected getOpenFileName to return a path or a null string. But when executed, and I clicked Cancel in the file dialog, it caused an error in the label.setText statement. Whatever got into path evaluated to True, but wasn't a string.

It turned out to be a tuple with two strings. I documented this to the pyqt mailing list and was embarrassed when Phil just replied with the above link to the list of incompatibilities, one of which is a change to the API of the whole family of five "get..." methods supported by QFileDialog. What had happened to cause this seemingly arbitrary breaking of an existing API? It seems that PyQt4 had introduced some variant methods "to avoid the need for mutable strings". Now these extra "get...Filter" methods were being dropped and their function folded into the basic "get..." methods. And that entailed changing the return value of getOpenFileName from a simple string to a tuple of two strings.

It still seems arbitrary to me, breaking existing code in an unexpected way for no very good reason. But it's a done deal, so how to make sure that this incompatibility, and all the other subtle incompatibilities in the list, don't get overlooked? (And don't miss the fact that one item in the list is open-ended, saying "PyQt5 does not support any parts of the Qt API that are marked as deprecated or obsolete in Qt v5.0." What are those? Are they numerous?)

I decided it wouldn't be hard to write a tool to find and point out all, or anyway a lot of, these issues. In two afternoons of work I put together q45aide.py (click the link to see the Readme and get the code from Github). This is a straightforward source scanner that copies a program and inserts comments above any line that looks as if it will have an upgrade problem.

I'm particularly pleased with two features of this program. One is the way of finding out the modules that contain every Qt class. I needed this because one annoying change from Qt4 to Qt5 is that many classes moved from one import module to another. That invalidates most existing from PyQt4.module import (class-list) statements. I wanted to generate correct, minimal import statements from the class-names used in the program. But that meant having a dictionary whose keys were all the valid Qt class-names (over 880 of them, it turns out) and whose values were the module names that contain them.

I pondered quite a while over how to get such a list of class-names by module. I thought about manually or programatically scraping some pages from qt-project.org. But finally I realized, I could build a complete, accurate list dynamically in the program.

When you import a module, Python creates a namespace. And the names defined in a namespace can be interrogated by querying namespace.__dict__. So the program contains code like this:

    def load_namespace( ):
        global module_dict, import_dict
        # pick off QtXxxx from "PyQt5.QtXxxx"
        module_name = namespace.__name__.split('.')[1]
        for name in namespace.__dict__ :
            if name.startswith('Q') : # ignore e.g. __file__
                module_dict[name] = module_name

    import PyQt5.Qt as namespace ; load_namespace()
    import PyQt5.QtBluetooth as namespace ; load_namespace()
    ...

This loads module_dict with exactly the 880+ class-names related to their include modules, automatically updating should PyQt5 be updated with new or changed class-names.

The other thing that I got a kick out of writing was the way to write a list of class-names in either of two formats, in one statement. The program will generate one "from PyQt5.modulename import (class-name-list)" for each module that the input requires. A program option is -v, asking for the list to be stacked vertically. The only difference is that the class-name-list is either punctuated with comma-space, or with comma-newline-indent. And this is how it comes out:

                out_file.write('from PyQt5.{0} import\n   ('.format(mod_name))
                join_string = ',\n    ' if arg_v else ', '
                out_file.write(join_string.join(sorted(class_set)))
                out_file.write(')\n')

Badda-boom.