Saturday, May 31, 2014

What a tangled web we weave...

The mainwindow class defines everything to do with the File menu: new, open, close, save-as, and a submenu of "recent" files. The latter is a feature I often find useful. I don't have to remember anything about a file except that not long ago I edited it. Just go to the end of the File menu and there it is, without having to navigate the file system to find it. However this is a feature that has some subtle implementation details, some of which I am only just now remembering as I start to code the version 2 of it. I should have reviewed V1 more carefully.

When coding the initial form of this I decided rather hastily that I would keep the list of "recent" files as a dict with filenames as keys and their paths as values: {filename.typ:path-to-file}. Of course you see immediately the problem with this, don't you?

Duplicate filenames! What if the user opens ~/documents/fred.txt and (then or another day) opens /pgdp/history/fred.txt? They are (presumably) different files but a Python dict allows only one value for the key fred.txt. Looking back at V1 code I see that I kept a simple list of full pathnames, with the most recent at the front. Much better.

Unfortunately the {filename:path} dictionary idea is in several places in the code and in the unit test driver. So now I am looking at changing 50 lines of code or more scattered in two or three modules. Because I didn't take the time to look at the working V1 code. Oh, bleagh.

Another subtlety of the recent-files list is, what if the file no longer exists? Or, much the same thing, what if it was opened from a mountable drive (e.g. a USB dongle) that isn't mounted just now? The file could appear or disappear between uses of the File menu.

The answer to that is, that the Recent sub-menu has to be (re)built dynamically, on the fly, when the File menu is about to be displayed. There is an aboutToShow signal emitted by the File menu action itself. The slot for that signal must have the code to clear and repopulate the Recent sub-menu with an action for each file in its list that: (a) is not already open but (b) still exists and is accessible. (Note that if a file is not now accessible, it doesn't go in the menu, but does stay in the list, so if it comes back at some future time, we'll show it.) I modeled the V1 code after an example in Summerfield's book. I remember thinking at the time, really? All this code gets executed between the mouse's click on the word File, and the painting of the menu? And nobody notices a delay? I still think it's remarkable.

A related issue is one that did not arise in V1. When the user selects File>Open and chooses a file, what to do if that's a file that's already open? We do not want to open a second copy for sure. But that means it needs to be easy to detect when a chosen filepath is identical to one of the possibly-several files already open. If it is, don't proceed with the open, but do "focus" that file: make it the currently selected tab in the edit tabset, so it's visible.

Yet another difference from V1: when the close signal is received, V1 looked to see if its one-and-only document was modified, and if so, gave the user the choice Yes to save, No to not save and continue with the Quit, or Cancel to not Quit. But with V2, it is possible there are multiple modified documents at Quit time. So the warning message at least needs to say how many modified documents there are. But should a Yes reply mean, save everything? That is, do a File>Save for each modified document? A problem with that is, some of them might be modified New documents with filenames of "Untitled-n" and no related path string. They need to be treated as Save-As, with a file-save dialog so a proper name and folder can be chosen.

Or, should the close event just loop through all open documents, and present a separate modal dialog for each one that's modified: "File filename.typ is modified. Save it now?" Yes/No/Cancel. Or better, Save/Don't Save/Cancel Quit. For normal files, clicking Yes gets an instant save. For "Untitled-n" documents, a Yes is immediately followed by presentation of the standard Save-As dialog.

That makes everything clear, but it could mean that the user who tried to Quit or clicked the [x] button in the window border is presented with a sequence of modal dialogs one after another. If the user is under stress (told to hurry up and get off that machine for some reason) this could be quite annoying. But what alternative is there?

BBEdit's way

Well, here's one alternative. When I tell BBEdit to quit with two open, modified documents, it... quits! No dialogs, no warnings. And the files on disk do not reflect the changes, so there was no secret saving going on.

When BBEdit is restarted, it shows a small progress message "Restoring BBEdit State" and then it opens the two modified documents and shows them with the modified text and a state of modified. How does it do that?!? It must save a secret copy of any modified file so it can recover that state on startup. But where?

Friday, May 30, 2014

Dude, where's my tab?

Well, rolling along here at ppqt central, we have the basics of a main window, now it is time to add the File menu and the actions it commands. First up is File>New. Easy-peasy one would think, since the _new() method is already in place and being used as part of initialization.

        self.file_menu = self.menu_bar.addMenu(_TR('Menu bar', '&File'))
        # Populate the File menu with actions.
        #  New -> _new()
        work = self.file_menu.addAction( _TR('File menu','&New') )
        work.setShortcut(C.CTL_N)
        work.setToolTip( _TR('File:New tooltip','Create a new, empty document') )
        work.triggered.connect(self._new)

Add a QMenu('File') (appropriately translated) to the application QMenuBar. To it, add an action 'New' and connect its signal to the proper slot. And this should work. So test it. Up comes the app and oooh! It has a File menu with only one item, New. And selecting that executes the code of the _new() method. I have verified this and also that each time File>New is selected, the app creates an empty Book with the names Untitled-0, Untitled-1, Untitled-2 and so on, and adds them to the edit tabset on the left side. And the tabset.addTab() method returns a tab index of 0, 1, 2 as expected.

Except ... the tabs don't appear. The added tabs are not shown; the only visible tab is the first one, for "Untitled-0". Wut?

The code for adding a New document is exactly the same as for adding an existing opened file, and that works. The unit test driver fakes up settings showing two previously opened files and the app opens them and they appear in separate tabs (as shown two posts back).

OK run the unit test and it opens two existing books. Now do New. Hmm. The debug printout reports "added Untitled-1 at index 1" even though there are two open books, so the name should be "Untitled-2" and the index, 2. Add the printout to the _open() method as well.

added realbook.txt at index 0
added small_book.txt at index 1
added Untitled-1 at index 1

No, that's not right. Should be Untitled-2 at index 2. It's like the _new() was happening in a completely different Mainwindow object, operating on a different book sequence number and different tabset. Could that be?

Yes it could be! The unit test driver created three MainWindow objects in sequence. The first is used to check that various settings are properly saved, like this:

settings.clear()
mw = mainwindow.MainWindow(settings)
...do things like mw.move(new position)...
mw.close() # force writing settings
...read values out of settings and assert things about them...
mw = None # destroy main window

And then another round of that: load the QSettings with fake info; create a new MainWindow object in mw; close it; look at settings values; assign mw=None.

Then finally do that one more time: load the QSettings object this time with two valid "previously-open" files; create mw=mainwindow.MainWindow(settings); call app._exec() and interact with the window.

If the first two rounds are commented out, so only the third MainWindow is created, it works: File>New makes a new file that appears in the edit tab bar.

Conclusion: somehow the QApplication is holding onto one or both of the first-created MainWindow objects. And it remembers how that (or those) objects created File menus. And somehow the File>New action is being directed to the local data owned by one of those zombie main windows, even though both of them had executed their .close() methods and all references to both had been overwritten in the Python code.

So: split the unit-test into two or three separate programs... boring.

Edit: On later consideration, I think the link is in the File menu QActions. Each QAction has a reference to the "slot" to handle its the triggered() signal. Each slot is specified using the main window's "self" pointer, e.g.

    act.triggered.connect(self._new)

That "self" value is a reference to the current instance of the MainWindow class, a QWidget derivative. Each time I created a MainWindow instance, it initialized by creating a QMenuBar and putting File>New and File>Open actions in it. Somehow the first of these "stuck" in the app's memory, and kept that object alive, not garbage-collected, even though I'd deleted every reference I had to it. Any use of the Mac OS menu bar's File>New, regardless of the fact that MainWindow #3 was running and active, still invoked the code in the zombie MainWindow #1 (or #2?).

Monday, May 26, 2014

Don't think of it as a bug...

...think of it as material for a blog post!

OK so there is an annoying bug in the edit display. I am putting a pale highlight on the current line (the line with the cursor) -- see the image on this post. This fixes an annoyance I had with version 1, that on a large screen it was easy to lose track of the cursor, especially after paging up or down. I'd often have to rattle the left and right arrow keys so I could spot the cursor moving.

Here's the basic code, cut down:

Note: The following is not the correct way to set a current-line highlight. Do not emulate this code. See this post for the problem with it and a later post for the correct approach.

    def _cursor_moved(self, force=False):
        tc = self.Editor.textCursor()
        self.ColNumber.setText( str( tc.positionInBlock() ) )
        tb = tc.block()
        if tb == self.last_text_block and not force :
            return # still on same line, nothing more to do
        # ...here fill in line-number, scan filename, and folio widgets...
        # Clear any highlight on the previous current line
        bc = QTextCursor(self.last_text_block)
        bc.setBlockFormat(self.normal_line_fmt)
        # remember this new current line
        self.last_text_block = tb
        # and set its highlight
        tc.setBlockFormat(self.current_line_fmt)

That's pretty simple: if the cursor is on the same line as last time, set the column number widget and bug out quick. (The force argument is so this same method can be used in certain rare cases by other routines to repaint the widgets even if the cursor hasn't moved.)

If the cursor is on a different line than the last time it moved, which we will know because the QTextBlock returned by the current cursor is not the same as before, then clear out the highlight on the previous line/block, and set a highlight on the new line/block. It never sets a highlight without also clearing one.

And it mostly works. Move the cursor with the arrow keys or by mouse-clicking, and the pale highlight moves around with it. Great!

Except in one case: if I select multiple lines—by dragging or by shift-clicking or by shift-down-arrowing—then all selected lines get the current-line highlight and it does not clear if I click somewhere else. It only clears if I run the cursor back over those lines. And here's the kicker: as I drag or shift-down-arrow to extend the selection, the line-number widget is updating properly. So the code above is being executed, recognizing that it is on a different line (or it wouldn't update the line-number widget), which means it must be clearing the highlight. But it doesn't. WUT?

Time to put in the print statements.

        dbg1 = tb.blockNumber()
        dbg2 = self.last_text_block.blockNumber()
        print('clearing block {0} setting block {1}'.format(dbg2,dbg1))

And it reports exactly what it should: Put the cursor on line 0, key shift-down-arrow to make a selection, it reports "cleaning block 0 setting block 1, clearing block 1 setting block 2" and so on, but the highlights on line 0 and 1 are not clearing. They are partly hidden by the brighter selection highlight, but there. Here's a picture.

If I click on line 4, the highlight on line 2, the last current line, is cleared, as is the selection highlight, but the highlights on lines 0 and 1 remain.

So there's no logic error, but some issue with setting a block format when the block is under another format, the selection highlight. I note that setting the current_line_fmt works, even when selection is in progress. You can see it sticking out from under the selection on line 2 above. But setting the normal_line_fmt doesn't take. What's the difference? It is initialized so:

        self.normal_line_fmt = QTextBlockFormat()

It's an empty, default format object. Whereas the current line format is set up:

        self.current_line_fmt.setBackground(colors.get_current_line_brush())

It is a format object with an explicit background brush. (The colors module returns basically QBrush(QColor('#FAFAE0')); eventually the user will be able to choose this color as a Preference.) Anyway the current line format has an explicit background brush and the normal format does not. Should I give it one, and if so, what color? Back to the Qt Assistant...

Answer: No. I changed the initialization of the normal format:

        self.normal_line_fmt = QTextBlockFormat()
        self.normal_line_fmt.setBackground(QBrush(Qt.white))

I also tried Qt.color0, the transparent color. The bug continues. So it is not the lack of an explicit background brush that prevents the clearing. Back to the Assistant...

Ah. Re-reading the doc for QTextCursor.setBlockFormat() I note it reads "Sets the block format of the current block (or all blocks that are contained in the selection) to format." Oops.

The problem, I betcha, is in the line

tc.setBlockFormat(self.current_line_fmt)

Because tc is the actual cursor of the current document, the cursor that represents the current selection. And when it is used to set the block format of the "current" block, it sets a block format for "all blocks in the selection" which includes the prior block whose format was just cleared!

And that's it. If I set the current line format using a new cursor with no selection, it all works. So the final lines of code now read:

        # clear any highlight on the previous current line
        temp_cursor = QTextCursor(self.last_text_block)
        temp_cursor.setBlockFormat(self.normal_line_fmt)
        # remember this new current line
        self.last_text_block = tb
        # and set its highlight
        temp_cursor = QTextCursor(tb)
        temp_cursor.setBlockFormat(self.current_line_fmt)

Make a new text cursor based on the current block, one which has (by default) no selection. Use that to set the format for the current line. It affects only the actual current line, not any other lines that might be in the current user selection. Bug fixed! Do a commit and break for lunch!

Friday, May 23, 2014

Mainwindow debut

The main window and the Book classes, and the relationships between them, are now pretty well laid out and functional, as proof of which see:

Click to see the whole retina-scale megillah

Note there are two documents open (two tabs on the left side). This is the first actual user-visible enhancement beyond version 1.

There are a few minor issues immediately apparent when trying it out, but they should be easy to sort out. (The editview seems to have regressed a bit, for one thing.) After fixing those, the next piece of code to write is the File menu and a few actions for it.

So, whee! And a pleasant Memorial Day Weekend to those in the USA, and a pleasant ordinary weekend to anyone else!

First Python2 to Python3 Problem

PPQT, like its inspiration Guiguts, maintains a bunch of metadata in a separate file. When saving bookname.txt it also saves bookname.meta with things like the user's notes, the file positions of all page breaks, and much else. This is an inherently fragile scheme because nothing links the two files except their names. The operating system, for example, has no idea the two files should always be copied together. It works adequately because the users are schooled in the habit of keeping everything about one project in one folder, so there's rarely a problem.

However, in the early days of PPQT an early adopter exposed the weakness when he had to restore a project from backup, and restored the book file but not the meta file. So all the metadata were wrong, page breaks in the wrong place, etc.

In response, I added a simple hash signature. On saving a file, PPQT takes an SHA-1 hash of the document text, and writes the hash signature into the meta file. On opening a file, the text is hashed and the signature compared to the metadata. If they differ, the user gets a strongly-worded warning message.

The code to output the signature is basically this:

    cuisineart = QCryptographicHash(QCryptographicHash.Sha1)
    ...
    cuisineart.addData(the_document.toPlainText())
    meta_stream << '{{DOCHASH '
    meta_stream << bytes(cuisineart.result()).__repr__()
    meta_stream << ' }}\n'

("Cuisineart" is the name of a line of food processors. Ha ha.) So the whole text of the book file is poured into the blender. A signature was obtained as cuisineart.result(). This is a QByteArray. In PyQt4, one needed to coerce that to a Python bytes() class. That value's .__repr__() converted it to something printable. The result was a line in the .meta file like this:

{{DOCHASH '\x9a\x9fjG\x99\xd0\x1b\xea\x84\xdeT\x8f:\xb8\xfb\xd5\x06\x82\x10|' }}

Does anyone see the problem with this? In Python 2, the __repr__() of a bytes value is a character string. In Python 2, we still used the old comfortable, sloppy assumption that a char was a byte was a char.

In Python 3, it is a bytes string, because Python 3 draws a sharp distinction between characters—which are numeric tokens that stand for letters and have a bit precision somewhere between 8 and 32—and bytes, which are 8-bit unsigned numbers with no defined character representation.

Practical result? Code identical to the above, executed by Python 3, produces this:

{{DOCHASH "b'\\x9a\\x9fjG\\x99\\xd0\\x1b\\xea\\x84\\xdeT\\x8f:\\xb8\\xfb\\xd5\\x06\\x82\\x10|'" }}

Even when the byte values started out identical, '\xde\xad' != "b'\\xde\\xad'".

In PPQT2, metadata reading is distributed. Various modules "register" to read and write different metadata sections. A reader function is called when a line starts with {{SECTION. It is passed the SECTION value, a version number, and a "parm" containing whatever string followed the SECTION on that line. In the case of the DOCHASH section, the parm is the signature string. The reader for the DOCHASH section began like this:

    def _read_hash(self, stream, sentinel, v, parm) :
        cuisineart = QCryptographicHash(QCryptographicHash.Sha1)
        cuisineart.addData(the_document.toPlainText())
        if parm != cuisineart.result().__repr__() :
            '''issue horrible warning to user'''

And of course, being executed in Python 3, it didn't work; the repr of the new signature has a b in it. Fortunately there is the version parameter. That is read earlier in the metadata stream. In early files it's omitted and defaulted to 0; later files have it as {{VERSION 0}}. So the DOCHASH reader can do this:

    def _read_hash(self, stream, sentinel, v, parm) :
        if v < '2' :
            '''do something to make an old signature compatible'''

But, um, what? Experimenting on the command line, we find this:

 >>> b = b'\xde\xad\xbe\xef'
 >>> c = '\xde\xad\xbe\xef'
 >>> b == c
 False
 >>> # very well, coerce c into bytes
 >>> bytes(c)
Traceback (most recent call last):
  File "", line 1, in 
builtins.TypeError: string argument without an encoding
 >>> bytes(c,'Latin-1','ignore')
b'\xde\xad\xbe\xef'
 >>> bytes(c,'Latin-1','ignore) == b
True
 >>> b.__repr__() == bytes(c,'Latin-1','ignore').__repr__()
True

So the answer is to convert the old char value into a bytes value. The bytes() function insists that I say how the chars are encoded. This is reasonable for the general case, but I happen to know these chars aren't chars; they're just bytes. I looked for an encoding type that would convey that, but didn't see any. So I use 'Latin-1' with 'ignore' to say, if some byte isn't Latin-1, just pass it along thankyouverymuch.

I think this'll work:

    def _read_hash(self, stream, sentinel, v, parm) :
        if v < 2 :
            parm = bytes(parm,'Latin-1','ignore').__repr__()
        cuisineart = QCryptographicHash(QCryptographicHash.Sha1)
        cuisineart.addData(the_document.toPlainText())
        if parm != cuisineart.result().__repr__() :
            '''issue horrible warning to user'''

Wednesday, May 21, 2014

Not Dead, the Blog

Apologies for the gap between posts. A couple of good coding sessions and I nearly have the Book and Main Window modules under control. A bit of tidying and I will have something to show in a day or so.

In the meantime, while working with the tedious and complex business of opening a file, I updated my improved QTextStream class. This started as a simple work-around of the annoying problem that a QTextStream, when based on a QFile, does not take ownership of the QFile. If the Python programmer carelessly lets the QFile go out of scope, the next attempt to use the QTextStream: what? produces a log message? returns an error code? Oh no no! It seg-faults the Python interpreter.

But in the course of using the class several times and ways in the main window code I found it expedient to keep adding simple functions to it, so as to avoid having to create nonce QFileInfo objects. Here's it's latest incarnation.

class FileBasedTextStream(QTextStream):
    def __init__(self, qfile):
        super().__init__(qfile)
        self.saved_file = qfile
        self.qfi = None # may never need this
    def rewind(self):
        self.seek(0)
    def writeLine(self, str):
        self << str
        self << '\n'
    def basename(self):
        if self.qfi is None:
            self.qfi = QFileInfo(self.saved_file)
        return self.qfi.baseName()
    def filename(self):
        if self.qfi is None:
            self.qfi = QFileInfo(self.saved_file)
        return self.qfi.fileName()
    def filepath(self):
        if self.qfi is None:
            self.qfi = QFileInfo(self.saved_file)
        return self.qfi.absolutePath()

Thursday, May 15, 2014

Thrashing

Work on PPQT has been broken up this week by other events and responsibilities. In the short times I've had to work on it, I've been frustrated by the complexity of the work. I've structured the project pretty well so that pieces can be developed independently and in order—mostly. It has worked that way until now.

Unfortunately there is a tight, broad-band relationship between the Main Window and the Book, and as I got into coding bits of the mainwindow module I increasingly had to go into the book module and add things that were blithely commented as #TODO weeks or months ago. Some of these have mushroomed in size, as TODO items are wont to do. So I found myself deeply nested in logical parentheses without any big time-chunks to work out of them properly.

So what I need to do is finish adding code to the book module, and update its unit-test driver to exercise all the added code, before I can go back to the main window. Also the little function I've added to what was a stub mainwindow.py has broken the unit test of at least one other module. So that needs to be resolved. And find a way to carve out multi-hour chunks to get properly stuck-in to these tasks. Tomorrow and next week I hope.

Friday, May 9, 2014

Joys of Parenthood

So I was doing some test spadework before launching into a lot of code in the mainwindow module, just to make sure I actually could actually accomplish what I want to accomplish. And I found something that puzzled me a lot, and taught me something.

The basic UI for PPQT2 is a refinement of the previous version: a single window divided into two horizontal panes by a splitter. In V.1 the left pane is the single edit window. The right pane contains a tab-set with a bunch of activity panels. There's the image view panel that shows the OCR scan images, the notes panel where the user can jot notes about the project, the find/replace panel, etc. etc.

So in V.2, the left pane will be not just one edit widget, but a tabset that might contain one edit widget, or two, or however many documents the user wants to open. But what about the right pane? The activities there are used with all documents, but they contain document-unique items. The image view should show the scan images for the document that has the focus. The notes panel should show the notes for the document that has the focus. And so on.

Whenever the user selects a different edit widget from the tabset on the left, all the panels on the right need to change their contents to match. (Only one of eight needn't change when a different document comes into focus, the Help panel.) I thought a bit about how to make this happen.

One way not to do it, would be to design the different view objects, imageview and pageview and notesview and so on, to have some way to switch to a different model object. "Hey, image view, use the pagedata from this other book; hey words view, use the word counts from this other book, etc." Although conceptually a reasonable use of the model/view paradigm, this would complicate the coding of those view classes, and worse, has a performance implication. Some of the views (word view especially) consist of QTables with thousands of rows, and these can take a long time to repopulate. Switching from one 15,000-word vocabulary to a different 15,000 word vocabulary (even if the word lists were substantially the same, as in switching between two versions of the same book), could cause an unpleasant delay. Plus, there are more things than just the data contents that have to change. Perhaps in one book the user has sorted the words table ascending, case-sensitive, and in another book, descending and ignore-case and filtered to show only spellcheck errors. All the settings of these incidental controls need to switch, smoothly and quickly, when the user moves from one document to another.

So what I'd planned to do was to represent each open document with a Book object; and have that Book be the owner of a complete set of objects related to that document, models and views alike. It will own a notes view, an image view, everything—even its own Find/Replace panel, because the user will have set different options and have different lists of recent find/replace strings, for that document.

When the main window finds out that the edit view of a given document has received the focus, the main window needs to go to that document's book, and replace all the activity tabs on the right with the corresponding objects for the book where the user just clicked. They would be sitting in memory, all initialized and ready to show, so this should not take much time. But is this kind of wholesale replacement of tab contents feasible? I wrote a little test program to find out. And here it is in its entirety. Read it; copy it and run it if you like.

from PyQt5.QtCore import Qt, QPoint, QSize
from PyQt5.QtWidgets import QLabel, QMainWindow, QSplitter, QTabWidget

class mypanel(QLabel):
    def __init__(self, parent, text):
        super().__init__(text,None) #<-- see discussion
        self.owner = parent
        self.setFocusPolicy(Qt.StrongFocus)

    def focusInEvent(self,event):
        event.accept()
        self.owner.i_got_focus(int(self.text()))

class MainWindow(QMainWindow):
    def __init__(self, settings=None):
        super().__init__()
        self.editview_tabset = QTabWidget()
        self.panel_tabset = QTabWidget()
        # Create the splitter that contains the above two parts.
        self.splitter = QSplitter(Qt.Horizontal)
        self.splitter.addWidget(self.editview_tabset)
        self.splitter.addWidget(self.panel_tabset)
        # Set that splitter as the main window central (and only) widget
        self.setCentralWidget(self.splitter)
        self.resize(QSize(600, 600)) # make a reasonable size
        # Make some widgies
        self.pinks = []
        for j in range(4):
            self.pinks.append( mypanel(self, str(j)) )
            self.pinks[j].setStyleSheet("color:'#000080'; background:'#e0c0c0';")
        self.blues = []
        for j in range(4):
            self.blues.append( mypanel(self, str(j+4)) )
            self.blues[j].setStyleSheet("color:'#600000'; background:'#00e0d0';")
        # On the left, one pink and one blue with pink one active
        self.editview_tabset.addTab(self.pinks[0],self.pinks[0].text())
        self.editview_tabset.addTab(self.blues[0],self.blues[0].text())
        self.editview_tabset.setCurrentIndex(0)
        # On the right, two pinks to match the focussed one on the left
        self.panel_tabset.addTab(self.pinks[2],self.pinks[2].text())
        self.panel_tabset.addTab(self.pinks[3],self.pinks[3].text())

    def i_got_focus(self,num):
        print('num=',num) # document what's happening
        self.panel_tabset.setUpdatesEnabled(False)
        if num == 0 : # pink left widget got focus, make right side pink
            self.panel_tabset.removeTab(0)
            self.panel_tabset.insertTab(0,self.pinks[2],self.pinks[2].text())
            self.panel_tabset.removeTab(1)
            self.panel_tabset.insertTab(1,self.pinks[3],self.pinks[3].text())
        elif num == 4 : # blue left widget got focus, make right side blue
            self.panel_tabset.removeTab(0)
            self.panel_tabset.insertTab(0,self.blues[2],self.blues[2].text())
            self.panel_tabset.removeTab(1)
            self.panel_tabset.insertTab(1,self.blues[3],self.blues[3].text())
        self.panel_tabset.setUpdatesEnabled(True)

from PyQt5.QtWidgets import QApplication
app = QApplication([])
mw = MainWindow()
mw.show()
app.exec_()

If you run this you see a window divided in half. If you click on the left, nothing much happens (except "num=0" prints on stdout). If you select the other tab on the left, though, the panels on the right turn blue. Select the first tab on the left, the right turns pink. So this small-scale version works: when one of the two tabs on the left receive a focusInEvent, they call the main window's i_got_focus() method, which switches all two of the tabs on the right.

There are some oddities that you can discover if you experiment (put the focus on the right then click on a tab, not a window, on the left) but I think I see how to fix them. What I want to talk about in this post is a bug that had me puzzled for an hour.

Look at the initialization of the mypanel class where it says "see discussion". Originally it read:

        super().__init__(text,parent)

That's automatic Qt boilerplate that I wrote without even thinking. Change the above program to read that way, and run it. What do you see?

In the upper left corner of the window, overlapping the corner of the splitter, is a small blue rectangle with the number 7. What?!?

Real Qt mavens know immediately what has happened, but it took me a bit of head-scratching. In the main window initialization, when each of eight instances of mypane are created, the main window passes self. The panel widgets need that reference if they are to call the main window's i_got_focus method. But I was also passing it as the "parent" argument to the QLabel initializer. So each of the eight Qlabels was a child of the main window.

Being a parent of a widget means, among other things, that you display that widget. Four of the eight widgets are added to the tabsets, and that re-parents them to their tab widget. But pink 3 and blues 5, 6 and 7 remain parented by the main window. It displays them in the default location, upper left. Blue 7 is created last so it goes in front, hiding the others. Passing no-parent to QLabel fixed the problem.

So just as in life, things go better if you choose the right parents.

Tuesday, May 6, 2014

Signal from a module

So the fonts module knows all about QFonts. Its main purpose is to keep track of the user's preference for the mono font (used in the editor) and the general font (used by all the UI widgets), getting these from the app settings at startup and recording them in settings at shutdown, and supporting the yet-to-be written preferences dialog. It offers services like

  • get_fixed() to return a QFont set to the user's preferred monospaced family and size,
  • scale(updown,qfont) to return a qfont scaled up or down one point,
  • ask_font(mono=True) to present a QFontDialog with an appropriate title and return a selected QFont.

When the user chooses a different mono font, any editview needs to know, because it has to explicitly set the font of its QPlainTextEdit. There could be one editview, or several, or even none. Similarly the main window needs to know if the user wants to use a different font for the UI (unlikely, but why not?). This is just the right place to use a Qt signal. Any editview, or the main window can connect itself to that signal while initializing.

However this turned out to be tricky because fonts is a module, not a class. Thereby hangs a short boring story...

Modul-ism

When I was writing PPQT V.1 I didn't really understand Python namespaces. I was stuck in the mindset of assembly language and C—hey, I'm old, what do you want?—so I vaguely equated Python import with a C #include. They are sort of alike in that both bring some externally-defined code into the scope of a source module. But as I bet you know, they are very, very different in implementation.

Take a stupid example. Let's say you have the following modules:

common.py:
    GLOB = 1
unit_a.py:
    import common
    common.GLOB = 2
unit_b.py:
    import common
    print(common.GLOB)
main.py:
    import unit_a
    import unit_b

When you run main.py, what happens? The first mental hurdle to get over is to realize that import is an executable statement: any code in the imported thing is executed. Normally that's declarative code like def or class but the assignments and the print() in these modules will also execute. So something is going to be printed: what?

If import was like #include, it would print 1 because the common imported into unit_b would be an independent copy of the file common.py. But that's not what happens. The first time import common is executed—as part of executing import unit_a—Python creates a common namespace with the one entry, GLOB bound to a 1, which unit_a then reassigns to 2. The next time import common is executed—as part of executing import unit_b—all that happens is that unit_b's namespace gets a reference to the same common namespace, from which it prints 2.

Although I understood this in a theoretical way, I couldn't quite shake the suspicion that importing the same module more than once was somehow a risky proposition. So I took pains to create a class to hold global values. I instantiated one of that class early, and passed that object into each sub-module. It was over-complicated, unnecessary, and in fact a bad design because the global-holding object ended up an overstuffed portmanteau full of unrelated things.

So, V.2, we do things pythonically. As I said, anything font-related gets handled in the fonts module, which has a number of global values with names like _MONO_FAMILY. These get set when the main window calls fonts.initialize(settings), they may get reset when the yet-to-be-written preferences calls fonts.set_general() or fonts.set_fixed(), and so on. And any module that imports fonts will be using the one and only fonts namespace and the same global values.

Signalling

Fine, but what about that signal? Say that preferences calls fonts.set_fixed() with a new choice of QFont. The fontsChanged signal needs to be emitted. But how, or from what?

The new PyQt5 signal/slot API insists that a signal has to be bound to a QObject instance. Fonts is a module, and it has no need define a class or make an object. But it wants to emit a signal. So this is what I had to do:

class Signaller(QObject):
    fontChange = pyqtSignal(bool)
    def connect(self, slot):
        self.fontChange.connect(slot)
    def send(self,boola):
        self.fontChange.emit(boola)

_SIGNALLER = Signaller()

def notify_me(slot):
    _SIGNALLER.connect(slot)
def _emit_signal(boola):
    _SIGNALLER.send(boola)

Signaller is a QObject with just one attribute, a class variable fontChange that is a signal. The signal carries one parameter, a boolean. (It's True if the font changed was the mono font, False if it was the UI font.)

Signaller has two methods, one to connect its signal to a slot, and one to emit the signal. One Signaller object is created and saved in a global reference.

Now, a call to fonts.notify_me() can be used to hook up any Python executable to the fontChange signal. Within the fonts module, a function like fonts.set_fixed() can call _emit_signal(True) to send the signal.

This works fine; the unit-test driver hooked up a function, called fonts.set_fixed(), and its function was invoked.

Monday, May 5, 2014

Being Resourceful

According to blogger, after each new post here, there's a bit of a spike in views. So at least a dozen people out there have this blog in their RSS readers. Howdy! I'll try to keep it going.

Another Unknown is Known

One of the unknowns that's been niggling at the back of my mind for weeks, is how to use the Qt resource system. I have a font that I want to carry along in the app so I can be sure it will be available. I think there will be some custom icons as well, like this one: which I think may be the icon for a button in the Find dialog that establishes a limited range of text for find/replace.

Anyway all such things shouldn't be carried along as separate files, but should be incorporated right into the program as resources. I knew Qt had a scheme for this and PyQt supported it; and I knew I would need to use it; but I didn't know how it worked and hadn't stirred myself to find out. So today I did.

As usual with these things, once you lay it out clearly it's no big deal. In a nutshell,

  • You list your resource files using XML syntax, in a file with type .rcc
  • You run a utility pyrcc5 which writes a python module.
  • You import that python module.

Then at run time, any file you named in the .rcc is available using the path prefix :/, as in :/hand-gripping.png.

The .rcc file consists of the following lines:

<!DOCTYPE RCC><RCC version="1.0">
<qresource>
   a list of files to embed, one line per file
</qresource>
</RCC>

Each file to embed is described with a line like this:

<file alias='run-time-name-of-resource'>relative-path-to-file</file>

For example, <file alias='hand-gripping.png'>hand-gripping.png</file>

And that's about it. I set up a folder Resources at the same level as my source modules. In it I put the .rcc file and the various files it named. Then from the command line, at the module level, I gave the command

pyrcc5 -o resources.py Resources/resources.rcc

And magically a 1.5MB file resources.py appeared. That file starts out with:

qt_resource_data = b"\
\x00\x04\xc8\x40\
\x00\
\x01\x00\x00\x00\x12\x01\x00\x00\x04\x00\x20\x46\x46\x54\x4d\x61\
\x13\x1d\x0b\x00\x04\x91\x38\x00\x00\x00\x1c\x47\x44\x45\x46\x79\

And ends, 20,000 lines later, with

def qInitResources():
    QtCore.qRegisterResourceData(0x01, qt_resource_struct, qt_resource_name, qt_resource_data)

qInitResources()

And that, executed upon the first import of the module, presumably tells the QApplication about these resources. It's pretty straightforward. Now I can get on with finalizing initialization of the font resources.

Saturday, May 3, 2014

Remembering

So I've moved on to a big, amorphous chunk of code, the "main window" of the app. This is what creates the main window, has all the menu "actions" like Save-As and Open, and generally sets things up and tears things down. It's a lot of code but I've started by nibbling at the edges, with initializing and shutting down. This involves using QSettings.

A QSettings object provides a way to tuck items into a persistent key-value store and get them back later. One common use is to remember the program's screen geometry. The final steps in the main window's termination code are:

        self.settings.setValue("mainwindow/size",self.size())
        self.settings.setValue("mainwindow/position",self.pos())
        self.settings.setValue("mainwindow/splitter",self.splitter.saveState())

The size, screen position, and the relative position of the "splitter" that divides the window into two sections are stored away. When the program starts up, the final steps in the main window's GUI initialization are:

        self.resize(self.settings.value("mainwindow/size", QSize(400, 400)))
        self.move(self.settings.value("mainwindow/position", QPoint(100,100)))
        self.splitter.restoreState(
            self.settings.value("mainwindow/splitter",QByteArray()))

The call to self.resize() sets the window size to whatever was stored in the settings, or, if this is the very first time the app's been launched on this system, to a default value of 400x400.

Version 1 stored a number of things in the settings, but the list of things stored will change a lot for V.2. This is partly because V.2 has more global items to remember: it will support user preferences for colors and fonts and paths to this and that, all of which are best remembered in settings. But values saved in QSettings are by definition global to the application. In V.1, lots of things were treated as global that were actually unique to the current document. That was because there was only one document. In V.2 we support multiple open Books (documents) and that, as I have said ruefully a number times lately, changes everything.

For just one example, the Find panel keeps lists of the last ten search-strings and last ten replace-strings. In V.1 it stored those lists in the settings at shutdown. But those lists shouldn't be global; they are document-unique and will change each time the user brings a different Book into focus. So those will get saved and restored from the each Book's meta-data, not from the global settings.

On the other hand, in V.2 I want to do what Firefox does when it starts up: offer to restore the previous session. I do not want to do what a number of Mac OS apps do since the Mountain Lion release, and restore the last session automatically. Maybe you don't want to re-open all those documents. But I want to offer the option, and that means, at shutdown, recording the list of open documents and their paths in the settings; and recovering the list at startup.

QVariant vanishes

As documented for C++ (link above) the QSettings API makes heavy use of the QVariant class, a way to wrap any other class as something that can be stored as a byte-string. The first argument to both value() and setValue() are the character-string keys, but the second in both cases is supposed to be a QVariant. The value being stored from self.size() is a QSize; the value returned by self.pos() is a QPoint, and the splitter status from self.splitter.saveState() is a 23-element QByteArray. C++ doesn't permit passing such a potpourri of types in the same argument position, so Qt tells you to swaddle them decently as QVariants.

Similarly, the value returned by QSettings.value() is documented as being a QVariant; and you are supposed to use one of that QVariant's many "toXxxx" methods to restore it to whatever class was wrapped in it. For example, in V1 of PPQT, the code to restore the main window splitter's state looked like:

 self.hSplitter.restoreState(
              self.settings.value("main/splitter").toByteArray() )

The call to value() evaluates to a QVariant; that object's toByteArray() method reveals that the value is really a QByteArray, and that is accepted by the splitter's restoreState method.

Well, in PyQt5, Maestro Phil and company did away entirely with QVariant calls. Presumably they are still using them under the covers, in the interface to C++, but the Python programmer is not supposed to use them, and in fact there is no way to import the name QVariant from a PyQt5 module. This simplifies the code of a PyQt5 app, but it adds yet another consideration when moving code from PyQt4.

Where does it all go?

Where does the QSettings object actually put the keys and values you give it? In different places depending on the platform. It's documented under "Platform Considerations" in the QSettings page, but here's the bottom line:

Linux
$HOME/.config/orgname/appname.conf
Mac OSX
$HOME/Library/Preferences/com.orgname.appname.plist
Windows (registry)
HKEY_CURRENT_USER\Software\orgname\appname

In each case, orgname and appname are defined at startup to the QApplication, in my case with code like this:

app.setOrganizationName("PGDP")
app.setApplicationName("PPQT2")

Eventually that will be in the top-level module, PPQT2. Since that doesn't exist, it's currently in the unit-test driver, mainwindow_test.py, which prepares an empty settings object, creates the mainwindow object, sends it a close event, and then looks in the settings to verify it stored what it should have.

Actually, QSettings has a multi-level scheme; it will look first for user-level values; then at a system level if necessary. I'm not worrying about that; it's all automatic and platform-independent at the PyQt level. Which is nice.