Me 'n Sikuli are good; Recursive bug

So it turned out that Sikuli actually had a feature that made all my laborious capture code yesterday unnecessary. Completely so. And on top of that the feature is discussed in their top-level "hello world" tutorial. Maybe I'm losing it. Anyway you can capture parts of another app very nicely, interactively. Which I started doing today and got about a third of the way through a mainwindow test before I got distracted by a couple of bugs that turned up. Which is what testing is for, ok.

One bug was quite interesting although hard to describe. The symptom was that, when two books were open and one opened a third, suddenly the second and third books were sharing the same set of "panels". Same image display, primarily, as that's the only functioning panel, although the other placeholder panels were also swapped. So after opening book 3, it and book 2 had the identical image displays, even though they should show different images. Verrrry peculiar.

Well, long story short, after about a half hour of debugging I realized what had to be happening was that the "focus me" function, which is called when any book's edit window gets the focus in order to display that book's other panels, was being called recursively! It would be half-way through focusing the newly opened book when it got called again to focus the newly opened book. A little more tracing and I figured out why that happened: when the new book's edit window got added to its tabset, it received a focus-in event from Qt, so it called the focus-me routine which was still unfinished. Hilarity ensued.

That was easy to fix, once figured out. But another is still pending. This is another issue of getting the Qt focusIn event. As just recounted, one of those events is delivered to an edit window the instant it is added to a tab set. And later on, when you click on the tab for a book's edit window, it should get another focus-in event. And it does—but only if it has had at least one real user mouse-click.

I put a print in the focus-event handler to prove it. Here's the sequence: open three books. Each one gets its edit panel added to the edit tabset. And each one at that time gets a focus-in event.

Now, click back and forth on the three tabs. Each edit panel is revealed appropriately. But they do not get a focus-in event. Click all the tabs you want; they are displayed but they don't get the event, and therefor they don't call focus-me and their image and other panels don't get displayed.

Now click the mouse in one of the panels. It gets a focus-in event and from then on it also gets a focus-in event every time its tab comes up. No manual click: no events. One manual click: events.

I've tried variations on calls to QWidget.setFocus() during the creation of the edit view, but it doesn't have any effect. Very mystifying.

GUI Testing with Sikuli

Sikuli is an open-source tool for automating tests of GUI applications. It is platform-independent (has Mac OS, Linux, and Windows versions) and is not linked in any way to a GUI package such as Qt. It uses little screen-grabs to locate targets of operation. Here, for example, are four lines from a Sikuli test script:

Line 28 finds the image of my editview context menu somewhere on the screen. That verifies that the menu is active. The next line sends the mouse to click on the third line of the menu. That opens a Mac OS file selection dialog, and I have arranged that the current working directory at this time will include the file I want. So line 30 of the script finds and clicks-on that filename, and the next line finds and clicks on the Open button of the file dialog.

This exactly answers the problems I was listing in the immediately prior blog post. You can specify mouse actions in terms of the look of their targets, regardless of whether these are Qt widgets or operating system dialogs. You can make it "find" the exact image of what you expect to have on the screen; for example, to find the image of a checkable menu item with its check mark in place, or to find the image of a word with or without a spelling-error underline.

A Sikuli script is a Python 2.7 script, but that's not a problem even though my code uses Python 3. The Sikuli app is a Java app which embeds its own Jython 2.7! So Sikuli's Python is completely unconnected from the target app's Python.

The test setup needed to combine Sikuli tests and ordinary Python test scripts is a bit complex. Here's the folder setup:

ppqt
   all app source scripts
   Tests
      all unit-test scripts run by pytest
      Sikuli
         all Sikuli test scripts

The unit-test driver, Tests/editview_sikuli.py reads like this:

path_to_self = os.path.realpath(__file__)
path_to_Sikuli = os.path.join(os.path.dirname(path_to_self),'Sikuli')
import subprocess
r = subprocess.call(['/Applications/SikuliX-IDE.app/Contents/runIDE',
                 '-r',
                 os.path.join(path_to_Sikuli,'editview.sikuli')])
assert not(r)

So this simple script just fires off the Sikuli app asking it to run Sikuli/editview.sikuli. That is actually a folder containing a python script and all the little .png files that represent target images.

The Sikuli script (which is in Python 2.7 syntax, remember) starts out by firing off a copy of the test GUI:

import subprocess
subprocess.Popen(['/usr/local/bin/python',
            '/Users/dcortes1/Dropbox/David/PPQT/V2/ppqt/Tests/Sikuli/editview_runner.py'])

The new subprocess based on editview_runner.py sets up and launches the editview test window. That's the window where Sikuli will find visual matches as it runs its "click" and "find" statements. It does that, and if everything matches, the script ends with "type('q',KeyModifier.META)", in other words, command-Quit. That terminates the test app. The Sikuli script ends with a 0 return code. Back in the test driver, the subprocess call ends and assert not(r) is satisfied.

The Sikuli interactive IDE is a bit clumsy to use, and preparing a useful test is a bit tedious. (But on the other hand, it also revealed a couple of small bugs.) I expect to get good use out it the rest of the way.

As of now, the editview module is functionally complete. As are several auxiliary modules: colors (keeps track of default colors and highlights, eventually will link up with a Preferences dialog); dictionaries (keeps track of available spelling dictionaries and defines the Speller class of spellcheck objects); and the basic skeleton of the Book class. So: on to imageview! When that's working, it will be time to set up the main window and actually have a whole app to play with.