maildev@lists.thunderbird.net

Thunderbird email developers

View all threads

Adding dogtail tests?

ST
Samuel Thibault
Thu, Feb 21, 2019 5:00 PM

Hello,

AFAIK, Thunderbird doesn't currently include a testsuite.

For accessibility testing, we would like to introduce tests to check the
behavior at the user interface itself, by using dogtail (developped by
Fedora, based on at-spi) to simulate keypresses, mouse clicks, and check
the results.

The idea is that at Hypra, we have a well-established documentation
of what keyboard shortcut sequences blind users would employ to use
thunderbird, and which we should thus make sure always work. The key
sequences might have to be changed if that's for a good reason, but at
least we would know about it and update the documentation accordingly.

I have put a prototype of what we are thinking of on
https://git.hypra.fr/hypra/regression-testing/tree/master

An example of what it would look like is:

pressCombo('<control><shift>k')
acc = app.child('Filter these messages <Ctrl+Shift+K>', 'entry')
waitFocus(acc)

pressCombo('<alt>f')
acc = app.child('New', 'menu')
waitFocus(acc)

etc.

which here allows to check that the control-shift-k shortcut properly
reaches the filter entry, that alt-f opens the file menu, etc.

I have written a "log" script that can generate such testcase code:
a user can run thunderbird and perform a usage scenario, and "log"
will get the interesting events from at-spi (keypresses, focus change,
checkbox status change, etc.) and emit the python code which simulates
keypresses and waits for the results. That way we can easily produce
testcases from our documentation, by just running the use cases
described in it, which are representative of what a blind user needs to
be able to do.

Dogtail depends on atspi, but also on python-cairo, python-gi,
gir1.2-gtk-3.0, I don't know if these could be included at least in the
Linux build environments, so such testsuite could be done regularly?
We (Hypra) could then feed thunderbird with such tests, from our
documentation.  We can probably produce dozens of them, which can
probably be tested within something like a few minutes.

Samuel

Hello, AFAIK, Thunderbird doesn't currently include a testsuite. For accessibility testing, we would like to introduce tests to check the behavior at the user interface itself, by using dogtail (developped by Fedora, based on at-spi) to simulate keypresses, mouse clicks, and check the results. The idea is that at Hypra, we have a well-established documentation of what keyboard shortcut sequences blind users would employ to use thunderbird, and which we should thus make sure always work. The key sequences might have to be changed if that's for a good reason, but at least we would know about it and update the documentation accordingly. I have put a prototype of what we are thinking of on https://git.hypra.fr/hypra/regression-testing/tree/master An example of what it would look like is: pressCombo('<control><shift>k') acc = app.child('Filter these messages <Ctrl+Shift+K>', 'entry') waitFocus(acc) pressCombo('<alt>f') acc = app.child('New', 'menu') waitFocus(acc) etc. which here allows to check that the control-shift-k shortcut properly reaches the filter entry, that alt-f opens the file menu, etc. I have written a "log" script that can generate such testcase code: a user can run thunderbird and perform a usage scenario, and "log" will get the interesting events from at-spi (keypresses, focus change, checkbox status change, etc.) and emit the python code which simulates keypresses and waits for the results. That way we can easily produce testcases from our documentation, by just running the use cases described in it, which are representative of what a blind user needs to be able to do. Dogtail depends on atspi, but also on python-cairo, python-gi, gir1.2-gtk-3.0, I don't know if these could be included at least in the Linux build environments, so such testsuite could be done regularly? We (Hypra) could then feed thunderbird with such tests, from our documentation. We can probably produce dozens of them, which can probably be tested within something like a few minutes. Samuel