[rdo-list] Automation and Identifiers

[Udi Kalifon]

Hello.

We are currently updating the automation infrastructure to support the
changes in the GUI in OSP12. The work to support the new deployment
counters (which replace the assign nodes) is finished, and the validators
didn't require any work to be supported but we still refactored the
infrastructure to work much faster (more on that later). The rest of the
infrastructure will be updated hopefully in the next few days, when some
bug fixes land in upstream and we can play with the features.

We also need your help. In many places in the automation, we are finding
the elements we need without having any identifiers (no ids or any unique
classes or other attributes) at all. We find these elements by exhaustively
searching the DOM, starting from a parent element that we can identify, and
searching its children and their children etc... We also have to rely on
the text in the elements in some places, which will be a blocker for us if
we ever want to support i18n.

This way of working is not only slow and takes a lot of effort from us, but
can also fail the tests sometimes. For example, when an element gets
unmapped from the DOM (when it's updated, or when a dialog closes, or when
we navigate to another page), we need to re-search it. We don't always know
when an element becomes "stale" like that because it's not always something
that we initiated, and it still surprises us here and there.

The problem of stale elements would not be a problem for us if we had an
identifier for all the elements - because our infrastructure is already
pretty smart and handles the StaleElementException for us by automatically
and transparently searching for the element again, using the same
identifier that was used the first time. This means that, for elements with
identifiers, an event like that can happen at any point in the middle of
any script and the test developer doesn't need to bother with it. But when
an element doesn't have an identifier this magic can't work, and we crash.
With elements that have no identifiers we need to be able to predict when
they'll become stale, and it's always ugly and heuristic.

Another problem with lack of identifiers is that sooner or later we'll just
get stuck and won't be able to interact with certain elements in the GUI,
because we simply won't be able to tell some of the elements from each
other. Some of the new features which I haven't started automating yet have
a very complex GUI and it will probably present a challenge.

In the meantime, on a more optimistic note, I was able to get much better
at identifying hard-to-identify elements. These identifiers are still
fragile and they rely on the structure of the DOM - but they eliminate the
need for an exhaustive search and the infrastructure can handle these
elements if they become stale. I refactored some of our code with this
approach and even got a significant performance increase with the
validators infrastructure, which was so slow that it sometimes missed if a
validator ran or not.

I will update the Automation Identifiers document in the next few days.
I'll delete all the irrelevant sections of it (which are probably 99% of
the document) and start listing which identifiers we'd ask to have where.
We should also make it a rule that in the future, any new feature that is
delivered in the GUI, will have identifiers for the automation built-in
from the start.

Sorry for the long email, I wanted to report on what's going on in the
automation front in addition to bringing up the request for identifiers
again. Thanks a lot for listening.

Regards,
Udi Kalifon; Senior QE; RHOS-UI Automation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20170625/f69c0847/attachment.html>