[rdo-list] Automation and Identifiers

[Udi Kalifon]

Sorry, this was sent to the wrong mailing list. Please ignore :)


Regards,
Udi Kalifon; Senior QE; RHOS-UI Automation


On Sun, Jun 25, 2017 at 8:16 PM, Udi Kalifon <ukalifon at redhat.com> wrote:

> Hello.
>
> We are currently updating the automation infrastructure to support the
> changes in the GUI in OSP12. The work to support the new deployment
> counters (which replace the assign nodes) is finished, and the validators
> didn't require any work to be supported but we still refactored the
> infrastructure to work much faster (more on that later). The rest of the
> infrastructure will be updated hopefully in the next few days, when some
> bug fixes land in upstream and we can play with the features.
>
> We also need your help. In many places in the automation, we are finding
> the elements we need without having any identifiers (no ids or any unique
> classes or other attributes) at all. We find these elements by exhaustively
> searching the DOM, starting from a parent element that we can identify, and
> searching its children and their children etc... We also have to rely on
> the text in the elements in some places, which will be a blocker for us if
> we ever want to support i18n.
>
> This way of working is not only slow and takes a lot of effort from us,
> but can also fail the tests sometimes. For example, when an element gets
> unmapped from the DOM (when it's updated, or when a dialog closes, or when
> we navigate to another page), we need to re-search it. We don't always know
> when an element becomes "stale" like that because it's not always something
> that we initiated, and it still surprises us here and there.
>
> The problem of stale elements would not be a problem for us if we had an
> identifier for all the elements - because our infrastructure is already
> pretty smart and handles the StaleElementException for us by automatically
> and transparently searching for the element again, using the same
> identifier that was used the first time. This means that, for elements with
> identifiers, an event like that can happen at any point in the middle of
> any script and the test developer doesn't need to bother with it. But when
> an element doesn't have an identifier this magic can't work, and we crash.
> With elements that have no identifiers we need to be able to predict when
> they'll become stale, and it's always ugly and heuristic.
>
> Another problem with lack of identifiers is that sooner or later we'll
> just get stuck and won't be able to interact with certain elements in the
> GUI, because we simply won't be able to tell some of the elements from each
> other. Some of the new features which I haven't started automating yet have
> a very complex GUI and it will probably present a challenge.
>
> In the meantime, on a more optimistic note, I was able to get much better
> at identifying hard-to-identify elements. These identifiers are still
> fragile and they rely on the structure of the DOM - but they eliminate the
> need for an exhaustive search and the infrastructure can handle these
> elements if they become stale. I refactored some of our code with this
> approach and even got a significant performance increase with the
> validators infrastructure, which was so slow that it sometimes missed if a
> validator ran or not.
>
> I will update the Automation Identifiers document in the next few days.
> I'll delete all the irrelevant sections of it (which are probably 99% of
> the document) and start listing which identifiers we'd ask to have where.
> We should also make it a rule that in the future, any new feature that is
> delivered in the GUI, will have identifiers for the automation built-in
> from the start.
>
> Sorry for the long email, I wanted to report on what's going on in the
> automation front in addition to bringing up the request for identifiers
> again. Thanks a lot for listening.
>
> Regards,
> Udi Kalifon; Senior QE; RHOS-UI Automation
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20170626/22ff8e08/attachment.html>