Re: [rdo-list] Improving the test day experience
On 09/20/2016 11:11 AM, Petr Kovar wrote:
On Thu, 15 Sep 2016 15:59:22 -0400
Rich Bowen <rbowen(a)redhat.com> wrote:
> In our RDO Community Meeting (on IRC) yesterday, the topic of improving
> test day was brought up.
>
> Test day is, for the most part, about getting people to beat on the
> latest packages and find problems. However, since most of these problems
> are being found in CI, it's time to make test day also about giving
> people an opportunity to see what's coming in future releases. To that
> end, user experience is pretty important on test day.
>
> Several suggestions were made to improve the test day experience for
> folks that aren't already experts:
>
>
> * Test setup matrix cleanup? Remove tests that are no longer relevant.
>
> * More detailed, beginner-friendly instructions for testing. (Test day
> as product preview for potential users, or existing users who are not
> deep experts.)
I can start working on this but would probably need more input wrt the
required scope of the instructions. Like, say, besides setting up a VM with
CentOS/RHEL for running packstack, are there other steps we want to cover as
part of preparing your environment?
On the test day instructions page -
https://www.rdoproject.org/testday/newton/testedsetups_rc - each test
has a HOWTO link, and almost all of these link to the Quickstart or
nothing at all. As a beginner, that's baffling. How are these separate
tests if they all have exactly the same instructions? If I want to
participate in testing, what am I actually supposed to do?
What I'd like to see, eventually, is a HowTo page (or script, or ...
something) for every one of these test scenarios, showing what steps
need to be taken to exercise the specified functionality.
I presume that in many cases, there are in fact scripts that can be
pointed at.
These instructions provide multiple purposes - they also show someone
how to set up these scenarios for production, and explain what the
difference scenarios are good for.
--
Rich Bowen - rbowen(a)redhat.com
RDO Community Liaison
http://rdoproject.org
@RDOCommunity