[rdo-list] Improving the test day experience

[Rich Bowen]

On 09/20/2016 11:11 AM, Petr Kovar wrote:
> On Thu, 15 Sep 2016 15:59:22 -0400
> Rich Bowen <rbowen at redhat.com> wrote:
> 
>> In our RDO Community Meeting (on IRC) yesterday, the topic of improving
>> test day was brought up.
>>
>> Test day is, for the most part, about getting people to beat on the
>> latest packages and find problems. However, since most of these problems
>> are being found in CI, it's time to make test day also about giving
>> people an opportunity to see what's coming in future releases. To that
>> end, user experience is pretty important on test day.
>>
>> Several suggestions were made to improve the test day experience for
>> folks that aren't already experts:
>>
>>
>> * Test setup matrix cleanup? Remove tests that are no longer relevant.
>>
>> * More detailed, beginner-friendly instructions for testing. (Test day
>> as product preview for potential users, or existing users who are not
>> deep experts.)
> 
> I can start working on this but would probably need more input wrt the
> required scope of the instructions. Like, say, besides setting up a VM with
> CentOS/RHEL for running packstack, are there other steps we want to cover as
> part of preparing your environment?


On the test day instructions page -
https://www.rdoproject.org/testday/newton/testedsetups_rc - each test
has a HOWTO link, and almost all of these link to the Quickstart or
nothing at all. As a beginner, that's baffling. How are these separate
tests if they all have exactly the same instructions? If I want to
participate in testing, what am I actually supposed to do?

What I'd like to see, eventually, is a HowTo page (or script, or ...
something) for every one of these test scenarios, showing what steps
need to be taken to exercise the specified functionality.

I presume that in many cases, there are in fact scripts that can be
pointed at.

These instructions provide multiple purposes - they also show someone
how to set up these scenarios for production, and explain what the
difference scenarios are good for.


-- 
Rich Bowen - rbowen at redhat.com
RDO Community Liaison
http://rdoproject.org
@RDOCommunity