[rdo-list] Improving the test day experience

[Ilja Maslov]

Hi,

I'd like to see realistic production baremetal scenarios tested, i.e. with network isolation, Ceph storage, firstboot hooks that set correct tuned profile.

Thanks,
Ilja

-----Original Message-----
From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of Rich Bowen
Sent: Tuesday, September 20, 2016 11:52 AM
To: Petr Kovar <pkovar at redhat.com>
Cc: rdo-list at redhat.com
Subject: Re: [rdo-list] Improving the test day experience



On 09/20/2016 11:11 AM, Petr Kovar wrote:
> On Thu, 15 Sep 2016 15:59:22 -0400
> Rich Bowen <rbowen at redhat.com> wrote:
> 
>> In our RDO Community Meeting (on IRC) yesterday, the topic of 
>> improving test day was brought up.
>>
>> Test day is, for the most part, about getting people to beat on the 
>> latest packages and find problems. However, since most of these 
>> problems are being found in CI, it's time to make test day also about 
>> giving people an opportunity to see what's coming in future releases. 
>> To that end, user experience is pretty important on test day.
>>
>> Several suggestions were made to improve the test day experience for 
>> folks that aren't already experts:
>>
>>
>> * Test setup matrix cleanup? Remove tests that are no longer relevant.
>>
>> * More detailed, beginner-friendly instructions for testing. (Test 
>> day as product preview for potential users, or existing users who are 
>> not deep experts.)
> 
> I can start working on this but would probably need more input wrt the 
> required scope of the instructions. Like, say, besides setting up a VM 
> with CentOS/RHEL for running packstack, are there other steps we want 
> to cover as part of preparing your environment?


On the test day instructions page -
https://www.rdoproject.org/testday/newton/testedsetups_rc - each test has a HOWTO link, and almost all of these link to the Quickstart or nothing at all. As a beginner, that's baffling. How are these separate tests if they all have exactly the same instructions? If I want to participate in testing, what am I actually supposed to do?

What I'd like to see, eventually, is a HowTo page (or script, or ...
something) for every one of these test scenarios, showing what steps need to be taken to exercise the specified functionality.

I presume that in many cases, there are in fact scripts that can be pointed at.

These instructions provide multiple purposes - they also show someone how to set up these scenarios for production, and explain what the difference scenarios are good for.


--
Rich Bowen - rbowen at redhat.com
RDO Community Liaison
http://rdoproject.org
@RDOCommunity

_______________________________________________
rdo-list mailing list
rdo-list at redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list

To unsubscribe: rdo-list-unsubscribe at redhat.com