These are (obvious) examples, the point is to come up with a detailed list of release criteria.

2016-01-12 15:34 GMT+01:00 Mohammed Arafa <mohammed.arafa(a)gmail.com&gt;: Yes, through improving our test days scenarios.

2016-01-12 15:52 GMT+01:00 Rich Bowen <rbowen(a)redhat.com&gt;: Yes, this also *must* be part of our Definition of Done (DoD). Originally, this was discussed with John in order to have better coordination w/ RDO Manager releases, but this should not be limited to technical artefacts. Marketing, documentation (including upstream's) should be part of it.

I am resurrecting this thread, because we did not get any feedback the first time around, and it is coming up on the time we will need to define our official mitaka release as done. I have been thinking about the technical criteria for TripleO (RIP RDO-Manager), and I think my previous proposal is not CI'able with a 32G RAM virt setup. I think we need to have our technical definition of done aligned with what we can actually CI. A collection of my thoughts on this topic: * A 5 node virt setup barely fits on a 32G RAM host now, and this is the minimum for HA (undercloud + 3x control + 1x compute). * There are not enough resources left over in the above scenario to have reliable tempest results, since we run tempest from the undercloud. * Ceph is included in the original proposal, but I don't think we should be including Ceph in or DoD for RDO until the packages are available from a CentOS SIG. (We currently pull them from EPEL). * We also need some post-deploy validation specific to Ceph, since it is not covered by tempest except in an indirect way. One possible solution for mitaka that could be stretched for future releases would be: * Deploy 3 node virt setup (undercloud + control + compute) using the pacemaker environment, and run some specific list of tempest tests that optimizes component coverage with minimal resource usage. * Deploy 5 node virt setup (undercloud + 3x control + compute), and run the pingtest from tripleo.sh. By using the pacemaker environment for the 3 node setup, we would at least be testing that the pacemaker code path works. We would potentially miss some issues where steps should only get performed on one of the three control nodes, so we also do a real HA setup with only basic validation. In any case, we need to define something as the mitaka release is rapidly approaching. I will add this to the meeting agenda for this week. On 01/19/2016 06:49 AM, John Trowbridge wrote: