[rdo-list] RDO CI hardware requirements for CentOS altarch

Fri Aug 12 20:30:50 UTC 2016

note that a key element to the messaging from our side is that the
content is tested the way users use the code, not the way developers
deliver the code - so ideally, we want to get down to baremetal
deployments and validating distro v/s rdo and rdo v/s distro.

we typically see hundreds of bare metal deployments per day for rdo /
cloud SIG, and at the very least we should make an effort to sync
across the arch's if we can.

so, as we work through what needs doing, lets make sure we factor in
the user story we want to deliver at the other end.

Regards

On 11/08/16 09:05, Xiandong Meng wrote:
> OK, so the bottleneck is in "INSTALL / TEST (IMPORT IMAGES)" stage
> with 2 Triple-O scenario, 3 Packstack and 3 OpenStack-Puppet test
> jobs. So we may need up to (4 core, 8 GB memory)*3 to cover
> PackStack and OpenStack-Puppet path as each Triple-O scenario will
> need about 90 minutes. (So the shortest time is limited to 90
> minutes assuming we maximize the parallelism level. )
> 
> Now i need some input about the resource requirements for Triple-O
> test scenarios.
> 
> 
> 
> Regards,
> 
> Alex Meng<mailto:mengxiandong at gmail.com> mengxiandong at gmail.com
> <mailto:mengxiandong at gmail.com>
> 
> On Thu, Aug 11, 2016 at 4:11 PM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>> wrote:
> 
> Right now the jobs are sequenced as outlined in the pipeline [1].
> 
> First, the job "rdo-promote-get-hash-master" will run. If it is 
> successful, it triggers the next "stage" of the pipeline. Then, the
> job "tripleo-quickstart-promote-master-delorean-build-images" will
> run. If it is successfull, it triggers the next "stage" of the 
> pipeline. After that you have 8 jobs that, if capacity allows
> (usually the case), will all run simultaneously. Otherwise, builds
> can be queued until a slave can process them. ... and so on.
> 
> We can't realistically run all these jobs in a sequence. 
> Considering they complete in 45 minutes (or more), we'd be looking
> at a pipeline of over 6 hours of builds.
> 
> [1]: 
> https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/
>
> 
<https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/>
> 
> David Moreau Simard Senior Software Engineer | Openstack RDO
> 
> dmsimard = [irc, github, twitter]
> 
> 
> On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng 
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>> wrote:
>> OK, i mean on the same time, how many concurrent CI jobs may be
> running? For
>> example, at peak load, will
> weirdo-master-promote-packstack-scenario001 ,
>> weirdo-master-promote-packstack-scenario002 , 
>> weirdo-master-promote-packstack-scenario003 run at the same
>> time?
> Or they
>> will run in sequence by design?
>> 
>> 
>> Regards,
>> 
>> Alex Meng mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>
>> 
>> On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>> wrote:
>>> 
>>> Alex,
>>> 
>>> Can you expand on what you mean by that ? The concurrency (or
>>> lack thereof) of the jobs are more about the design of the job
>>> itself -- or the environment it is run from as well as the
>>> environment it is run on.
>>> 
>>> The jobs part of the promotion pipeline [1] run a couple times
> per day.
>>> 
>>> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
> <https://ci.centos.org/view/rdo/view/promotion-pipeline/>
>>> 
>>> David Moreau Simard Senior Software Engineer | Openstack RDO
>>> 
>>> dmsimard = [irc, github, twitter]
>>> 
>>> 
>>> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>>
>>> wrote:
>>>> David, thank you for your response.
>>>> 
>>>> For Packstack and Puppet-OpenStack jobs, I noticed that
>>>> usually
> each job
>>>> takes no more than 45 minutes. How many jobs may run in
> parallel and how
>>>> often they are triggered?
>>>> 
>>>> Regards,
>>>> 
>>>> Alex Meng mengxiandong at gmail.com
>>>> <mailto:mengxiandong at gmail.com>
>>>> 
>>>> On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>>
>>>> wrote:
>>>>> 
>>>>> Hi Alex,
>>>>> 
>>>>> I don't know the specifics of resource usage for
>>>>> alternative architectures but I can tell about x86_64.
>>>>> 
>>>>> Packstack and Puppet-OpenStack jobs are designed to run
>>>>> within
> 8GB of
>>>>> RAM - either on a single virtual machine or on a single
>>>>> bare metal server. I would say 4 cores is the minimum (or
>>>>> otherwise job length is severely affected), 8 is best. Disk
>>>>> space is not generally a concern, easily fitting within
> 50GB of
>>>>> space.
>>>>> 
>>>>> I don't have the numbers for TripleO so I'll let someone
>>>>> else
> chime in
>>>>> on that.
>>>>> 
>>>>> David Moreau Simard Senior Software Engineer | Openstack
>>>>> RDO
>>>>> 
>>>>> dmsimard = [irc, github, twitter]
>>>>> 
>>>>> 
>>>>> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>>
>>>>> wrote:
>>>>>> We had discussed it a bit in previous RDO meeting on irc.
>>>>>> I
> want to
>>>>>> write a separate mail for more broad and in-depth
>>>>>> discussion here.
>>>>>> 
>>>>>> 
>>>>>> For a master release (for now it is Newton), i noticed
>>>>>> the
> promotion
>>>>>> pipes fall into three different categories: - Triple-O
>>>>>> based CI test - Packstack based test - OpenStack-Puppet
>>>>>> based test
>>>>>> 
>>>>>> So what is the base minimal CI requirements to start
>>>>>> with
> for AltArch
>>>>>> support? Since many of the CI test should work even with
>>>>>> VMs
> instead
>>>>>> of physical nodes, can we start with 1-2 physical
>>>>>> servers?
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Alex Meng mengxiandong at gmail.com
>>>>>> <mailto:mengxiandong at gmail.com>
>>>>>> 
>>>>>> _______________________________________________ rdo-list
>>>>>> mailing list rdo-list at redhat.com
>>>>>> <mailto:rdo-list at redhat.com> 
>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
> <https://www.redhat.com/mailman/listinfo/rdo-list>
>>>>>> 
>>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>>>> 
>>>> 
>> 
>> 
> 
> 

-- 
Karanbir Singh, Project Lead, The CentOS Project
+44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS
GnuPG Key : http://www.karan.org/publickey.asc

-- 
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
GnuPG Key : http://www.karan.org/publickey.asc