[rdo-list] RDO CI hardware requirements for CentOS altarch

Sat Aug 13 05:06:20 UTC 2016

________________________________
From: rdo-list-bounces at redhat.com <rdo-list-bounces at redhat.com> on behalf of Karanbir Singh <mail-lists at karan.org>
Sent: Friday, August 12, 2016 4:30 PM
To: Xiandong Meng; David Moreau Simard
Cc: rdo-list
Subject: Re: [rdo-list] RDO CI hardware requirements for CentOS altarch

note that a key element to the messaging from our side is that the
content is tested the way users use the code, not the way developers
deliver the code - so ideally, we want to get down to baremetal
deployments and validating distro v/s rdo and rdo v/s distro.

> we typically see hundreds of bare metal deployments per day for rdo /
> cloud SIG, and at the very least we should make an effort to sync
> across the arch's if we can.

In regards of RDO how much is percentage of TripleO bare metal
deployments  per Mitaka ( per Liberty ) ?
If this info is closed by some reasons , sorry for question been asked.

Thanks.
Boris

>so, as we work through what needs doing, lets make sure we factor in
>the user story we want to deliver at the other end.

Regards

On 11/08/16 09:05, Xiandong Meng wrote:
> OK, so the bottleneck is in "INSTALL / TEST (IMPORT IMAGES)" stage
> with 2 Triple-O scenario, 3 Packstack and 3 OpenStack-Puppet test
> jobs. So we may need up to (4 core, 8 GB memory)*3 to cover
> PackStack and OpenStack-Puppet path as each Triple-O scenario will
> need about 90 minutes. (So the shortest time is limited to 90
> minutes assuming we maximize the parallelism level. )
>
> Now i need some input about the resource requirements for Triple-O
> test scenarios.
>
>
>
> Regards,
>
> Alex Meng<mailto:mengxiandong at gmail.com> mengxiandong at gmail.com
> <mailto:mengxiandong at gmail.com>
>
> On Thu, Aug 11, 2016 at 4:11 PM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>> wrote:
>
> Right now the jobs are sequenced as outlined in the pipeline [1].
>
> First, the job "rdo-promote-get-hash-master" will run. If it is
> successful, it triggers the next "stage" of the pipeline. Then, the
> job "tripleo-quickstart-promote-master-delorean-build-images" will
> run. If it is successfull, it triggers the next "stage" of the
> pipeline. After that you have 8 jobs that, if capacity allows
> (usually the case), will all run simultaneously. Otherwise, builds
> can be queued until a slave can process them. ... and so on.
>
> We can't realistically run all these jobs in a sequence.
> Considering they complete in 45 minutes (or more), we'd be looking
> at a pipeline of over 6 hours of builds.
>
> [1]:
> https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/
>
>
<https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/>
>
> David Moreau Simard Senior Software Engineer | Openstack RDO
>
> dmsimard = [irc, github, twitter]
>
>
> On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>> wrote:
>> OK, i mean on the same time, how many concurrent CI jobs may be
> running? For
>> example, at peak load, will
> weirdo-master-promote-packstack-scenario001 ,
>> weirdo-master-promote-packstack-scenario002 ,
>> weirdo-master-promote-packstack-scenario003 run at the same
>> time?
> Or they
>> will run in sequence by design?
>>
>>
>> Regards,
>>
>> Alex Meng mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>
>>
>> On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>> wrote:
>>>
>>> Alex,
>>>
>>> Can you expand on what you mean by that ? The concurrency (or
>>> lack thereof) of the jobs are more about the design of the job
>>> itself -- or the environment it is run from as well as the
>>> environment it is run on.
>>>
>>> The jobs part of the promotion pipeline [1] run a couple times
> per day.
>>>
>>> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
> <https://ci.centos.org/view/rdo/view/promotion-pipeline/>
>>>
>>> David Moreau Simard Senior Software Engineer | Openstack RDO
>>>
>>> dmsimard = [irc, github, twitter]
>>>
>>>
>>> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>>
>>> wrote:
>>>> David, thank you for your response.
>>>>
>>>> For Packstack and Puppet-OpenStack jobs, I noticed that
>>>> usually
> each job
>>>> takes no more than 45 minutes. How many jobs may run in
> parallel and how
>>>> often they are triggered?
>>>>
>>>> Regards,
>>>>
>>>> Alex Meng mengxiandong at gmail.com
>>>> <mailto:mengxiandong at gmail.com>
>>>>
>>>> On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard
> <dms at redhat.com <mailto:dms at redhat.com>>
>>>> wrote:
>>>>>
>>>>> Hi Alex,
>>>>>
>>>>> I don't know the specifics of resource usage for
>>>>> alternative architectures but I can tell about x86_64.
>>>>>
>>>>> Packstack and Puppet-OpenStack jobs are designed to run
>>>>> within
> 8GB of
>>>>> RAM - either on a single virtual machine or on a single
>>>>> bare metal server. I would say 4 cores is the minimum (or
>>>>> otherwise job length is severely affected), 8 is best. Disk
>>>>> space is not generally a concern, easily fitting within
> 50GB of
>>>>> space.
>>>>>
>>>>> I don't have the numbers for TripleO so I'll let someone
>>>>> else
> chime in
>>>>> on that.
>>>>>
>>>>> David Moreau Simard Senior Software Engineer | Openstack
>>>>> RDO
>>>>>
>>>>> dmsimard = [irc, github, twitter]
>>>>>
>>>>>
>>>>> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng
> <mengxiandong at gmail.com <mailto:mengxiandong at gmail.com>>
>>>>> wrote:
>>>>>> We had discussed it a bit in previous RDO meeting on irc.
>>>>>> I
> want to
>>>>>> write a separate mail for more broad and in-depth
>>>>>> discussion here.
>>>>>>
>>>>>>
>>>>>> For a master release (for now it is Newton), i noticed
>>>>>> the
> promotion
>>>>>> pipes fall into three different categories: - Triple-O
>>>>>> based CI test - Packstack based test - OpenStack-Puppet
>>>>>> based test
>>>>>>
>>>>>> So what is the base minimal CI requirements to start
>>>>>> with
> for AltArch
>>>>>> support? Since many of the CI test should work even with
>>>>>> VMs
> instead
>>>>>> of physical nodes, can we start with 1-2 physical
>>>>>> servers?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Alex Meng mengxiandong at gmail.com
>>>>>> <mailto:mengxiandong at gmail.com>
>>>>>>
>>>>>> _______________________________________________ rdo-list
>>>>>> mailing list rdo-list at redhat.com
>>>>>> <mailto:rdo-list at redhat.com>
>>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
> <https://www.redhat.com/mailman/listinfo/rdo-list>
>>>>>>
>>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
> <mailto:rdo-list-unsubscribe at redhat.com>
>>>>
>>>>
>>
>>
>
>

--
Karanbir Singh, Project Lead, The CentOS Project
+44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS
GnuPG Key : http://www.karan.org/publickey.asc

--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
GnuPG Key : http://www.karan.org/publickey.asc

_______________________________________________
rdo-list mailing list
rdo-list at redhat.com
https://www.redhat.com/mailman/listinfo/rdo-list

To unsubscribe: rdo-list-unsubscribe at redhat.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20160813/54022323/attachment.html>