[rdo-list] RDO CI hardware requirements for CentOS altarch

[Xiandong Meng]

OK, so the bottleneck is in "INSTALL / TEST (IMPORT IMAGES)" stage with 2
Triple-O scenario, 3 Packstack and 3 OpenStack-Puppet test jobs. So we may
need up to (4 core, 8 GB memory)*3 to cover PackStack and OpenStack-Puppet
path as each Triple-O scenario will need about 90 minutes. (So the shortest
time is limited to 90 minutes assuming we maximize the parallelism level. )

Now i need some input about the resource requirements for Triple-O test
scenarios.



Regards,

Alex Meng <mengxiandong at gmail.com>
mengxiandong at gmail.com

On Thu, Aug 11, 2016 at 4:11 PM, David Moreau Simard <dms at redhat.com> wrote:

> Right now the jobs are sequenced as outlined in the pipeline [1].
>
> First, the job "rdo-promote-get-hash-master" will run. If it is
> successful, it triggers the next "stage" of the pipeline.
> Then, the job "tripleo-quickstart-promote-master-delorean-build-images"
> will run. If it is successfull, it triggers the next "stage" of the
> pipeline.
> After that you have 8 jobs that, if capacity allows (usually the
> case), will all run simultaneously. Otherwise, builds can be queued
> until a slave can process them.
> ... and so on.
>
> We can't realistically run all these jobs in a sequence.
> Considering they complete in 45 minutes (or more), we'd be looking at
> a pipeline of over 6 hours of builds.
>
> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
> job/rdo-delorean-promote-master/
>
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
>
> dmsimard = [irc, github, twitter]
>
>
> On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng <mengxiandong at gmail.com>
> wrote:
> > OK, i mean on the same time, how many concurrent CI jobs may be running?
> For
> > example, at peak load, will weirdo-master-promote-packstack-scenario001
> ,
> > weirdo-master-promote-packstack-scenario002 ,
> > weirdo-master-promote-packstack-scenario003 run at the same time? Or
> they
> > will run in sequence by design?
> >
> >
> > Regards,
> >
> > Alex Meng
> > mengxiandong at gmail.com
> >
> > On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard <dms at redhat.com>
> wrote:
> >>
> >> Alex,
> >>
> >> Can you expand on what you mean by that ?
> >> The concurrency (or lack thereof) of the jobs are more about the
> >> design of the job itself -- or the environment it is run from as well
> >> as the environment it is run on.
> >>
> >> The jobs part of the promotion pipeline [1] run a couple times per day.
> >>
> >> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/
> >>
> >> David Moreau Simard
> >> Senior Software Engineer | Openstack RDO
> >>
> >> dmsimard = [irc, github, twitter]
> >>
> >>
> >> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng <mengxiandong at gmail.com>
> >> wrote:
> >> > David, thank you for your response.
> >> >
> >> > For Packstack and Puppet-OpenStack jobs, I noticed that usually each
> job
> >> > takes no more than 45 minutes. How many jobs may run in parallel and
> how
> >> > often they are triggered?
> >> >
> >> > Regards,
> >> >
> >> > Alex Meng
> >> > mengxiandong at gmail.com
> >> >
> >> > On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard <dms at redhat.com>
> >> > wrote:
> >> >>
> >> >> Hi Alex,
> >> >>
> >> >> I don't know the specifics of resource usage for alternative
> >> >> architectures but I can tell about x86_64.
> >> >>
> >> >> Packstack and Puppet-OpenStack jobs are designed to run within 8GB of
> >> >> RAM - either on a single virtual machine or on a single bare metal
> >> >> server.
> >> >> I would say 4 cores is the minimum (or otherwise job length is
> >> >> severely affected), 8 is best.
> >> >> Disk space is not generally a concern, easily fitting within 50GB of
> >> >> space.
> >> >>
> >> >> I don't have the numbers for TripleO so I'll let someone else chime
> in
> >> >> on
> >> >> that.
> >> >>
> >> >> David Moreau Simard
> >> >> Senior Software Engineer | Openstack RDO
> >> >>
> >> >> dmsimard = [irc, github, twitter]
> >> >>
> >> >>
> >> >> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng <
> mengxiandong at gmail.com>
> >> >> wrote:
> >> >> > We had discussed it a bit in previous RDO meeting on irc. I want to
> >> >> > write a
> >> >> > separate mail
> >> >> > for more broad and in-depth discussion here.
> >> >> >
> >> >> >
> >> >> > For a master release (for now it is Newton), i noticed the
> promotion
> >> >> > pipes
> >> >> > fall into three different categories:
> >> >> > - Triple-O based CI test
> >> >> > - Packstack based test
> >> >> > - OpenStack-Puppet based test
> >> >> >
> >> >> > So what is the base minimal CI requirements to start with for
> AltArch
> >> >> > support? Since many of the CI test should work even with VMs
> instead
> >> >> > of
> >> >> > physical nodes, can we start
> >> >> > with 1-2 physical servers?
> >> >> >
> >> >> > Regards,
> >> >> >
> >> >> > Alex Meng
> >> >> > mengxiandong at gmail.com
> >> >> >
> >> >> > _______________________________________________
> >> >> > rdo-list mailing list
> >> >> > rdo-list at redhat.com
> >> >> > https://www.redhat.com/mailman/listinfo/rdo-list
> >> >> >
> >> >> > To unsubscribe: rdo-list-unsubscribe at redhat.com
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20160811/2e96d4b6/attachment.html>