Re: [rdo-list] RDO CI hardware requirements for CentOS altarch

OK, so the bottleneck is in "INSTALL / TEST (IMPORT IMAGES)" stage with 2 Triple-O scenario, 3 Packstack and 3 OpenStack-Puppet test jobs. So we may need up to (4 core, 8 GB memory)*3 to cover PackStack and OpenStack-Puppet path as each Triple-O scenario will need about 90 minutes. (So the shortest time is limited to 90 minutes assuming we maximize the parallelism level. ) Now i need some input about the resource requirements for Triple-O test scenarios. Regards, Alex Meng<mailto:mengxiandong@gmail.com> mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com> On Thu, Aug 11, 2016 at 4:11 PM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> wrote: Right now the jobs are sequenced as outlined in the pipeline [1]. First, the job "rdo-promote-get-hash-master" will run. If it is successful, it triggers the next "stage" of the pipeline. Then, the job "tripleo-quickstart-promote-master-delorean-build-images" will run. If it is successfull, it triggers the next "stage" of the pipeline. After that you have 8 jobs that, if capacity allows (usually the case), will all run simultaneously. Otherwise, builds can be queued until a slave can process them. ... and so on. We can't realistically run all these jobs in a sequence. Considering they complete in 45 minutes (or more), we'd be looking at a pipeline of over 6 hours of builds. [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-p...

David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> wrote: > OK, i mean on the same time, how many concurrent CI jobs may be running? For > example, at peak load, will weirdo-master-promote-packstack-scenario001 , > weirdo-master-promote-packstack-scenario002 , > weirdo-master-promote-packstack-scenario003 run at the same > time? Or they > will run in sequence by design? > > > Regards, > > Alex Meng mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com> > > On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> wrote: >> >> Alex, >> >> Can you expand on what you mean by that ? The concurrency (or >> lack thereof) of the jobs are more about the design of the job >> itself -- or the environment it is run from as well as the >> environment it is run on. >> >> The jobs part of the promotion pipeline [1] run a couple times per day. >> >> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ <https://ci.centos.org/view/rdo/view/promotion-pipeline/> >> >> David Moreau Simard Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] >> >> >> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> >> wrote: >>> David, thank you for your response. >>> >>> For Packstack and Puppet-OpenStack jobs, I noticed that >>> usually each job >>> takes no more than 45 minutes. How many jobs may run in parallel and how >>> often they are triggered? >>> >>> Regards, >>> >>> Alex Meng mengxiandong(a)gmail.com >>> <mailto:mengxiandong@gmail.com> >>> >>> On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> >>> wrote: >>>> >>>> Hi Alex, >>>> >>>> I don't know the specifics of resource usage for >>>> alternative architectures but I can tell about x86_64. >>>> >>>> Packstack and Puppet-OpenStack jobs are designed to run >>>> within 8GB of >>>> RAM - either on a single virtual machine or on a single >>>> bare metal server. I would say 4 cores is the minimum (or >>>> otherwise job length is severely affected), 8 is best. Disk >>>> space is not generally a concern, easily fitting within 50GB of >>>> space. >>>> >>>> I don't have the numbers for TripleO so I'll let someone >>>> else chime in >>>> on that. >>>> >>>> David Moreau Simard Senior Software Engineer | Openstack >>>> RDO >>>> >>>> dmsimard = [irc, github, twitter] >>>> >>>> >>>> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> >>>> wrote: >>>>> We had discussed it a bit in previous RDO meeting on irc. >>>>> I want to >>>>> write a separate mail for more broad and in-depth >>>>> discussion here. >>>>> >>>>> >>>>> For a master release (for now it is Newton), i noticed >>>>> the promotion >>>>> pipes fall into three different categories: - Triple-O >>>>> based CI test - Packstack based test - OpenStack-Puppet >>>>> based test >>>>> >>>>> So what is the base minimal CI requirements to start >>>>> with for AltArch >>>>> support? Since many of the CI test should work even with >>>>> VMs instead >>>>> of physical nodes, can we start with 1-2 physical >>>>> servers? >>>>> >>>>> Regards, >>>>> >>>>> Alex Meng mengxiandong(a)gmail.com >>>>> <mailto:mengxiandong@gmail.com> >>>>> >>>>> _______________________________________________ rdo-list >>>>> mailing list rdo-list(a)redhat.com >>>>> <mailto:rdo-list@redhat.com> >>>>> https://www.redhat.com/mailman/listinfo/rdo-list <https://www.redhat.com/mailman/listinfo/rdo-list> >>>>> >>>>> To unsubscribe: rdo-list-unsubscribe(a)redhat.com <mailto:rdo-list-unsubscribe@redhat.com> >>> >>> > >