Hi Alex, I don't know the specifics of resource usage for alternative architectures but I can tell about x86_64. Packstack and Puppet-OpenStack jobs are designed to run within 8GB of RAM - either on a single virtual machine or on a single bare metal server. I would say 4 cores is the minimum (or otherwise job length is severely affected), 8 is best. Disk space is not generally a concern, easily fitting within 50GB of space. I don't have the numbers for TripleO so I'll let someone else chime in on that. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng <mengxiandong(a)gmail.com&gt; wrote: > We had discussed it a bit in previous RDO meeting on irc. I want to write a > separate mail > for more broad and in-depth discussion here. > > > For a master release (for now it is Newton), i noticed the promotion pipes > fall into three different categories: > - Triple-O based CI test > - Packstack based test > - OpenStack-Puppet based test > > So what is the base minimal CI requirements to start with for AltArch > support? Since many of the CI test should work even with VMs instead of > physical nodes, can we start > with 1-2 physical servers? > > Regards, > > Alex Meng > mengxiandong(a)gmail.com > > _______________________________________________ > rdo-list mailing list > rdo-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > > To unsubscribe: rdo-list-unsubscribe(a)redhat.com

Alex, Can you expand on what you mean by that ? The concurrency (or lack thereof) of the jobs are more about the design of the job itself -- or the environment it is run from as well as the environment it is run on. The jobs part of the promotion pipeline [1] run a couple times per day. [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng <mengxiandong(a)gmail.com&gt; wrote: > David, thank you for your response. > > For Packstack and Puppet-OpenStack jobs, I noticed that usually each job > takes no more than 45 minutes. How many jobs may run in parallel and how > often they are triggered? > > Regards, > > Alex Meng > mengxiandong(a)gmail.com > > On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard <dms(a)redhat.com&gt; wrote: >> >> Hi Alex, >> >> I don't know the specifics of resource usage for alternative >> architectures but I can tell about x86_64. >> >> Packstack and Puppet-OpenStack jobs are designed to run within 8GB of >> RAM - either on a single virtual machine or on a single bare metal >> server. >> I would say 4 cores is the minimum (or otherwise job length is >> severely affected), 8 is best. >> Disk space is not generally a concern, easily fitting within 50GB of >> space. >> >> I don't have the numbers for TripleO so I'll let someone else chime in on >> that. >> >> David Moreau Simard >> Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] >> >> >> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng <mengxiandong(a)gmail.com&gt; >> wrote: >> > We had discussed it a bit in previous RDO meeting on irc. I want to >> > write a >> > separate mail >> > for more broad and in-depth discussion here. >> > >> > >> > For a master release (for now it is Newton), i noticed the promotion >> > pipes >> > fall into three different categories: >> > - Triple-O based CI test >> > - Packstack based test >> > - OpenStack-Puppet based test >> > >> > So what is the base minimal CI requirements to start with for AltArch >> > support? Since many of the CI test should work even with VMs instead of >> > physical nodes, can we start >> > with 1-2 physical servers? >> > >> > Regards, >> > >> > Alex Meng >> > mengxiandong(a)gmail.com >> > >> > _______________________________________________ >> > rdo-list mailing list >> > rdo-list(a)redhat.com >> > https://www.redhat.com/mailman/listinfo/rdo-list >> > >> > To unsubscribe: rdo-list-unsubscribe(a)redhat.com > >

Right now the jobs are sequenced as outlined in the pipeline [1]. First, the job "rdo-promote-get-hash-master" will run. If it is successful, it triggers the next "stage" of the pipeline. Then, the job "tripleo-quickstart-promote-master-delorean-build-images" will run. If it is successfull, it triggers the next "stage" of the pipeline. After that you have 8 jobs that, if capacity allows (usually the case), will all run simultaneously. Otherwise, builds can be queued until a slave can process them. ... and so on. We can't realistically run all these jobs in a sequence. Considering they complete in 45 minutes (or more), we'd be looking at a pipeline of over 6 hours of builds. [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ job/rdo-delorean-promote-master/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng <mengxiandong(a)gmail.com&gt; wrote: > OK, i mean on the same time, how many concurrent CI jobs may be running? For > example, at peak load, will weirdo-master-promote-packstack-scenario001 , > weirdo-master-promote-packstack-scenario002 , > weirdo-master-promote-packstack-scenario003 run at the same time? Or they > will run in sequence by design? > > > Regards, > > Alex Meng > mengxiandong(a)gmail.com > > On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard <dms(a)redhat.com&gt; wrote: >> >> Alex, >> >> Can you expand on what you mean by that ? >> The concurrency (or lack thereof) of the jobs are more about the >> design of the job itself -- or the environment it is run from as well >> as the environment it is run on. >> >> The jobs part of the promotion pipeline [1] run a couple times per day. >> >> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ >> >> David Moreau Simard >> Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] >> >> >> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng <mengxiandong(a)gmail.com&gt; >> wrote: >> > David, thank you for your response. >> > >> > For Packstack and Puppet-OpenStack jobs, I noticed that usually each job >> > takes no more than 45 minutes. How many jobs may run in parallel and how >> > often they are triggered? >> > >> > Regards, >> > >> > Alex Meng >> > mengxiandong(a)gmail.com >> > >> > On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard <dms(a)redhat.com&gt; >> > wrote: >> >> >> >> Hi Alex, >> >> >> >> I don't know the specifics of resource usage for alternative >> >> architectures but I can tell about x86_64. >> >> >> >> Packstack and Puppet-OpenStack jobs are designed to run within 8GB of >> >> RAM - either on a single virtual machine or on a single bare metal >> >> server. >> >> I would say 4 cores is the minimum (or otherwise job length is >> >> severely affected), 8 is best. >> >> Disk space is not generally a concern, easily fitting within 50GB of >> >> space. >> >> >> >> I don't have the numbers for TripleO so I'll let someone else chime in >> >> on >> >> that. >> >> >> >> David Moreau Simard >> >> Senior Software Engineer | Openstack RDO >> >> >> >> dmsimard = [irc, github, twitter] >> >> >> >> >> >> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng < mengxiandong(a)gmail.com&gt; >> >> wrote: >> >> > We had discussed it a bit in previous RDO meeting on irc. I want to >> >> > write a >> >> > separate mail >> >> > for more broad and in-depth discussion here. >> >> > >> >> > >> >> > For a master release (for now it is Newton), i noticed the promotion >> >> > pipes >> >> > fall into three different categories: >> >> > - Triple-O based CI test >> >> > - Packstack based test >> >> > - OpenStack-Puppet based test >> >> > >> >> > So what is the base minimal CI requirements to start with for AltArch >> >> > support? Since many of the CI test should work even with VMs instead >> >> > of >> >> > physical nodes, can we start >> >> > with 1-2 physical servers? >> >> > >> >> > Regards, >> >> > >> >> > Alex Meng >> >> > mengxiandong(a)gmail.com >> >> > >> >> > _______________________________________________ >> >> > rdo-list mailing list >> >> > rdo-list(a)redhat.com >> >> > https://www.redhat.com/mailman/listinfo/rdo-list >> >> > >> >> > To unsubscribe: rdo-list-unsubscribe(a)redhat.com >> > >> > > >

we typically see hundreds of bare metal deployments per day for rdo / cloud SIG, and at the very least we should make an effort to sync across the arch's if we can.

so, as we work through what needs doing, lets make sure we factor in the user story we want to deliver at the other end.

OK, so the bottleneck is in "INSTALL / TEST (IMPORT IMAGES)" stage with 2 Triple-O scenario, 3 Packstack and 3 OpenStack-Puppet test jobs. So we may need up to (4 core, 8 GB memory)*3 to cover PackStack and OpenStack-Puppet path as each Triple-O scenario will need about 90 minutes. (So the shortest time is limited to 90 minutes assuming we maximize the parallelism level. ) Now i need some input about the resource requirements for Triple-O test scenarios. Regards, Alex Meng<mailto:mengxiandong@gmail.com> mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com> On Thu, Aug 11, 2016 at 4:11 PM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> wrote: Right now the jobs are sequenced as outlined in the pipeline [1]. First, the job "rdo-promote-get-hash-master" will run. If it is successful, it triggers the next "stage" of the pipeline. Then, the job "tripleo-quickstart-promote-master-delorean-build-images" will run. If it is successfull, it triggers the next "stage" of the pipeline. After that you have 8 jobs that, if capacity allows (usually the case), will all run simultaneously. Otherwise, builds can be queued until a slave can process them. ... and so on. We can't realistically run all these jobs in a sequence. Considering they complete in 45 minutes (or more), we'd be looking at a pipeline of over 6 hours of builds. [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-p...

David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Aug 10, 2016 at 11:52 PM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> wrote: > OK, i mean on the same time, how many concurrent CI jobs may be running? For > example, at peak load, will weirdo-master-promote-packstack-scenario001 , > weirdo-master-promote-packstack-scenario002 , > weirdo-master-promote-packstack-scenario003 run at the same > time? Or they > will run in sequence by design? > > > Regards, > > Alex Meng mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com> > > On Thu, Aug 11, 2016 at 2:50 PM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> wrote: >> >> Alex, >> >> Can you expand on what you mean by that ? The concurrency (or >> lack thereof) of the jobs are more about the design of the job >> itself -- or the environment it is run from as well as the >> environment it is run on. >> >> The jobs part of the promotion pipeline [1] run a couple times per day. >> >> [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/ <https://ci.centos.org/view/rdo/view/promotion-pipeline/> >> >> David Moreau Simard Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] >> >> >> On Wed, Aug 10, 2016 at 9:45 PM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> >> wrote: >>> David, thank you for your response. >>> >>> For Packstack and Puppet-OpenStack jobs, I noticed that >>> usually each job >>> takes no more than 45 minutes. How many jobs may run in parallel and how >>> often they are triggered? >>> >>> Regards, >>> >>> Alex Meng mengxiandong(a)gmail.com >>> <mailto:mengxiandong@gmail.com> >>> >>> On Thu, Aug 11, 2016 at 2:24 AM, David Moreau Simard <dms(a)redhat.com <mailto:dms@redhat.com>> >>> wrote: >>>> >>>> Hi Alex, >>>> >>>> I don't know the specifics of resource usage for >>>> alternative architectures but I can tell about x86_64. >>>> >>>> Packstack and Puppet-OpenStack jobs are designed to run >>>> within 8GB of >>>> RAM - either on a single virtual machine or on a single >>>> bare metal server. I would say 4 cores is the minimum (or >>>> otherwise job length is severely affected), 8 is best. Disk >>>> space is not generally a concern, easily fitting within 50GB of >>>> space. >>>> >>>> I don't have the numbers for TripleO so I'll let someone >>>> else chime in >>>> on that. >>>> >>>> David Moreau Simard Senior Software Engineer | Openstack >>>> RDO >>>> >>>> dmsimard = [irc, github, twitter] >>>> >>>> >>>> On Wed, Aug 10, 2016 at 6:18 AM, Xiandong Meng <mengxiandong(a)gmail.com <mailto:mengxiandong@gmail.com>> >>>> wrote: >>>>> We had discussed it a bit in previous RDO meeting on irc. >>>>> I want to >>>>> write a separate mail for more broad and in-depth >>>>> discussion here. >>>>> >>>>> >>>>> For a master release (for now it is Newton), i noticed >>>>> the promotion >>>>> pipes fall into three different categories: - Triple-O >>>>> based CI test - Packstack based test - OpenStack-Puppet >>>>> based test >>>>> >>>>> So what is the base minimal CI requirements to start >>>>> with for AltArch >>>>> support? Since many of the CI test should work even with >>>>> VMs instead >>>>> of physical nodes, can we start with 1-2 physical >>>>> servers? >>>>> >>>>> Regards, >>>>> >>>>> Alex Meng mengxiandong(a)gmail.com >>>>> <mailto:mengxiandong@gmail.com> >>>>> >>>>> _______________________________________________ rdo-list >>>>> mailing list rdo-list(a)redhat.com >>>>> <mailto:rdo-list@redhat.com> >>>>> https://www.redhat.com/mailman/listinfo/rdo-list <https://www.redhat.com/mailman/listinfo/rdo-list> >>>>> >>>>> To unsubscribe: rdo-list-unsubscribe(a)redhat.com <mailto:rdo-list-unsubscribe@redhat.com> >>> >>> > >