[rdo-list] FYI: New slaves now in use on ci.centos.org for RDO

[David Moreau Simard]

I've scaled down the amount of total threads we had after being
notified by the ci.centos.org team that our usage had gone
significantly up [1].

We had 16 threads before and with the new slaves we were up to 34 threads.
I've scaled down this number back to 24 for the time being to see if
that improves the situation.

[1] https://lists.centos.org/pipermail/ci-users/2016-August/000330.html

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Wed, Aug 3, 2016 at 2:31 PM, David Moreau Simard <dms at redhat.com> wrote:
> We noticed that the promotion jobs relied on a property file created
> locally on the slave.
>
> Since promotion jobs rely on that property file to share parameters
> (such as which trunk repository jobs should be testing) and jobs could
> be running on any of the four slaves, this was problematic.
>
> For the time being, promotion jobs have been pinned to a single slave
> but we are looking at a way to remove this limitation to benefit from
> the redundancy and increased capacity.
>
> David Moreau Simard
> Senior Software Engineer | Openstack RDO
>
> dmsimard = [irc, github, twitter]
>
>
> On Wed, Aug 3, 2016 at 12:40 PM, David Moreau Simard <dms at redhat.com> wrote:
>> Now that's a lot of running jobs [1] :)
>>
>> [1]: http://i.imgur.com/s7Cq53M.png
>>
>> David Moreau Simard
>> Senior Software Engineer | Openstack RDO
>>
>> dmsimard = [irc, github, twitter]
>>
>>
>> On Wed, Aug 3, 2016 at 12:14 PM, David Moreau Simard <dms at redhat.com> wrote:
>>> Hi,
>>>
>>> We've tested successfully three new slaves out of the "beta" OpenStack
>>> cloud on ci.centos.org.
>>> We're going to be lowering the amount of threads on our existing slave
>>> and spread the load evenly across the 4 slaves.
>>>
>>> The objective is two-fold:
>>> - Spread load evenly across four slaves rather than one: redundancy
>>> and additional capacity/concurrency
>>> - Test real workloads on the ci.centos.org OpenStack cloud before it
>>> is opened up to additional tenants
>>>
>>> I will be monitoring closely (moreso than usual) the jobs but
>>> everything /should/ work.
>>> You can tell on which slave a particular job was run from at the very
>>> beginning of the console output, it looks like this:
>>> "Building remotely on rdo-ci-cloudslave01 (rdo) in workspace [...]"
>>>
>>> If you notice anything odd about jobs running on the new cloudslaves
>>> machines, please let me know directly.
>>>
>>> Thanks !
>>>
>>> David Moreau Simard
>>> Senior Software Engineer | Openstack RDO
>>>
>>> dmsimard = [irc, github, twitter]