[Rdo-list] Concerning Rabbits

[Ryan O'Hara]

On Tue, Feb 04, 2014 at 12:28:31PM -0500, John Eckersberg wrote:
> (In the spirit of "Concerning Hobbits")

Thanks for kicking off this thread.

> Ryan O'Hara and I have been investigating RabbitMQ as it pertains to RDO
> recently.  There has been a lot of discussion on several disparate
> threads, so I wanted to try and capture it on the list for the benefit
> of everyone.
> 
> Ryan has been working on getting RabbitMQ running in a multi-node HA
> configuration.  I won't steal his thunder, and he can speak to it better
> than I can, so I'll defer to him on the details.

Right now I have a 3-node RabbitMQ cluster with mirrored queues. I
also put haproxy in front of this cluster and pointed all relevant
OpenStack services at the virtual IP address. This seems to work well
so far. Details instructions coming soon.

> As for me, I've been working on el7 support and bug squashing along the
> way.
> 
> The first bug[1] causes the daemon to load incredibly slow, or outright
> fail by timing out.  This is due to the SELinux policy disallowing
> name_bind on ports lower than 32768.  RabbitMQ tries to name_bind to a
> port starting at 10000, and increments if it fails.  So if you have
> SELinux in enforcing mode, you'll get 22768 AVC denials in the log
> before it finally starts.
> 
> The second bug[2] causes the daemon to intermittently fail to start due
> to a race condition in the creation of the erlang cookie file.  This
> happens only the first time the service starts.  Really this is an
> Erlang bug, but there's a workaround for the RabbitMQ case.
> 
> I've submitted patches for both issues.  Until those get merged in, I've
> rebuilt[3] RabbitMQ for F20 which includes the fixes.

Awesome.

> Beyond bugs, I've also built out RabbitMQ and all the build/runtime
> dependencies for el7.  I have a yum repo[4] on my fedorapeople page
> containing all the bits.  This is all the stuff that is presently
> missing from EPEL7.  In time, I would hope the maintainers build all
> this stuff, but for now it'll work for testing.  You will also need the
> EPEL 7 Beta repository[5] enabled.
> 
> As a side note, I built everything using mock with a local override repo
> on my workstation.  I've not used copr before but it seems relevant to
> this sort of thing, so if it's any benefit I'll look to rebuilt the el7
> stack there for easier consumption.
> 
> Hopefully this helps get the discussion into one place, and provide a
> baseline for further investigation by everyone interested in RabbitMQ.

John and I will be putting all of this in a wiki page on the RDO
website in the very near future. I'll send email to the list when it
is ready to be reviewed.

Ryan

> John.
> 
> ---
> [1] Is really two bugzillas, but the same bug:
> [1a] https://bugzilla.redhat.com/show_bug.cgi?id=998682
> [1b] https://bugzilla.redhat.com/show_bug.cgi?id=1032595
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1059913
> [3] http://jeckersb.fedorapeople.org/rabbitmq-server-3.1.5-3.fc20.noarch.rpm
> [4] http://jeckersb.fedorapeople.org/rabbitmq-el7/
> [5] http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/
> 
> _______________________________________________
> Rdo-list mailing list
> Rdo-list at redhat.com
> https://www.redhat.com/mailman/listinfo/rdo-list