Greetings, For those of you in the To: line, I believe you are all doing something with gluster and puppet at the moment. For anyone else on rdo-list that might be interested, jump in :)

Primarily I want to get everyone talking to make sure we don't step on each other's toes. I know James has done some great work with the puppet-gluster module,

and Gilles is currently working to switch off of the now-deprecated puppet-openstack-storage module and onto puppet-gluster. Crag, Jiří, and myself are working gluster-related bugs. So let's keep in touch. I'm working to configure libgfapi support on nova compute nodes. In the old gluster module, there was a gluster::client class that just installed the required glusterfs-fuse package. This class is used by astapor in a few places (compute/cinder/glance). However there's no gluster::client class in the new module, so we'll need to remedy that somehow.

There is a class, gluster::mount::base, that ensures the packages are installed, and that class is used by each instance of gluster::mount. I'd like to reuse some of this, but I don't think we need all of it on the compute nodes (really we just need to install glusterfs-api). The simple way would be to create a new class glusterfs::apiclient that just installs the package, and include that for the nova compute case.

However I'm concerned with the other places we were previously using gluster::client. Can we use the new gluster::mount define to replace all of these instances? Or are we going to need to refactor in those places as well? I'd like to have some idea where this is all going before I start ripping it apart.

Thoughts?

-John

On 04/08/2014 02:42 PM, John Eckersberg wrote: > Greetings, > > For those of you in the To: line, I believe you are all doing something > with gluster and puppet at the moment. For anyone else on rdo-list that > might be interested, jump in :) > > Primarily I want to get everyone talking to make sure we don't step on > each other's toes. I know James has done some great work with the > puppet-gluster module, and Gilles is currently working to switch off of > the now-deprecated puppet-openstack-storage module and onto > puppet-gluster. Crag, Jiří, and myself are working gluster-related > bugs. So let's keep in touch. > > I'm working to configure libgfapi support on nova compute nodes. In the > old gluster module, there was a gluster::client class that just > installed the required glusterfs-fuse package. This class is used by > astapor in a few places (compute/cinder/glance). However there's no > gluster::client class in the new module, so we'll need to remedy that > somehow. > > There is a class, gluster::mount::base, that ensures the packages are > installed, and that class is used by each instance of gluster::mount. > I'd like to reuse some of this, but I don't think we need all of it on > the compute nodes (really we just need to install glusterfs-api). The > simple way would be to create a new class glusterfs::apiclient that just > installs the package, and include that for the nova compute case. > However I'm concerned with the other places we were previously using > gluster::client. Can we use the new gluster::mount define to replace > all of these instances? Or are we going to need to refactor in those > places as well? I'd like to have some idea where this is all going > before I start ripping it apart. > > Thoughts? > > -John > [Also CC'ing Steve and Jacob who worked a bit with gluster / foreman in recent history] In the context of the HA-all-in-one-controller host group, I believe we just would need to include the gluster::mount::base class so that we are capable of mounting glusterfs volumes. Pacemaker would be responsible for mounting the shared storage, and would do that the way Steve illustrated here: https://bugzilla.redhat.com/show_bug.cgi?id=1064050#c4 Not that that helps clarify any of your above questions. :-) --Crag

James Shubin <jshubin(a)redhat.com&gt; writes: > On Tue, 2014-04-08 at 17:42 -0400, John Eckersberg wrote: >> I'm working to configure libgfapi support on nova compute nodes. In the >> old gluster module, there was a gluster::client class that just >> installed the required glusterfs-fuse package. This class is used by >> astapor in a few places (compute/cinder/glance). However there's no >> gluster::client class in the new module, so we'll need to remedy that >> somehow. > I think you've got the right idea in the next paragraph. I'll add some > more info... > >> >> There is a class, gluster::mount::base, that ensures the packages are >> installed, and that class is used by each instance of gluster::mount. >> I'd like to reuse some of this, but I don't think we need all of it on >> the compute nodes (really we just need to install glusterfs-api). The >> simple way would be to create a new class glusterfs::apiclient that just >> installs the package, and include that for the nova compute case. > Can you detail what functionality is missing and needed? Please make > sure to state if this is relevant to upstream (GlusterFS) or just > downstream (RHS/RHEL OSP, etc...) Is it just to install one package or > is there anything else? Focusing just on this bit for now... Libvirt supports using gluster as a storage backend. It does this by calling libgfapi to interface with gluster directly. There is no fuse layer or filesystem mounts needed. If nova is configured with qemu_allowed_storage_backend=gluster, then it will generate the necessary libvirt xml to make this happen magically at launch time. The only prerequisite is for libgfapi to be available on the compute nodes (and for libvirt to have been build with gluster support enabled). The glusterfs-api package is what provides the required library.

So, I just need a class to include which provides installation of the glusterfs-api package. I was going to just add it as a normal package resource on my end, but I'm guessing this is something other folks might want to take advantage of, and the puppet-gluster module seems like the correct place to put it. It's not particular to the downstream case; anyone who might want to use libgfapi as a client would want a canonical way to set it up.

I'll submit a pull request with a new glusterfs::apiclient class that does nothing else but install the glusterfs-api package, and we can work from there. Let me know if you'd prefer another name. Maybe something like gluster::client::api, if you anticipate having other stuff under a gluster::client namespace?

John

> https://github.com/purpleidea/puppet-gluster/tree/feat/libgfapi > > To involve you more, I didn't even test it :) Please test and let me > know. If it meets your needs, I'll merge it into master. Works for me! I didn't test the server bit, but it looks sane by visual

inspection. Thanks for knocking it out so quickly.

John

Good point, I only checked by declaring gluster::api. When I declare gluster::mount::base instead, I get: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid tag "::gluster::api" at /usr/share/packstack/modules/gluster/manifests/mount/base.pp:55 on node compute.example.org Changing the second parameter of ensure_resource from '::gluster::api' to 'gluster::api' seems to make it work, for better or worse.

James Shubin <jshubin(a)redhat.com&gt; writes: > On Wed, 2014-04-09 at 17:04 -0400, John Eckersberg wrote: >> Good point, I only checked by declaring gluster::api. When I declare >> gluster::mount::base instead, I get: >> >> Error: Could not retrieve catalog from remote server: Error 400 on >> SERVER: Invalid tag "::gluster::api" >> at /usr/share/packstack/modules/gluster/manifests/mount/base.pp:55 on >> node compute.example.org >> >> Changing the second parameter of ensure_resource from '::gluster::api' >> to 'gluster::api' seems to make it work, for better or worse. > > Sounds like you're using Puppet < 3.x Can you confirm? > I think the leading :: is recommended for 3.x, but I should test if it's > required. This is with: puppet-3.2.4-3.el6_5.noarch puppet-server-3.2.4-3.el6_5.noarch