[Rdo-list] Tripleo Liberty Cinder permission denied

Fri Apr 29 13:55:52 UTC 2016

Hi Charles,

On Fri, 2016-04-29 at 14:32 +0100, Charles Short wrote:
> ok applying specific uid/gid 165 to the NetApp volume solved the
> permission error.
> Cinder now successfully writes .cinderSecureEnvIndicator to the
> export.

Great stuff.

> But I have a new error now and the service is still down...
>
> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
> cinder.volume.manager [req-a4544310-84c6-4602-a944-7efaee5ff90f - - -
> -
> -] Failed to initialize driver.
> ...
> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
> cinder.volume.manager     raise NaApiError('Unexpected error')
> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
> cinder.volume.manager NaApiError: NetApp API failed. Reason -
> Unexpected
> error:unknown
>
> Have you seen this one?

No but maybe the conf might be pointing to the mgmt IP rather than the
data IP? Are you using 7 mode or ontap? Feel free to post the conf file
(redacting security stuff obviously) if you like. Also check
authentication perhaps.

> Charles
>
>
> On 29/04/2016 12:40, Charles Short wrote:
> >
> > Hi,
> >
> > Thanks for this.
> >
> > 1) Yes unlikely as root can write to it.
> >
> > 2) Already set to permissive.
> >
> > 3) When we set up our previous OSP6 (Juno) environment using the
> > same
> > NetApp storage system, only root had permission to write to the
> > NetApp
> > volume and all worked fine. When our storage team set up this
> > volume,
> > it was also as root (same settings as the last setup). I suspect
> > that
> > Cinder uid usage is now enforced. I will get the storage team to
> > make
> > the changes and see if this helps
> >
> > Regards
> >
> > Charles
> >
> >
> > On 29/04/2016 11:49, Christopher Brown wrote:
> > >
> > > Hi Charles,
> > >
> > > I had similar problems with a netapp deployment. Three
> > > possibilities to
> > > check:
> > >
> > > 1. Security on the export shipped by default with a missing
> > > netmask on
> > > the export so 0.0.0.0 should be 0.0.0.0/24 or whatever you want
> > > to
> > > restrict to. Though as you can write with sudo probably not the
> > > issue.
> > >
> > > 2. SELinux - I wonder if you try temporarily running setenforce 0
> > > and
> > > re-mounting if it has the same problem?
> > >
> > > 3. Cinder and Glance exports should be created with their
> > > respective
> > > UIDs as owner. I blogged about it here:
> > >
> > > https://chruz.wordpress.com/2016/03/31/openstack-and-clustered-da
> > > ta-ont
> > > ap/
> > >
> > > Hope some of this is helpful but if not would be glad to hear of
> > > outcome.
> > >
> > > Regards
> > >
> > > On Fri, 2016-04-29 at 11:30 +0100, Charles Short wrote:
> > > >
> > > > Hi,
> > > >
> > > > Deployed Tripleo Liberty stable on baremetal, but NetApp NFS
> > > > Cinder
> > > > backend is not working.
> > > >
> > > > It is auto-mounting no problem, and I can write to it with
> > > > sudo, but
> > > > the
> > > > 'tripleo_netapp' backend is enabled with state 'down' as it
> > > > cannot
> > > > write
> > > > to the mount point.
> > > >
> > > >    cinder service-list | grep tripleo_netapp
> > > > >
> > > > >   cinder-volume   | hostgroup at tripleo_netapp | nova | enabled
> > > > > | down
> > > > [heat-admin at overcloud-controller-0 ~]$ mount | grep cinder
> > > > [ip addr]:/[mount] on
> > > > /var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f type nfs4
> > > > (rw,relatime,vers=4.1,rsize=65536,wsize=65536,namlen=255,hard,p
> > > > roto=t
> > > > cp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=[ip
> > > > addr],local_lock=none,addr=[ip addr])
> > > >
> > > > I can write to it -
> > > >
> > > > [heat-admin at overcloud-controller-0 ~]$ sudo touch
> > > > /var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f/test
> > > > [heat-admin at overcloud-controller-0 ~]$
> > > >
> > > > But Cinder cannot -
> > > >
> > > > /var/log/cinder/volume.log:2016-04-29 09:43:49.870 56696 ERROR
> > > > cinder.volume.drivers.remotefs [req-99928048-2446-4967-99ba-
> > > > 0e85c2ba5712
> > > > - - - - -] Failed to created Cinder secure environment
> > > > indicator
> > > > file:
> > > > [Errno 13] Permission denied:
> > > > '/var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f/.cinderSe
> > > > cureEn
> > > > vIndicator'
> > > >
> > > > So this look like an issue with the user that Cinder is using
> > > > to
> > > > write
> > > > to the export (cinder?)?
> > > >
> > > > I have tried setting this option in cinder.conf, but it makes
> > > > no
> > > > difference
> > > >
> > > > nas_secure_file_operations = False
> > > >
> > > > "Allow network-attached storage systems to operate in a secure
> > > > environment where root level access is not permitted. If set to
> > > > False,
> > > > access is as the root user and insecure. If set to True, access
> > > > is
> > > > not
> > > > as root. If set to auto, a check is done to determine if this
> > > > is a
> > > > new
> > > > installation: True is used if so, otherwise False. Default is
> > > > auto"
> > > >
> > > > Any help appreciated
> > > >
> > > > Thanks
> > > >
> > > > Charles
> > > >
> > > > --
> > > > Charles Short
> > > > Cloud Engineer
> > > > Virtualization and Cloud Team
> > > > European Bioinformatics Institute (EMBL-EBI)
> > > > Tel: +44 (0)1223 494205
> > > >
> > > > _______________________________________________
> > > > Rdo-list mailing list
> > > > Rdo-list at redhat.com
> > > > https://www.redhat.com/mailman/listinfo/rdo-list
> > > >
> > > > To unsubscribe: rdo-list-unsubscribe at redhat.com
> > > --
> > > Regards,
> > >
> > > Christopher Brown
> > > OpenStack Engineer
> > > OCF plc
> > >
> > > Tel: +44 (0)114 257 2200
> > > Web: www.ocf.co.uk
> > > Blog: blog.ocf.co.uk
> > > Twitter: @ocfplc
> > >
> > > Please note, any emails relating to an OCF Support request must
> > > always
> > > be sent to support at ocf.co.uk for a ticket number to be generated
> > > or
> > > existing support ticket to be updated. Should this not be done
> > > then OCF
> > >
> > > cannot be held responsible for requests not dealt with in a
> > > timely
> > > manner.
> > >
> > > OCF plc is a company registered in England and Wales. Registered
> > > number
> > >
> > > 4132533, VAT number GB 780 6803 14. Registered office address:
> > > OCF plc,
> > >
> > > 5 Rotunda Business Centre, Thorncliffe Park, Chapeltown,
> > > Sheffield S35
> > > 2PG.
> > >
> > > If you have received this message in error, please notify us
> > > immediately and remove it from your system.
> --
> Charles Short
> Cloud Engineer
> Virtualization and Cloud Team
> European Bioinformatics Institute (EMBL-EBI)
> Tel: +44 (0)1223 494205
>
--
Regards,

Christopher Brown
OpenStack Engineer
OCF plc

Tel: +44 (0)114 257 2200
Web: www.ocf.co.uk
Blog: blog.ocf.co.uk
Twitter: @ocfplc

Please note, any emails relating to an OCF Support request must always
be sent to support at ocf.co.uk for a ticket number to be generated or
existing support ticket to be updated. Should this not be done then OCF

cannot be held responsible for requests not dealt with in a timely
manner.

OCF plc is a company registered in England and Wales. Registered number

4132533, VAT number GB 780 6803 14. Registered office address: OCF plc,

5 Rotunda Business Centre, Thorncliffe Park, Chapeltown, Sheffield S35
2PG.

If you have received this message in error, please notify us
immediately and remove it from your system.