[Rdo-list] Tripleo Liberty Cinder permission denied

Fri Apr 29 14:26:44 UTC 2016

Hi,

Yes many thanks, I simply had the same IP for the NetApp server name  as 
the NetApp vServer name which meant the API was trying to use the data 
path not mgmt path.
Fixed

Charles

On 29/04/2016 14:55, Christopher Brown wrote:
> Hi Charles,
>
> On Fri, 2016-04-29 at 14:32 +0100, Charles Short wrote:
>> ok applying specific uid/gid 165 to the NetApp volume solved the
>> permission error.
>> Cinder now successfully writes .cinderSecureEnvIndicator to the
>> export.
> Great stuff.
>
>> But I have a new error now and the service is still down...
>>
>> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
>> cinder.volume.manager [req-a4544310-84c6-4602-a944-7efaee5ff90f - - -
>> -
>> -] Failed to initialize driver.
>> ...
>> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
>> cinder.volume.manager     raise NaApiError('Unexpected error')
>> /var/log/cinder/volume.log:2016-04-29 13:20:24.004 3902 ERROR
>> cinder.volume.manager NaApiError: NetApp API failed. Reason -
>> Unexpected
>> error:unknown
>>
>> Have you seen this one?
> No but maybe the conf might be pointing to the mgmt IP rather than the
> data IP? Are you using 7 mode or ontap? Feel free to post the conf file
> (redacting security stuff obviously) if you like. Also check
> authentication perhaps.
>
>> Charles
>>
>>
>> On 29/04/2016 12:40, Charles Short wrote:
>>> Hi,
>>>
>>> Thanks for this.
>>>
>>> 1) Yes unlikely as root can write to it.
>>>
>>> 2) Already set to permissive.
>>>
>>> 3) When we set up our previous OSP6 (Juno) environment using the
>>> same
>>> NetApp storage system, only root had permission to write to the
>>> NetApp
>>> volume and all worked fine. When our storage team set up this
>>> volume,
>>> it was also as root (same settings as the last setup). I suspect
>>> that
>>> Cinder uid usage is now enforced. I will get the storage team to
>>> make
>>> the changes and see if this helps
>>>
>>> Regards
>>>
>>> Charles
>>>
>>>
>>> On 29/04/2016 11:49, Christopher Brown wrote:
>>>> Hi Charles,
>>>>
>>>> I had similar problems with a netapp deployment. Three
>>>> possibilities to
>>>> check:
>>>>
>>>> 1. Security on the export shipped by default with a missing
>>>> netmask on
>>>> the export so 0.0.0.0 should be 0.0.0.0/24 or whatever you want
>>>> to
>>>> restrict to. Though as you can write with sudo probably not the
>>>> issue.
>>>>
>>>> 2. SELinux - I wonder if you try temporarily running setenforce 0
>>>> and
>>>> re-mounting if it has the same problem?
>>>>
>>>> 3. Cinder and Glance exports should be created with their
>>>> respective
>>>> UIDs as owner. I blogged about it here:
>>>>
>>>> https://chruz.wordpress.com/2016/03/31/openstack-and-clustered-da
>>>> ta-ont
>>>> ap/
>>>>
>>>> Hope some of this is helpful but if not would be glad to hear of
>>>> outcome.
>>>>
>>>> Regards
>>>>
>>>> On Fri, 2016-04-29 at 11:30 +0100, Charles Short wrote:
>>>>> Hi,
>>>>>
>>>>> Deployed Tripleo Liberty stable on baremetal, but NetApp NFS
>>>>> Cinder
>>>>> backend is not working.
>>>>>
>>>>> It is auto-mounting no problem, and I can write to it with
>>>>> sudo, but
>>>>> the
>>>>> 'tripleo_netapp' backend is enabled with state 'down' as it
>>>>> cannot
>>>>> write
>>>>> to the mount point.
>>>>>
>>>>>     cinder service-list | grep tripleo_netapp
>>>>>>    cinder-volume   | hostgroup at tripleo_netapp | nova | enabled
>>>>>> | down
>>>>> [heat-admin at overcloud-controller-0 ~]$ mount | grep cinder
>>>>> [ip addr]:/[mount] on
>>>>> /var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f type nfs4
>>>>> (rw,relatime,vers=4.1,rsize=65536,wsize=65536,namlen=255,hard,p
>>>>> roto=t
>>>>> cp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=[ip
>>>>> addr],local_lock=none,addr=[ip addr])
>>>>>
>>>>> I can write to it -
>>>>>
>>>>> [heat-admin at overcloud-controller-0 ~]$ sudo touch
>>>>> /var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f/test
>>>>> [heat-admin at overcloud-controller-0 ~]$
>>>>>
>>>>> But Cinder cannot -
>>>>>
>>>>> /var/log/cinder/volume.log:2016-04-29 09:43:49.870 56696 ERROR
>>>>> cinder.volume.drivers.remotefs [req-99928048-2446-4967-99ba-
>>>>> 0e85c2ba5712
>>>>> - - - - -] Failed to created Cinder secure environment
>>>>> indicator
>>>>> file:
>>>>> [Errno 13] Permission denied:
>>>>> '/var/lib/cinder/mnt/3fb6f6744c383eacbe46593911aa4b0f/.cinderSe
>>>>> cureEn
>>>>> vIndicator'
>>>>>
>>>>> So this look like an issue with the user that Cinder is using
>>>>> to
>>>>> write
>>>>> to the export (cinder?)?
>>>>>
>>>>> I have tried setting this option in cinder.conf, but it makes
>>>>> no
>>>>> difference
>>>>>
>>>>> nas_secure_file_operations = False
>>>>>
>>>>> "Allow network-attached storage systems to operate in a secure
>>>>> environment where root level access is not permitted. If set to
>>>>> False,
>>>>> access is as the root user and insecure. If set to True, access
>>>>> is
>>>>> not
>>>>> as root. If set to auto, a check is done to determine if this
>>>>> is a
>>>>> new
>>>>> installation: True is used if so, otherwise False. Default is
>>>>> auto"
>>>>>
>>>>> Any help appreciated
>>>>>
>>>>> Thanks
>>>>>
>>>>> Charles
>>>>>
>>>>> --
>>>>> Charles Short
>>>>> Cloud Engineer
>>>>> Virtualization and Cloud Team
>>>>> European Bioinformatics Institute (EMBL-EBI)
>>>>> Tel: +44 (0)1223 494205
>>>>>
>>>>> _______________________________________________
>>>>> Rdo-list mailing list
>>>>> Rdo-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>>>
>>>>> To unsubscribe: rdo-list-unsubscribe at redhat.com
>>>> --
>>>> Regards,
>>>>
>>>> Christopher Brown
>>>> OpenStack Engineer
>>>> OCF plc
>>>>
>>>> Tel: +44 (0)114 257 2200
>>>> Web: www.ocf.co.uk
>>>> Blog: blog.ocf.co.uk
>>>> Twitter: @ocfplc
>>>>
>>>> Please note, any emails relating to an OCF Support request must
>>>> always
>>>> be sent to support at ocf.co.uk for a ticket number to be generated
>>>> or
>>>> existing support ticket to be updated. Should this not be done
>>>> then OCF
>>>>
>>>> cannot be held responsible for requests not dealt with in a
>>>> timely
>>>> manner.
>>>>
>>>> OCF plc is a company registered in England and Wales. Registered
>>>> number
>>>>
>>>> 4132533, VAT number GB 780 6803 14. Registered office address:
>>>> OCF plc,
>>>>
>>>> 5 Rotunda Business Centre, Thorncliffe Park, Chapeltown,
>>>> Sheffield S35
>>>> 2PG.
>>>>
>>>> If you have received this message in error, please notify us
>>>> immediately and remove it from your system.
>> --
>> Charles Short
>> Cloud Engineer
>> Virtualization and Cloud Team
>> European Bioinformatics Institute (EMBL-EBI)
>> Tel: +44 (0)1223 494205
>>
> --
> Regards,
>
> Christopher Brown
> OpenStack Engineer
> OCF plc
>
> Tel: +44 (0)114 257 2200
> Web: www.ocf.co.uk
> Blog: blog.ocf.co.uk
> Twitter: @ocfplc
>
> Please note, any emails relating to an OCF Support request must always
> be sent to support at ocf.co.uk for a ticket number to be generated or
> existing support ticket to be updated. Should this not be done then OCF
>
> cannot be held responsible for requests not dealt with in a timely
> manner.
>
> OCF plc is a company registered in England and Wales. Registered number
>
> 4132533, VAT number GB 780 6803 14. Registered office address: OCF plc,
>
> 5 Rotunda Business Centre, Thorncliffe Park, Chapeltown, Sheffield S35
> 2PG.
>
> If you have received this message in error, please notify us
> immediately and remove it from your system.

-- 
Charles Short
Cloud Engineer
Virtualization and Cloud Team
European Bioinformatics Institute (EMBL-EBI)
Tel: +44 (0)1223 494205