[Rdo-list] Permission denied errors after installing a Storage Node in a Swift Cluster

[Diogo Vieira]

On May 15, 2014, at 6:05 PM, Diogo Vieira <dfv at eurotux.com> wrote:

> On May 6, 2014, at 7:35 PM, Pádraig Brady <pbrady at redhat.com> wrote:
> 
>> On 05/06/2014 06:47 PM, Diogo Vieira wrote:
>>> On May 6, 2014, at 11:06 AM, Pádraig Brady <pbrady at redhat.com> wrote:
>>> 
>>>> On 05/06/2014 09:47 AM, Diogo Vieira wrote:
>>>>> On May 6, 2014, at 2:53 AM, Pádraig Brady <pbrady at redhat.com <mailto:pbrady at redhat.com>> wrote:
>>>>> 
>>>>>> What version of swift are you using?
>>>>>> 
>>>>>> swift-1.13.1.rc2 could have permissions errors,
>>>>>> while we included a patch in the RDO icehouse swift-1.13.1 release to fix
>>>>>> http://pad.lv/1302700 which on first glance could be related?
>>>>>> 
>>>>>> thanks,
>>>>>> Pádraig.
>>>>> 
>>>>> I'm using 1.13.1-1.fc21 (I'm using Fedora) as you can see:
>>>>> 
>>>>>  # yum info openstack-swift
>>>>>  Loaded plugins: priorities
>>>>>  196 packages excluded due to repository priority protections
>>>>>  Installed Packages
>>>>>  Name        : openstack-swift
>>>>>  Arch        : noarch
>>>>>  Version     : 1.13.1
>>>>>  Release     : 1.fc21
>>>>> 
>>>>> 
>>>>> So the fix should already be present right?
>>>> 
>>>> Yes, must be something else so.
>>>> 
>>>> thanks,
>>>> Pádraig.
>>> 
>>> That's unfortunate then. One thing's for sure: these errors aren't supposed to happen right?
>>> 
>>> If someone else has any idea of what could be the problem I would greatly appreciate since this is a recurring problem (even between different Openstack  and Packstack versions, since it was tested in Havana and Icehouse).
>>> 
>>> Thank you very much,
>>> Diogo Vieira
>>> 
>> 
>> Ah you see this in Havana repos also, that's NB info.
>> 
>> Pete any ideas?
>> 
>> thanks,
>> Pádraig.
> 
> Yes, the issue happened with an older Havana installation and with the new Icehouse version.
> 
> Sorry to bring this up again, but I'm really lost and have no idea what could the problem be.
> 
> Nobody has an idea for trying to resolve the issue or issues (since I don't know if they're related)? Should I file a bug report?
> 
> Thank you,
> 
> Diogo Vieira <dfv at eurotux.com>
> Programador 
> Eurotux Informática, S.A. | www.eurotux.com
> (t) +351 253 680 300

Hello again,

I'm sorry for answering my own email but I found something I believe to be the problem. After some time trying to see what the problem was I ended up searching for what could cause the permission problems. This was one of the errors I found in the syslog:

May 16 10:07:00 host-10-10-6-30 object-auditor: ERROR Trying to audit /srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/swift/obj/auditor.py", line 173, in failsafe_object_audit#012 self.object_audit(location)#012 File "/usr/lib/python2.7/site-packages/swift/obj/auditor.py", line 191, in object_audit#012 with df.open():#012 File "/usr/lib/python2.7/site-packages/swift/obj/diskfile.py", line 1029, in open#012 data_file, meta_file)#012 File "/usr/lib/python2.7/site-packages/swift/obj/diskfile.py", line 1247, in _construct_from_data_file#012 fp = open(data_file, 'rb')#012IOError: [Errno 13] Permission denied: '/srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a/1399288004.47239.data'

So the problems was in '/srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a/1399288004.47239.data'. I tried the following set of commands after I made sure that the swift-object-auditor was run by the swift user (using 'ps faxu'):

# su - swift

-bash-4.2$ ls -lha /srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a
ls: cannot access /srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a/.: Permission denied
ls: cannot access /srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a/..: Permission denied
ls: cannot access /srv/node/device3/objects/134106/f2a/82f6a3461bb69f80918a1a508a8bdf2a/1399288004.47239.data: Permission denied
total 0
d????????? ? ? ? ?            ? .
d????????? ? ? ? ?            ? ..
?????????? ? ? ? ?            ? 1399288004.47239.data

-bash-4.2$ ls -lha /srv/node/device3/objects/134106/f2a/
total 0
drwxr--r-- 3 swift swift 45 May  5 11:06 .
drwxr-xr-x 3 swift swift 45 May 16 10:57 ..
drw-r--r-- 2 swift swift 34 May  5 11:06 82f6a3461bb69f80918a1a508a8bdf2a

So you see '82f6a3461bb69f80918a1a508a8bdf2a' doesn't have execute permissions and I believe that is the problem.

My question now is, could this be caused by some misconfiguration on my part or is it a bug in swift, since it was not me that created the folder?

Thank you once again,

Diogo Vieira <dfv at eurotux.com>
Programador 
Eurotux Informática, S.A. | www.eurotux.com
(t) +351 253 680 300



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rdoproject.org/pipermail/dev/attachments/20140516/9a5cd6d5/attachment.html>