I created an Icehouse cluster that uses Cinder for object storage (using the NFS driver). I use an Isilon NL cluster and NFS mount to that. I'm getting these errors concerning "Circular Directory Structure" and I can't create volumes because of it. From the commands it looks like "snapshots" are supposed to be ignored but apparently they are not. Any suggestions would be greatly appreciated. Thank you, Richard Text from /var/log/cinder/volume.log: 2014-08-12 15:24:12.035 9485 AUDIT cinder.service [-] Starting cinder-volume node (version 2014.1.1) 2014-08-12 15:24:12.036 9485 INFO cinder.volume.manager [req-36376eb6-d919-4451-9c83-a2bae72c85c5 - - - - -] Starting volume driver NfsDriver (1.1.0) 2014-08-12 15:24:12.329 9485 INFO cinder.volume.manager [req-36376eb6-d919-4451-9c83-a2bae72c85c5 - - - - -] Updating volume status 2014-08-12 15:24:12.334 9485 INFO cinder.brick.remotefs.remotefs [req-36376eb6-d919-4451-9c83-a2bae72c85c5 - - - - -] Already mounted: /var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03 2014-08-12 15:24:12.428 9485 ERROR cinder.openstack.common.threadgroup [-] Unexpected error while running command. Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf du -sb --apparent-size --exclude *snapshot* /var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03 Exit code: 1 Stdout: '26\t/var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03\n' Stderr: "/usr/bin/du: WARNING: Circular directory structure.\nThis almost certainly means that you have a corrupted file system.\nNOTIFY YOUR SYSTEM MANAGER.\nThe following directory is part of the cycle:\n `/var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03/.snapshot'\n\n" 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup Traceback (most recent call last): 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/openstack/common/threadgroup.py", line 125, in wait 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup x.wait() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/openstack/common/threadgroup.py", line 47, in wait 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup return self.thread.wait() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 168, in wait 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup return self._exit_event.wait() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in wait 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup return hubs.get_hub().switch() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 187, in switch 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup return self.greenlet.switch() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 194, in main 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup result = function(*args, **kwargs) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/openstack/common/service.py", line 486, in run_service 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup service.start() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/service.py", line 103, in start 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup self.manager.init_host() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 308, in init_host 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup self.publish_service_capabilities(ctxt) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 1106, in publish_service_capabilities 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup self._report_driver_status(context) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 1095, in _report_driver_status 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup volume_stats = self.driver.get_volume_stats(refresh=True) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/nfs.py", line 340, in get_volume_stats 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup self._update_volume_stats() 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/nfs.py", line 359, in _update_volume_stats 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup capacity, free, used = self._get_capacity_info(share) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/nfs.py", line 567, in _get_capacity_info 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup '*snapshot*', mount_point, run_as_root=True) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 136, in execute 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup return processutils.execute(*cmd, **kwargs) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup File "/usr/lib/python2.6/site-packages/cinder/openstack/common/processutils.py", line 173, in execute 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup cmd=' '.join(cmd)) 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup ProcessExecutionError: Unexpected error while running command. 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf du -sb --apparent-size --exclude *snapshot* /var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup Exit code: 1 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup Stdout: '26\t/var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03\n' 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup Stderr: "/usr/bin/du: WARNING: Circular directory structure.\nThis almost certainly means that you have a corrupted file system.\nNOTIFY YOUR SYSTEM MANAGER.\nThe following directory is part of the cycle:\n `/var/lib/cinder/mnt/ade98462e07fef2a453d3328ce54ac03/.snapshot'\n\n" 2014-08-12 15:24:12.428 9485 TRACE cinder.openstack.common.threadgroup 2014-08-12 15:24:12.439 2662 INFO cinder.openstack.common.service [-] Child 9485 exited with status 0 2014-08-12 15:24:12.439 2662 INFO cinder.openstack.common.service [-] Forking too fast, sleeping Richard Minton Lockheed Martin - D&IS LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482