[Rdo-list] [QA] Tempest - is volume testing actually testing anything with block storage?

[Giulio Fidente]

On 08/11/2014 10:00 PM, Kaul, Yaniv wrote:
> (Some of you may know me from my previous work at Red Hat - good to see some familiar faces!)

yep, hi there :)

> I'm working on testing a Cinder driver for OpenStack - IceHouse, Havana, and Juno (regretfully, in that order).
> I've quickly found out Tempest is not really testing much for real (example: if I don't configure iSCSI, I still pass all tests but 5 of them!), so I'm doing some 'manual' tests.

Indeed, only Nova nodes connect to the volumes via iSCSI (and only for 
certain Cinder backends), not the Cinder nodes so a majority of the 
tests for Cinder succeed regardless of the iSCSI setup

> 1. Is there any test actually booting or running the VMs from the block storage? I have it configured correctly (I think), but none of the relevant tempest.api.volume* tests actually do much with it. Volumes are created, mapped, snapshotted, removed, etc, but nothing is really written to it...

tempest.api.volume tests are testing Cinder API only

there are tests stressing the Nova Volume API (eg. attach) in:

tempest.api.compute.volumes

Some of these would require proper iSCSI setup for instance.

There are tests doing actual writes into the volumes too (eg. boot from 
volume, snapshot, clone, reboot) in:

tempest/scenario/test_stamp_pattern.py

and

tempest/scenario/test_volume_boot_pattern.py

and

tempest/scenario/test_snapshot_pattern.py

lastly, with Tempest you can also run any of the API test (eg 
tempest.api.*) in a loop with some parallelization and this is handful 
to test a driver as operations may hang if the backend is not behaving 
as expected

> 2. Is there a way to test multi-backend for real? Looks like in Tempest there's a single 'storage_protocol' entry?

not that I know of
-- 
Giulio Fidente
GPG KEY: 08D733BA