I've been using packstack to build an allinone controller and then add compute nodes. I do some modification of cinder backend/size, nova networking, openvswitch, neutron, vlans and interface NICs. I modify settings after packstack so I exclude previously configured nodes from being modified when I add a new compute node.
We develop an application that uses the openstack APIs and it is very useful to be stand up an allinone and a compute node the same way every time. It's more important for us to have multiple clouds than multiple nodes. Having 2 nodes means we can do some internode work that won't be on an allinone.
We can also use this setup for a customer POC to see our software run w/o spending lots on a real cloud. Some will only purchase on physical system so it's important that we don't require a 2nd system just to standup the cloud. Running off a DVD with no internet access in a customer lab means they see our software run.
I am interested in KVM on KVM. It would let me have more cloud controllers :-)
I've been able to stand up a slow allinone in virtualbox or vmware with 1 GB RAM, 1 CPU and 20 GB. I haven't been able to get a compute node to go with it. Probably because connecting the compute NICs on layer 2 isn't done quite right in the hypervisor.