[Rdo-list] [OFI] data model reusing Foreman's AR models

[Scott Seago]

On 03/06/2014 11:54 AM, Petr Chalupa wrote:
> Hi guys,
>
> After yesterdays IRC talk we (Marek and Petr) started to look at OFI 
> models design. Unfortunately I haven't received any email with link to 
> that so we jumped in quite late. Anyway we've tried to decode as much 
> information as possible about staypuft model from etherpad [1]. Here's 
> ER diagram we were able to construct from those information.
>
> <see model A>
>
> Our main concern is that we are recreating models that are already 
> present in Foreman. We understand Astapor issue that it was very hard 
> for user understand all Foreman specifics so now we try to do it 
> Openstack way and use Foreman as provisioning and configuration 
> backend. However we still think that we should built on top of 
> existing and tested models and just create a new easy to use and 
> intuitive UI.
>
> inecas: +1 on reusing the Foreman model: the foreman model seems to 
> have most of the things to support what is required, building 
> something custom means:
>   * duplication of what is already there
>   * writing throw-away code, that nobody except the OFI itself will 
> use: the power of Foreman is in the community: building it on the 
> Foreman model (and enhancing it when needed) means the same work will 
> be used outside of OFI with all the benefits that come with it
> I would like to avoid doing that, especially when it doesn't seem that 
> much work to use the existing model
>
> So what we propose is following DB schema:
>
> <see model B>
>
> Each deployment is contained under an Organization for separation. 
> Which also means that same Foreman instance could be normally used 
> when user leaves the wizard.
>
> Left side is ER model right side is representing relationships between 
> Hostgroups instances. The one called 'Deploy' is lets say an common 
> parent Hostgroup for all other hostgroups in the deployment. 'Deploy' 
> allows sharing of configuration to the other hostgroups. Other 
> hostgroups are representing Roles (like controller, compute, etc.)
>
> Basically we'd map:
> OpenStackDeployment -> Organization
> OpenStackLayout -> organization's parameter
> OpenStackRole -> Hostgroup (*)
> OpenStackService -> N/A (at least for April)
> OpenStackSeviceParam -> LookupKey (aka Smart Variable)
> OpenStackDeployService -> N/A
> OpenStackDeployServiceParam -> LookupKey (aka Smart Variable Value)
> OpenStackDeployRole -> assignment of Host to Hostgroup (*)
>
> At the moment, the only thing that's missing is that Foreman 1.4.1 
> does not allow us to set one hostgroup to be used in two parent 
> hostgroups (one service in two roles). This is going to change in 1.5 
> when ConfigRoles will be merged. We suppose to use quickstack modules 
> for April version so we still have just puppet classes on role level 
> (not on service level) and therefore we won't allow user to customize 
> services in roles. Since we need this post April we can stick with 
> current Foreman model. Later we could either back-port ConfigRoles or 
> base on newer Foreman.
>
> Benefits we see by reusing foreman models
> + we save work on need to model everything again
> + we save work on writing logic that will copy all 
> (Deploy)Service/Role (Deploy)ServiceParameters values to 
> Hostgroup/LookupKey/LookupValue
> + ability to turn off the wizard plugin (something like removing the 
> training wheels) and keeping the Foreman instance still usable
>
> Cons
> - people may spend some time to better understand existing Foreman code
> inecas: that should be under Pros, and by people I don't mean the 
> actual users of OFI, but its developers
>
> Potential concerns we understood from yesterday
> 1. too foreman specific
> 2. need of doing some business logic before Deploy is triggered
> 3. writing some easy UI around existing models is bigger pain that 
> creating new objects from scratch
> 4. to able to delay host provisioning until whole deployment is 
> configured
>
> Our answers to those
> 1. this is just about building new UI that we'll need anyway and 
> re-labeling (we already have Smart Variable which in fact is LookupKey 
> model), we could use STI or composition to modify Foreman objects 
> behavior if they are too restricting
> 2. It's easy to extend Dynflow process with a plugin injecting any 
> additional steps into the deployment process.
> 3. if this really becomes a problem we could always wrap existing 
> models by some other classes
> 4. If the hosts are kept in unmanaged state nothing gets applied to 
> them. Until orchestration takes over and switches them to managed state.
>
> Are we sure that we need to allow user to prepare changes to an 
> existing-deployment-configuration and then apply them all at the same 
> time in April? I think we can postpone this for later and keep 
> Foreman's behavior for now that changes made to a hostgroup are 
> immediately applied.
>
> [1]: http://etherpad.corp.redhat.com/staypuft-model-design-notes
>
> Looking forward to your feedback Petr and Marek.
>
Thanks for the writeup. I had a few comments/concerns around some things 
that  this proposal may not address as-is, but perhaps can be tweaked to 
cover it. Also, if we do adopt this approach, I wonder if we'll need to 
do so with perhaps some additional plugin models to handle some of the 
bits that may not be describable with only foreman object types.

Anyway, here are the concerns -- let me know what you think:

1) I'm not seeing any distinction between default objs (i.e. 
OpenStackRole) and
    per-deployment values (OpenStackDeployRole) -- we'd need to solve this
    before allowing multiple deployments. In my model the initial set of 
default
    roles and values
2) What are the consequences of no services in April? This also means
    we will have no per-service params, so there are UI implications
    (i.e. instead of breaking down params by service in each role, we
    have one list of per-role settings)
3) Will LookupKey allow us to set other metadata (ui widget type,
    etc -- or would we need to extend this model with a shadow table type
    model -- i.e. OpenStackLookupKey with a 1-to-1 relationship with the
    foreman one to include extra elements we need for UI generation.
4) The new suggestion layout seems to have some issues:
    OrganizationParameter makes it seem like it belongs to an
    organization. Layout is a general model/object, which exists from
    seed data load before the first org is created. It's mapped to a
    list of OpenStackRoles/HostGroups in seeds.rb, so
    a) it needs to be able to be created before an Organization (and be
       associated with more than one post-April)
    b) we need to be able to associate Layouts/OrganizationParameters
       with OpenStackRoles/HostGroups in a many-to-many manner in seeds.rb

Scott