[Rdo-list] Glance does not save properties of images

Thu Feb 6 02:04:52 UTC 2014

Hello everyone,

Thanks for you help with that problem.

I was able to resolve it by examining all headers of requests and responses.
It turns out that the issue is not entirely Glance's fault and definitely not a problem of a package.

The properties that I was missing were passed through headers with _ (underscore) in their names:
x-image-meta-container_format: bare
x-image-meta-disk_format: qcow2
x-image-meta-is_public: True

These were the only ones that got lost.

I also incorrectly assumed that Glance would not accept a request to create an image without required parameters disk-format and container-format. It does accept it, this property is only enforced in Horizon. CLI also doesn't care if you pass them or not.

The ultimate root of the problem was the fact that I use nginx in front of the controller as SSL terminating proxy. Nginx is smart enough to reject headers with underscore in their name as invalid HTTP headers. This way the headers mentioned above were stripped and the rest went through. Too bad that nginx is so secret about doing so. There is a simple fix in case anyone has the same problem: http://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers

I submitted a bug report for Glance on that: https://bugs.launchpad.net/glance/+bug/1276887

Sorry for all the trouble related to debugging the issue and thanks again!

Best Regards,
Daniel Speichert

----- Original Message -----
> From: "Flavio Percoco" <flavio at redhat.com>
> To: "Daniel Speichert" <daniel at speichert.pl>
> Cc: "Tzach Shefi" <tshefi at redhat.com>, rdo-list at redhat.com
> Sent: Monday, February 3, 2014 2:46:17 AM
> Subject: Re: [Rdo-list] Glance does not save properties of images
> 
> On 03/02/14 05:59 +0100, Daniel Speichert wrote:
> >Yes, I tested that with an admin user. The bigger problem that
> >is_public is that required disk_format and container_format are
> >lost too.
> 
> This sounds like a bigger problem, though.
> 
> What do you mean with `disk_format` and `container_format` are lost?
> for what command? Could you please share the exact command you're
> running?
> 
> Notice that some of these parameters are available at creation time
> but not as part of updates. That said, I recall you said it used to
> work and stopped working after you updated / reinstalled your system.
> 
> Thanks!
> flaper
> 
> >
> >Regards,
> >Daniel Speichert
> >
> >----- Original Message -----
> >> From: "Tzach Shefi" <tshefi at redhat.com>
> >> To: "Daniel Speichert" <daniel at speichert.pl>
> >> Cc: "Flavio Percoco" <flavio at redhat.com>, rdo-list at redhat.com
> >> Sent: Sunday, February 2, 2014 10:43:49 AM
> >> Subject: Re: [Rdo-list] Glance does not save properties of images
> >>
> >> Unfortunately don't have other ideas, I'll try a setup with Rabbit
> >> just to make sure this isn't the cause.
> >>
> >> A long shoot but i'll ask any way the user that set is_public
> >> param,
> >> has admin permissions right?
> >> http://docs.openstack.org/developer/glance/glanceapi.html
> >>
> >> "Use of the is_public parameter is restricted to admin users. For
> >> all
> >> other users it will be ignored."
> >>
> >> Regards,
> >> Tzach
> >>
> >>
> >> ----- Original Message -----
> >> From: "Daniel Speichert" <daniel at speichert.pl>
> >> To: "Tzach Shefi" <tshefi at redhat.com>
> >> Cc: "Flavio Percoco" <flavio at redhat.com>, rdo-list at redhat.com
> >> Sent: Thursday, January 30, 2014 6:01:26 PM
> >> Subject: Re: [Rdo-list] Glance does not save properties of images
> >>
> >> Yes, Ceph is working well as backend (images are uploaded there
> >> successfully and retrieved as well).
> >> The same problem occurs also when the image is kept locally
> >> (default_store = file).
> >>
> >> We have checked multiple images and the problem is the same.
> >> Interestingly, we had the same setup before (we reinstalled last
> >> week) and it was working fine. That makes me think that something
> >> might be different in the package.
> >>
> >> We use rabbit and it seems to be working fine - ceilometer is
> >> getting
> >> these notifications.
> >>
> >> Do you have any other ideaas? I'm running the latest package
> >> version
> >> form RDO.
> >>
> >> Thanks,
> >> Daniel Speichert
> >>
> >> ----- Original Message -----
> >> > From: "Tzach Shefi" <tshefi at redhat.com>
> >> > To: "Daniel Speichert" <daniel at speichert.pl>
> >> > Cc: "Flavio Percoco" <flavio at redhat.com>, rdo-list at redhat.com
> >> > Sent: Thursday, January 30, 2014 7:22:58 AM
> >> > Subject: Re: [Rdo-list] Glance does not save properties of
> >> > images
> >> >
> >> > Hello Daniel,
> >> >
> >> > On your posted glance-api.con noticed default_store = rbd,
> >> > assuming
> >> > Glance back end is CEPH (correct?).
> >> >
> >> > Just installed AIO RDO Havana on RHEL 6.5.
> >> > Is_public updated successfully via Horizon\CLI, image shows up
> >> > as
> >> > public on Horizon\image-show.
> >> >
> >> >
> >> > I'm assuming (the obvious..) that you checked this on more than
> >> > just
> >> > that one image, eliminates a single bad image as source of this
> >> > problem.
> >> > BTW what type\source of image was used\checked?
> >> >
> >> > Another difference in setups mine notifier_strategy =QPID your's
> >> > uses
> >> > rabbit.
> >> >
> >> > Regards,
> >> > Tzach
> >> >
> >> > ----- Original Message -----
> >> > From: "Daniel Speichert" <daniel at speichert.pl>
> >> > To: "Flavio Percoco" <flavio at redhat.com>
> >> > Cc: rdo-list at redhat.com
> >> > Sent: Wednesday, January 29, 2014 4:30:56 PM
> >> > Subject: [Rdo-list] Glance does not save properties of images
> >> >
> >> > ----- Original Message -----
> >> > > From: "Flavio Percoco" <flavio at redhat.com>
> >> > > To: "Daniel Speichert" <daniel at speichert.pl>
> >> > > Cc: "Lars Kellogg-Stedman" <lars at redhat.com>,
> >> > > rdo-list at redhat.com
> >> > > Sent: Tuesday, January 28, 2014 3:37:27 AM
> >> > > Subject: Re: [Rdo-list] Glance does not save properties of
> >> > > images
> >> > >
> >> > >
> >> > > Could you please share the commands your using?
> >> > >
> >> > >     - How are you adding the properties?
> >> > >     - How are you reading the propoerties?
> >> > >
> >> > > You're sharing your registry.log configurations which means
> >> > > you've
> >> > > configured the registry. Could you share how you did that?
> >> > >
> >> > > Thanks,
> >> > > flaper
> >> > >
> >> >
> >> > These are required properties when adding any Glance image
> >> > (disk_format and container_format). I was trying to set
> >> > is_public
> >> > through 'glance image-edit' and Horizon. The change seems to be
> >> > accepted (no error) but on subsequent image-show it's just not
> >> > there.
> >> >
> >> > Registry and API are set up almost by defaults:
> >> > glance-registry.conf: http://pastebin.com/M7Ybjnf9
> >> > glance-api.conf: http://pastebin.com/5sPXr3mG
> >> >
> >> > Regards,
> >> > Daniel Speichert
> >> >
> >> > _______________________________________________
> >> > Rdo-list mailing list
> >> > Rdo-list at redhat.com
> >> > https://www.redhat.com/mailman/listinfo/rdo-list
> >> >
> >>
> 
> --
> @flaper87
> Flavio Percoco
> 