Re: [rdo-list] TripleO UI Packaging Strategy

On Fri, Jul 15, 2016 at 6:29 PM, Haïkel <hguemar(a)fedoraproject.org&gt; wrote: What can be interesting in situations like this is to attempt to explain what's an absolute requirement versus where exceptions might be acceptable. For everyone's reference, RDO's packaging guidelines are here: https://www.rdoproject.org/documentation/rdo-packaging-guidelines/ and there is a brief mention of RDO allowing for exceptions to Fedora's strict unbundling guidelines: https://fedoraproject.org/wiki/Bundled_Libraries Some more reading on Fedora's unbundling policy: https://lwn.net/Articles/660429/ Anyway, taking a step back beyond unbundling for one minute, we should always require that packages are built in an entirely reproducible way from source so that, for example, we always have the ability to rebuilt the package with a minor fix and know that there won't be side-effects other than the change we intended to make. Some of the implications of that: * The toolchain you use to build the package must also follow this basic rule - because fixing an issue might entail fixing the toolchain and rebuilding the package * Everything[1] must be rebuilt from source that is uploaded to the build system - we don't use pre-built artifacts supplied by upstream. * It should be possible to review (e.g. from a security POV) all of the source used to build the final package, and know that everything was built from that source code. If you can't imagine us actually doing this detailed a review, then think of it as us requiring the ability to do a spot-check review. * The buildsystem should not have access to the internet during a build, helping to ensure that the build process isn't using source files from outside of the buildsystem. Many of Fedora's packaging guidelines are very useful to achieve consistency for the user, like those around how packages are named or versioned, or where files should be installed. Other guidelines enforce a policies like Fedora should be built "exclusively from Free and Open Source software", and to ensure the wishes of the original copyright holders are respected. Back to unbundling ... as Haikel says, this is the one area where a relaxing of the Fedora packaging guidelines might make sense in this case, particularly for expediency. Having each dependency individually packaged has a number of benefits: * The correlation between package version and upstream dependency version is clear * It's easier to track when a a new upstream version is available * It's easier to identify whether a given package is affected by a published security issue * Updates for users are smaller - they may only have to download and update a small individual package rather than a large package containing everything * Package maintenance can be shared with others - if multiple applications all use the same dependency, it's better if they don't each package that dependency * On-disk and in-memory space can be reduced if multiple applications share the same dependency However, many of these benefits are probably quite small in the case of TripleO UI compared to the likely overhead of individually packaging everyting, and compared to the benefit of including TripleO UI in the distribution. The exception to this argument that individual packaging is of little benefit is that AIUI we are moving away from openstack-puppet-modules being one giant package with many puppet modules bundled, towards individual packages? The reason for this is that the maintenance overhead with a single giant o-p-m package was actually greater than using automation to maintain the individual packages? Is this likely to also be true in the TripleO UI case? I guess I don't even have a basic understanding of what's going on here - for example, it looks like Horizon bundles the Angular framework in its upstream git and tarball, but TripleO UI doesn't do anything like that with React? The starting point for the thread was that the package users install would just contain the minified version of the javascript ... which I guess would be fine so long as the minification happened in the buildsystem, using only "source" artifacts or artifacts built from source. Hope that helps, Mark. [1] - there are interesting nuances to this - e.g. for C projects, the upstream maintainer will usually include configure.in, Makefile.in, etc. files in their tarballs. These are likely generated with a different version of autoconf and automake that are in the build system. If, in future, we had to modify configure.ac and Makefile.am, and then regenerate configure.in and Makefile.in using autoconf and automake, we run the risk of introducing changes caused by differences in the autotools versions. AIUI, this is still an unresolved debate in Fedora, and left to discretion of packagers: https://fedoraproject.org/wiki/PackagingDrafts/AutoConf