[rdo-list] TripleO UI Packaging Strategy

[Mark McLoughlin]

On Tue, Jul 19, 2016 at 4:53 PM, Honza Pokorny <hpokorny at redhat.com> wrote:
> On 2016-07-19 16:29, Haïkel wrote:

>> Yes, though I'm not sure what you understand as build system.
>> Build system has no internet access, and except baseOS, only things
>> available are provided sources + dependencies declared as
>> BuildRequires (must be packaged)
>
> I guess this is the biggest problem: our build system is responsible for
> both fetching any dependencies and for actually building the project.
> Preferably, we'd like to run "npm install" from the RPM spec.  This step
> requires internet access, and therefore violates one of the rules. Since
> our dependencies are fixed/pinned, there is a fair degree of certainty
> that the dependencies will be the same each time the build is run[1].
> Npm install brings in sources, not artifacts.

It's really important that we all see the value of this "rule" and not
see it as some sort of hindrance imposed on us by a legacy view of
software development :)

The system is designed so that, in 18 months (or 10 years!) if we
isolate a bug to in one of the lower level dependencies in this
dependency stack, that we can fix that bug (think of something as
isolated as e.g. a missing i++ statement in a loop) and rebuild the
package with a high degree of confidence that the only thing that has
changed is the bugfix you made. And that all of this can happen really
quickly and predictably in a high-pressure crisis situation where the
bug is causing very serious, real-world problems.

If, the first time we built the package, 'npm install' in the build
system downloaded a source tarball from the internet, will it still be
there at that exact moment in 18 months time? Maybe not, so we have
the lookaside cache for tarballs. If 'npm install' downloads the
'latest' version of some dependency or tool, will it be a problem in
18 months time if there's a newer version now being used? Maybe, so
the dependencies and tools are installed from packages, and we can
reproduce an install of that exact set of packages later.

Maven is a build tool known for "downloading the internet", yet we
have managed to integrate it with our build system to a point that
"maven install" is perfectly aligned with these principles. See
https://fedoraproject.org/wiki/KojiMavenSupport

Distros like Fedora and Debian take this concept very seriously, and
for good reason. Distro developers are the type of people who yearn
for "reproducible build" to mean that in my example above, if you
rebuilt the package with the example i++ fix above, the *only* change
to the binary package would be the addition of that opcode. See
https://lwn.net/Articles/630074/ which I'll admit to finding
mind-blowing when I Lunar's FOSEM talk a couple of years ago:
https://archive.fosdem.org/2014/schedule/event/reproducibledebian/

Thanks,
Mark.