Package-only for release/N.x branches?

[alexreinking]

The fully-merged release branch builds do a significant amount of redundant work with:

1. The PRs that got merged to trigger the update to the branch in the first place (the code is identical)
2. Internally with the build and package steps

I propose only running package steps and then testing the apps on the package directly. If there are issues, we can fail. If the tests in test/{correctness,generator,...} were fully externalized, those could be run against the actual package, too.

On the other hand, maybe this is orthogonal and we should remove the package steps from release-branch-builders and run them on-demand from the buildbot website when we're ready to cut a release.

Basically, I want to make sure our buildbots are doing useful work when they test master, release/N.x, etc. and that it gets us some value beyond the PR testers.

[steven-johnson]

SGTM. TBH, though, if all this is only happening when we do release backports (ie, rarely), how much of a problem is it?

[alexreinking]

It's a bit more of a problem because that's exactly when latency is the most annoying. The backport gets merged and we might fairly quickly want to issue a patch release.