Re: Reorganizing the squid builds

On Mon, Aug 19, 2013 at 4:44 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 20/08/2013 12:56 a.m., Kinkie wrote:
>>
>> Hi all,
>> Now that the matrix jobs feature has matured in jenkins, I'd like to
>> restructure the jobs we use for our continuous integration testing to
>> reduce their number.
>>
>> I propose to use these jobs:
>> - X.Y-matrix -> gcc-only (for now), build a specific revision
>> - trunk-compilers-matrix -> uses a combination filter to test on any
>> unix-like platform using any available compiler (gcc, clang, icc, msvc
>> to come as soon as the windows port reaches stability)
>> - anybranch-wholefarm-matrix -> same as above, but with some
>> parameters (a branch to be used, whether to use ccache or not, and an
>> address to be notified).
>>
>> Ideally trunk-compilers-matrix should replace most 3.HEAD-* jobs,
>> while anybranch-wholefarm-matrix can replace most experimental jobs
>> (after all, a patch is just a push to lp away). The only nonmatrix
>> jobs that would remain are
>> - coverity-submit
>> - mswin
>> - possibly some support jobs
>>
>> What do you think?
>> I plan to go ahead on Aug 25th, unless there are no objections.
>>
>> Thanks!
>
>
> We have several problems with the existing matrix jobs before it makes much
> sense to use them this widely:
> * at the top level it appears that the stable releases are failing to build
> (red icon) if any one of the test nodes fail.
> * there is no soft-fail type indication when a failure was with Jenkins
> infrastructure.
>
> Have the matrix support advanced far enough to resolve those issues for us
> yet?

Not really.
Some advance was made with the Elastic Axis plugin (skip over
disconnected nodes) but that plugin can't be used due to other bugs.

> They currently provide a false impression of instability and prevents us
> testing all stable versions against all nodes simply because the newer nodes
> will be for OS which the stable was never coded to build on properly (ie 3.1
> wont build on FreeBSD 9/10 after compiler changes). It would be good to have
> such a picture of the versions buildability but I hope we can avoid giving
> the impression that none of our stables ever work.
>
> I think it may be worth a try, but not removing the existing jobs until the
> new ones have a proven record of usefulness.

How about dropping only relatively rare configurations? e.g. icc,
ALPHA-PATCH, old FreeBSD variants? That'd already go a long way
towards simplification without losing sight of the big picture.

-- 
    /kinkie