On 05/11/2017 12:25, Philipp Kewisch wrote:
<pre class="moz-quote-pre" wrap="">In general you need to allow any measure that allows effective
sheriffing. If I have a test run with 100 failures, whacking a star
onto them doesn't help detecting the 101st and 102nd additional
failures. Surely we can live with one or two failing tests for a
while. That is the case anyway when the failure needs to be fixed in M-C.
<pre class="moz-quote-pre" wrap="">I agree with that, but I'd like us to be a bit more creative as to what
these measures are.
Well, I see three measures:
- Disable the test. I've done this on a few occasions. When landing the real fix, the test gets enabled again.
- Land a band-aid: I've done this on a few occasions. When landing the real fix, the band-aid gets backed out.
- Land the real fix, these days with [PLR]. This is the most common case where C++ parameters have changed, or function names have changed and there are usually heaps of examples in the M-C of how the code needs to change (https://mzl.la/2iy3rx1).
1. and 2. are marked with [Thunderbird-temporary-fix] (https://mzl.la/2iwY3dH). There is also [Thunderbird-disabled-test] (https://mzl.la/2vZEjFe) where I have disabled tests because the have become perma-red on some platform and the failure doesn't reflect a functional failure but rather a defective test.
Currently we have two band-aids in place, most severely (pref("jsloader.shareGlobal", false);) in https://bugzilla.mozilla.org/show_bug.cgi?id=1401528 since TB would be totally obliterated without it. Sadly there isn't much interest in that bug from the peers, so that band-aid might go to beta soon.
In the past, four temporary fixes in categories 1. and 2. were used. Those bugs are resolved and the temporary fix was backed out: https://mzl.la/2ixEr9o.
Jörg.
