Fix testcase class results timings on build jenkins
As reported by , the Duration for testcase class results is being reported incorrectly on Jenkins.
look at the test results for an edx-platform-python-unittests-master build. e.g. https://build.testeng.edx.org/view/edx-platform-master-tests/job/edx-platform-python-unittests-master/7999/testReport/
Now drill into some results, eg. pavelib.paver_tests.test_assets
BUG: the durations of all the classes listed is the same, and incorrect. In this case it says 12 minutes.
You can see that the test results file from which the timings are derived is correct. In this case it's in artifact/reports/pavelib/paver_tests/shard_1/nosetests.xml, whose contents include:
It looks like this is a bug in the JUnit plugin for jenkins, and was introduced in version junit-1.12 (April 2016).
Note that we recently (Nov 7th, about a month ago) upgraded the version of this plugin from 1.3 to 1.21. See https://github.com/edx/configuration/pull/4198/files
This PR introduced the bug, seemingly because the author assumed that a testcase suite would only have a single testcase class in it. https://github.com/jenkinsci/junit-plugin/pull/35
This (open) PR purports to fix the issue: https://github.com/jenkinsci/junit-plugin/pull/66
A few possible options:
Revert the plugin upgrade
Champion the fix getting merged in
Merge the fix into a fork (not sure what version is the highest it will merge cleanly into) and run with that
Steps to Reproduce
Reason for Variance
User Impact Summary
This was fixed with an upgrade of the junit plugin on 2019-09-13.