| << Back to previous view |
|
[QB-490] Support JUnit reports containing several executions of the same JUnit test (cf. compatibility tests)
|
|
| Status: | Resolved |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major |
| Reporter: | AlSt | Assigned To: | Steve Luo |
| Resolution: | Fixed | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Environment: | PMEase QuickBuild, 2.0.12, 2009-12-17 | ||
| File Attachments: |
screenshot-1.jpg
screenshot-2.jpg
|
| Description |
|
See attached screenshots: We have a couple of critical tests (in this scenario just two) that must be executed with different versions of another component to check the compatibility. That is, one and the same test is executed several times. By specifying different report sub-folders the Ant JUnit HTML report task is able to more or less generate a useful report (note that the specialization in parentheses is due to an adaption of the default XSLT stylesheet).
Minimalistic requirement: * Detect all executions of the same test (i.e. 44 instead of 2 tests in this scenario). I don't dare to ask for a possibility to flag the tests e.g. via a System Property as we do it. Thus, we don't mind to nonetheless generate the default Ant JUnit HTML report with our XSLT adaptions, but currently the notifications and build summaries are quite confusing or rather misleading. Note that in an aggregate Ant JUnit XML report there seems to be a unique id attribute for each test suite element: <testsuite errors="0" failures="0" hostname="lab13" id="1" name="ClassCacheTest" package="com.dynatrace.diagnostics.integrationtest.instrumentation" tests="1" time="43.066" timestamp="2009-12-21T14:00:39"> |
| Comments |
| Comment by AlSt [ 21/Dec/09 03:39 PM ] |
| I hope I have not missed some setting or approach/workaround in the current JUnit plug-in... |
| Comment by AlSt [ 21/Dec/09 03:47 PM ] |
| In a naive attempt I tried to configure several JUnit report steps that e.g. have one sub-folder via the "files to process" attribute (e.g. "1/**/*.xml"), but it does not seem to be helpful. |
| Comment by Steve Luo [ 12/Jan/10 02:22 PM ] |
|
For this case, the user can transform the original junit xml report by using xslt before publish the report. For example, we have below report:
<testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48"> <properties> ...... <property name="java.vm.version" value="14.0-b16" /> ...... </properties> <testcase classname="com.pmease.quickbuild.FooTest" name="testBar" time="210.129"/> </testsuite> then, after transform the report, we will publish below report: <testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48"> <properties> ...... <property name="java.vm.version" value="14.0-b16" /> ...... </properties> <testcase classname="com.pmease.quickbuild.FooTest" name="testBar (jvm-14.0-b16)" time="210.129"/> </testsuite> |