[#QB-490] Support JUnit reports containing several executions of the same JUnit test (cf. compatibility tests)

QuickBuild

Support JUnit reports containing several executions of the same JUnit test (cf. compatibility tests)

Created: 21/Dec/09 03:38 PM Updated: 12/Jan/10 02:22 PM

Component/s:

None

Affects Version/s:

None

Fix Version/s:

None

Original Estimate:

Unknown

Remaining Estimate:

Unknown

Time Spent:

Unknown

File Attachments:

None

Image Attachments:

1. screenshot-1.jpg
(240 kb)

2. screenshot-2.jpg
(304 kb)

Environment:

PMEase QuickBuild, 2.0.12, 2009-12-17

Description

« Hide

See attached screenshots: We have a couple of critical tests (in this scenario just two) that must be executed with different versions of another component to check the compatibility. That is, one and the same test is executed several times. By specifying different report sub-folders the Ant JUnit HTML report task is able to more or less generate a useful report (note that the specialization in parentheses is due to an adaption of the default XSLT stylesheet).

Minimalistic requirement:
* Detect all executions of the same test (i.e. 44 instead of 2 tests in this scenario).

I don't dare to ask for a possibility to flag the tests e.g. via a System Property as we do it. Thus, we don't mind to nonetheless generate the default Ant JUnit HTML report with our XSLT adaptions, but currently the notifications and build summaries are quite confusing or rather misleading.

Note that in an aggregate Ant JUnit XML report there seems to be a unique id attribute for each test suite element:
<testsuite errors="0" failures="0" hostname="lab13" id="1" name="ClassCacheTest" package="com.dynatrace.diagnostics.integrationtest.instrumentation" tests="1" time="43.066" timestamp="2009-12-21T14:00:39">

All

Comments

Work Log

Change History

Sort Order:

Change by AlSt [21/Dec/09 03:38 PM]
Field	Original Value	New Value
Attachment		screenshot-1.jpg [ 10129 ]

[ Permlink | « Hide ]

AlSt [21/Dec/09 03:39 PM]

I hope I have not missed some setting or approach/workaround in the current JUnit plug-in...

[ Permlink | « Hide ]

AlSt [21/Dec/09 03:47 PM]

In a naive attempt I tried to configure several JUnit report steps that e.g. have one sub-folder via the "files to process" attribute (e.g. "1/**/*.xml"), but it does not seem to be helpful.

Change by AlSt [21/Dec/09 03:47 PM]
Attachment		screenshot-2.jpg [ 10130 ]

Change by Robin Shen [22/Dec/09 12:19 AM]
Assignee	Robin Shine [ robinshine ]	Steve Luo [ steve ]

Change by AlSt [11/Jan/10 10:58 AM]
Attachment		Email.htm [ 10133 ]

Change by Steve Luo [12/Jan/10 09:07 AM]
Attachment	Email.htm [ 10133 ]

Change by Steve Luo [12/Jan/10 02:19 PM]
Status	Open [ 1 ]	Resolved [ 5 ]
Fixed In Version		2.0.14 [ 11112 ]
Resolution		Fixed [ 1 ]

Change by Steve Luo [12/Jan/10 02:21 PM]
Comment	[ For this case, the user can transform the original junit xml report by using xslt before publish the report. For example, we have below report: <testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48"> <properties> ...... <property name="java.vm.version" value="14.0-b16" /> ...... </properties> <testcase classname="com.pmease.quickbuild.FooTest" name="testBar" time="210.129"/> </testsuite> then, after transform the report, we will publish below report: <testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48"> <properties> ...... <property name="java.vm.version" value="14.0-b16" /> ...... </properties> <testcase classname="com.dynatrace.diagnostics.communication.tcp.connection.ConnectionManagerTest" name="testBar (jvm-14.0-b16)" time="210.129"/> </testsuite> ]

[ Permlink | « Hide ]

Steve Luo [12/Jan/10 02:22 PM]

For this case, the user can transform the original junit xml report by using xslt before publish the report. For example, we have below report:
<testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48">
<properties>
......
<property name="java.vm.version" value="14.0-b16" />
......
</properties>

<testcase
  classname="com.pmease.quickbuild.FooTest"
  name="testBar"
  time="210.129"/>
</testsuite>

then, after transform the report, we will publish below report:
<testsuite errors="0" failures="38" hostname="thebiggerone" name="com.pmease.quickbuild.FooTest" tests="1" time="210.297" timestamp="2009-11-26T12:40:48">
<properties>
......
<property name="java.vm.version" value="14.0-b16" />
......
</properties>

<testcase
  classname="com.pmease.quickbuild.FooTest"
  name="testBar (jvm-14.0-b16)"
  time="210.129"/>
</testsuite>