| << Back to previous view |
|
[QB-2031] Running out of heap space while parsing big Junit report
|
|
| Status: | Resolved |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 5.1.23 |
| Type: | Bug | Priority: | Major |
| Reporter: | Maikel vd Hurk | Assigned To: | Steve Luo |
| Resolution: | Fixed | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Comments |
| Comment by Maikel vd Hurk [ 23/Apr/14 08:58 AM ] |
|
Hitted by accident enter, but the problem is that nodes are reporting:
{noformat} java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at com.pmease.quickbuild.bootstrap.BootstrapUtils.wrapAsUnchecked(BootstrapUtils.java:58) at com.pmease.quickbuild.util.ExceptionUtils.wrapAsUnchecked(ExceptionUtils.java:82) at com.pmease.quickbuild.stepsupport.Step.execute(Step.java:549) at com.pmease.quickbuild.stepsupport.StepExecutionJob.executeStepAwareJob(StepExecutionJob.java:30) at com.pmease.quickbuild.stepsupport.StepAwareJob.executeBuildAwareJob(StepAwareJob.java:45) at com.pmease.quickbuild.BuildAwareJob.execute(BuildAwareJob.java:60) at com.pmease.quickbuild.grid.GridJob.run(GridJob.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuffer.append(StringBuffer.java:224) at org.dom4j.io.SAXContentHandler.characters(SAXContentHandler.java:293) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.characters(AbstractSAXParser.java:538) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:489) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522) at org.dom4j.io.SAXReader.read(SAXReader.java:465) at org.dom4j.io.SAXReader.read(SAXReader.java:343) at com.pmease.quickbuild.plugin.report.engine.util.XMLHelper.readXML(XMLHelper.java:213) at com.pmease.quickbuild.plugin.report.engine.util.XMLHelper.readXML(XMLHelper.java:232) at com.pmease.quickbuild.plugin.report.junit.TestCaseProcessor.processResource(TestCaseProcessor.java:30) at com.pmease.quickbuild.plugin.report.engine.generator.AbstractProcessor.doExecute(AbstractProcessor.java:81) at com.pmease.quickbuild.plugin.report.engine.generator.GenerateAction.execute(GenerateAction.java:42) at com.pmease.quickbuild.plugin.report.engine.generator.AbstractGenerator.doExecute(AbstractGenerator.java:43) at com.pmease.quickbuild.plugin.report.engine.generator.GenerateAction.execute(GenerateAction.java:42) at com.pmease.quickbuild.plugin.report.engine.generator.DefaultGenerator.generate(DefaultGenerator.java:37) at com.pmease.quickbuild.plugin.report.engine.contribution.publisher.AbstractPublisher.doPublish(AbstractPublisher.java:162) at com.pmease.quickbuild.plugin.report.junit.JUnitPublisher$$EnhancerByCGLIB$$7095404f.CGLIB$doPublish$38(<generated>) at com.pmease.quickbuild.plugin.report.junit.JUnitPublisher$$EnhancerByCGLIB$$7095404f$$FastClassByCGLIB$$ca260f45.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215) at com.pmease.quickbuild.DefaultScriptEngine$Interpolator.intercept(DefaultScriptEngine.java:273) at com.pmease.quickbuild.plugin.report.junit.JUnitPublisher$$EnhancerByCGLIB$$7095404f.doPublish(<generated>) at com.pmease.quickbuild.plugin.report.engine.contribution.publisher.AbstractPublisher.run(AbstractPublisher.java:129) at com.pmease.quickbuild.plugin.report.junit.JUnitPublisher$$EnhancerByCGLIB$$7095404f.CGLIB$run$31(<generated>) at com.pmease.quickbuild.plugin.report.junit.JUnitPublisher$$EnhancerByCGLIB$$7095404f$$FastClassByCGLIB$$ca260f45.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215) {noformat} This is if the file is > 88MB. This could be solved by not loading everything in memory as only relevant data is the result data, not the 'system-out' and 'system-err' attribute which can be rather big, which is possible with the SAX parser which is used. |
| Comment by Steve Luo [ 24/Apr/14 03:16 AM ] |
|
Please edit wrapper.conf under ${quickbuild-home}/conf/ to increase your JVM memory like below:
wrapper.java.maxmemory=512 |
| Comment by Maikel vd Hurk [ 24/Apr/14 05:50 AM ] |
| Steve I know that this is possible solution, but in case there will be more system out / system err logging and file is growing I keep on increasing this maxmemory value. There is no added value in reading in the system attributes as for the JUnit report you require only the result data right? |
| Comment by Maikel vd Hurk [ 24/Apr/14 06:21 AM ] |
|
BTW maxmemory setting is already on 1024 and that is still not sufficient.
Maybe good to notice it is worth to look into: https://github.com/whummer/scaleDOM which has small memory footprint due to lazy loading of XML nodes. It only keeps a portion of the XML document in memory and re-loads nodes from the source file when necessary. |
| Comment by Steve Luo [ 24/Apr/14 07:18 AM ] |
|
Hi Maikel,
Thank you very much for the suggestion. But to change to another streaming API parser (like XPP) to parse the huge XML document may impact too many things in current report system. We can't do this in a short term as in our system, the report parser (not only for JUnit, but also for all other reports) are greatly relied on XPath which need loading the whole document into the system. Back to your case, I want to know what is your builder, Ant or Maven? And is it possible for you to publish the JUnit reports for each testsuite instead of an aggregated report, so we can reduce the XML report size. If you are using Maven, the surefire reports is what I mean. But if you are using Ant, you can try publish the test case XML reports instead of the aggregated one. That is, you needn't add JUnitReport task to aggregate the individual test suite report. |
| Comment by Maikel vd Hurk [ 24/Apr/14 07:39 AM ] |
| It is based on ant junit report, and it is XML of single testsuite. We have quite a large amount of logging which is being exposed in system-out/system-err always that is why the XML is that big. |
| Comment by Steve Luo [ 24/Apr/14 09:16 AM ] |
|
You can specify option: outputtoformatters="false" to ignore sending the output to XML file, see below page:
http://ant.apache.org/manual/Tasks/junit.html |
| Comment by Steve Luo [ 24/Apr/14 09:21 AM ] |
| Note, this option for junit task is added since Ant 1.7.0 |
| Comment by Maikel vd Hurk [ 24/Apr/14 09:22 AM ] |
| I know about that option, but we would like to have all information in single files available. There is no way that you don't load whole XML in memory? According to stacktrace you are using SAXParser which is able to load not all elements in memory when opening file in contradiction to dom. |
| Comment by Steve Luo [ 24/Apr/14 09:46 AM ] |
|
Even the system-out and system-err is in the XML, QuickBuild can't store them per issue Sorry, I have no any idea now. Yes, we are using dom4j SAXReader, but as aforementioned, we are using XPath and it requires to construct the document tree in the memory so the whole document will be loaded to the memory. I haven't found any solution to do this, if you know, please let me know. |
| Comment by Steve Luo [ 24/Apr/14 12:07 PM ] |
| Never mind, I think I found the solution now and will get it fixed in next patch release. |
| Comment by Steve Luo [ 24/Apr/14 04:37 PM ] |
|
Sorry, I have to update a bit more.
The memory needed in my solution depends on the max size of system-out or system-err element (in your case, these two elements may contain large of data) in your report. For example, if you have a JUnit report XML file with 1G size, but each system-out or system-err is small, say, 20M, then QuickBuild may only need about 512M memory. While although your report file is only 100M, there is only one system-out or system-err which contains 90M data, QuickBuild may need about 1G memory to process it. Now, the main issue is not caused by QuickBuild, instead, it is more related to how JDK processing String data in my opinion. A small test for this issue is below: StringBuffer sb = new StringBuffer(); for (int i = 0; i < 1024 * 1024 * 128; i++) sb.append("a"); here, we want to create a 128M data in memory, you will find that you may need about 512M memory or even more. Anyway, I'll include my solution in next patch release and hope that can handle most of your case. |
| Comment by Maikel vd Hurk [ 25/Apr/14 02:21 PM ] |
| Thanks for the investigation and comment about it. The stringbuffer problem is known problem in Java, as well listed in bugs on at least 1.6 as far as I know. |