History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-2594
Type: Bug Bug
Status: Closed Closed
Resolution: Won't Fix
Priority: Major Major
Assignee: Steve Luo
Reporter: David Keane
Votes: 0
Watchers: 1

If you were logged in you would be able to see more operations.

Testng publish task taking over 4 minutes

Created: 07/Dec/15 04:57 PM   Updated: 03/Apr/16 07:11 AM
Component/s: None
Affects Version/s: 6.0.28
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
File Attachments: 1. Text File qb.log (14 kb)

 Description  « Hide
When publishing a testng report it is taking over 4 minutes

see log entry below

6:28:00,642 INFO - Checking step execute condition...
16:28:00,643 INFO - Step execute condition satisfied, executing...
16:28:01,449 INFO - Executing pre-execute action...
16:28:01,449 INFO - Running step...
16:28:01,758 INFO - Start generating report 'methods'.
16:31:10,266 INFO - Finished generating report 'methods' successfully.
16:31:10,266 INFO - Start generating report 'classes'.
16:31:12,018 INFO - Finished generating report 'classes' successfully.
16:31:12,018 INFO - Start generating report 'packages'.
16:31:12,133 INFO - Finished generating report 'packages' successfully.
16:31:12,133 INFO - Start generating report 'tests'.
16:31:12,197 INFO - Finished generating report 'tests' successfully.
16:31:12,197 INFO - Start generating report 'groups'.
16:31:13,799 INFO - Finished generating report 'groups' successfully.
16:31:13,799 INFO - Start generating report 'suites'.
16:31:13,861 INFO - Finished generating report 'suites' successfully.
16:31:14,022 INFO - Initialize overview data ...
16:32:08,884 INFO - Executing post-execute action...

 All   Comments   Work Log   Change History      Sort Order:
Steve Luo [08/Dec/15 12:56 AM]
Is your testng report very big? Can you please share your testng report to me (steve at pmease) so that I can have a test?

David Keane [08/Dec/15 05:06 PM]
Attached the reports. We are actually collecting multiple xml files from different component builds.


Steve Luo [09/Dec/15 12:26 AM]
Hi David,

Thank you very much for sharing me the reports. I can publish the reports in less than 1 minutes in my computer.
Please turn your configuration log level to DEBUG by editing configuration General Setting so we can know more detailed on where is the slowest part.

By the way, if the attached reports contains any confidential information, let me know and I will remove it.

David Keane [09/Dec/15 09:46 AM]
Hi Steve,
   Yes can you please delete the reports if you are finished with them. I will switch on debug and send you the info shortly. one thing I have noticed is that if I clear doen the h2 db behing testng on the quickbuidl server this also runs in under a minute.


David Keane [09/Dec/15 11:03 AM]
Attached log for publish

Steve Luo [09/Dec/15 11:23 AM]
From your log, it seems that transferring files between your server and agent is very slow, see log below:

10:49:03,345 DEBUG - Start transfering previous report from server [e:\qb-storage\root\LAE\trunk\proofBuild\java\*****s\83904/reports/testng/DEFAULT] to local [/home/*****/app/pmease/*****agent-prod/temp/previouscategorycd431c62-d658-4f79-92c2-024b00186c8a/DEFAULT]
10:51:24,403 INFO - Finished generating report 'methods' successfully.

It cost about 2 minutes.

So I'd like to suggest you test copying the same file/directory [e:\qb-storage\root\LAE\trunk\proofBuild\java\*****s\83904/reports/testng/DEFAULT] to the agent and see how many time you need.
Or, you can try run Publish TestNG step on your server node directly to see whether it can become faster.

David Keane [09/Dec/15 12:24 PM]
I have gone into the advanced setting and switched it to store artifacts on the server but for some reason it is still doing the transfer? Do I need to do anything else?

12:16:32,639 INFO - Start processing report: methods
12:16:34,968 DEBUG - Start transfering previous report from server [e:\qb-storage\root\LAE\trunk\proofBuild\java\*****s\83946/reports/testng/DEFAULT] to local [/home/*****/app/pmease/*****agent-prod/temp/previouscategoryd383b4cd-9af3-406a-ae16-7c221e32a0f1/DEFAULT]
12:19:03,872 INFO - Finished generating report 'methods' successfully.

David Keane [09/Dec/15 01:34 PM]
I have also logged onto the quickbuild server and using scp copied the file listed above in under 30 seconds. The file is on 17M so the delay does nor make sense.


Steve Luo [09/Dec/15 02:47 PM]
Sorry, I don't mean storing artefacts on server node. I want you to test run publish step on server node only. A simple test is:

1. Copy all XML reports to some directory on server
2. Create a test configuration with one step - Publish TestNG reports which publishes the reports in that directory
3. Run a few builds
4. Let me know your results.

David Keane [09/Dec/15 11:06 PM]
So running this on the server directly takes 1 minute .


Steve Luo [10/Dec/15 01:00 AM]
So we can now confirm the slow is caused by transferring files here.

Was there a lot of transferring artefacts at that time? And what was your server load?

You may try run a build in some spare time to see what's the result.

Also, if there is a lot of transferring artefacts, I recommend you storing artefacts in your agents instead of server to reduce the server load. See below page for more details:


David Keane [10/Dec/15 05:26 PM]
Hi Steve,
 I agree the slow speed is during transfer but why? What protocol is used for the transfer? When I transfer the same file from the server to the agent using scp is takes seconds? Is the slow time down to server load ?

I am already storing artefacts on the agents but it sill transfers from the server to agent if the steps are not ran on the agent.


Robin Shen [11/Dec/15 03:11 AM]
QB only stores user published artifacts on agent, but various test reports are still saved on server for web UI accessibility. We transfer files via http download, and its speed is comparable with ftp in normal cases. However if server is overloaded when many files are transferred in the same time (for instance, QB has to send various reports, build logs to server), it might be slow. If possible please check if the transfer is still slow when there is only one build running.

David Keane [14/Dec/15 11:33 AM]
Hi Robin,
   Thanks for the info I will run some tests and get back yo you. I am also talking to the network team here to see if we can establish if there is any packet loss on the network.


Steve Luo [03/Apr/16 07:11 AM]
It seems that the slow is not caused by publish TestNG step. Please open another issue to continue the discussion if you still have the problem.