History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-3650
Type: Bug Bug
Status: Closed Closed
Resolution: Incomplete
Priority: Major Major
Assignee: Robin Shen
Reporter: Phong Trinh
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
QuickBuild

There was a deadlock in QuickBuild that caused the memory spike

Created: 17/Dec/20 11:04 PM   Updated: 08/Jan/22 10:23 AM
Component/s: None
Affects Version/s: 9.0.46
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
File Attachments: 1. Text File QuickBuild_Server_Deadlock.txt (168 kb)

Image Attachments:

1. QB Server - DB Config.PNG
(105 kb)


 Description  « Hide
 Recently our QuickBuild server went down by itself several times. We checked the server logs and found that there was a deadlock as follow which caused the memory spike. we set wrapper.java.maxmemory=4096, I think it should be fine. I attached the server log file for your references.

 =======================================================================================================================================

 2020-12-17 10:17:22,470 [C3P0PooledConnectionPoolManager[identityToken->z8kfltaesj634k1iyllxx|58cae5cb)-AdminTaskTimer] WARN com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@44547507 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
2020-12-17 10:20:56,673 [pool-1-thread-21150] ERROR com.pmease.quickbuild.grid.GridJob - Error notifying task node of job finishing (job class: 86efdbe6-673b-4b9b-8d92-373f930e71aa, job id: com.pmease.quickbuild.stepsupport.StepProcessJob, task node: QB_SERVER.com:8810)
    java.lang.OutOfMemoryError: Java heap space
=======================================================================================================================================

 Please look into the issue and advise.

 Thank you in advance,
ptrinh

 All   Comments   Work Log   Change History      Sort Order:
Change by Phong Trinh [17/Dec/20 11:05 PM]
Field Original Value New Value
Attachment QuickBuild_Server_Deadlock.txt [ 10873 ]

Change by Phong Trinh [13/Jan/21 03:23 PM]
Attachment QB Server - DB Config.PNG [ 10874 ]

Change by Robin Shen [08/Jan/22 10:23 AM]
Status Open [ 1 ] Closed [ 6 ]
Resolution Incomplete [ 4 ]