<< Back to previous view

[QB-2628] Out of Memory in QB Server
Created: 20/Jan/16  Updated: 13/Sep/22

Status: Closed
Project: QuickBuild
Component/s: None
Affects Version/s: 6.1.2
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Phong Trinh Assigned To: Robin Shen
Resolution: Fixed Votes: 0
Remaining Estimate: Unknown Time Spent: Unknown
Original Estimate: Unknown
Environment: Linux


 Description   
 We just upgraded our QuickBuild server to the latest version of QuickBuild and Java8. Then we are experiencing the memory issue which caused the server down several times since the upgrade. Look like the Java consumes a lot of memory. The physical RAM on the machine is 4GB which is as same as before the upgrade. Is QB compatible with Java8? The server log is as the follow, please advise:
   #
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 3145728 bytes for committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2627), pid=21777, tid=140167248578304
#
# JRE version: OpenJDK Runtime Environment (8.0_65-b17) (build 1.8.0_65-b17)
# Java VM: OpenJDK 64-Bit Server VM (25.65-b01 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
--------------- T H R E A D ---------------

Current thread (0x00007f7b54081800): VMThread [stack: 0x00007f7b3aff8000,0x00007f7b3b0f9000] [id=21781]

Stack: [0x00007f7b3aff8000,0x00007f7b3b0f9000], sp=0x00007f7b3b0f7460, free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xa05a3d]
V [libjvm.so+0x4c2c6a]V
[libjvm.so+0x869c50]
V [libjvm.so+0x86441e]
V [libjvm.so+0x8dac2b]
V [libjvm.so+0x8db900]
V [libjvm.so+0x8dbc5b]
V [libjvm.so+0x8d81cc]
V [libjvm.so+0x8d89a8]
V [libjvm.so+0x88d673]
V [libjvm.so+0xa07693]
V [libjvm.so+0xa0bbf7]
V [libjvm.so+0xa0a5e8]
V [libjvm.so+0xa0aab9]
V [libjvm.so+0xa0af11]
V [libjvm.so+0x8670e2]

VM_Operation (0x00007f7acfc359a0): ParallelGCFailedAllocation, mode: safepoint, requested by thread 0x00007f7af4033800

 Thanks,
Phong Trinh
 

 Comments   
Comment by Robin Shen [ 21/Jan/16 12:17 AM ]
It is not strange that different JVM versions has different memory consumptions. How many memory you've assigned to QuickBuild via wrapper.java.maxmemory option in "conf/wrapper.conf"?
Comment by Phong Trinh [ 21/Jan/16 03:03 AM ]
Thank you for promptly response. I am unable to login the server to obtain the current value of maxmemory in a moment. What ideal value for it? Thanks.
Comment by Robin Shen [ 21/Jan/16 03:27 AM ]
QB server typically needs more than 2G mem for production use (and may need more if you have thousands of configurations), and if your machine only has 4G physical memory, it might not be sufficient considering other applications in the OS.
Comment by Phong Trinh [ 21/Jan/16 03:42 AM ]
Thank you again for promptly response. This machine has 4GB of RAM and dedicates to QB server and local agent. I don't run builds on local agent, it is for maintenance tasks only. I have around a thousand configurations.
Comment by Robin Shen [ 21/Jan/16 05:08 AM ]
Please check "conf/wrapper.conf" of both QB server and QB local agent to see what is the value of "wrapper.java.maxmemory" is.
Comment by Phong Trinh [ 28/Jan/16 03:07 PM ]
The value of "wrapper.java.maxmemory" is 2048m. I found that QB itself had memory leak, and the issue is fixed 6.1.3.
Comment by Robin Shen [ 29/Jan/16 11:57 AM ]
Yes that is true, please upgrade to 6.1.3 to see if it solves the issue at your side.
Comment by Phong Trinh [ 29/Apr/16 02:47 PM ]
The issue is fixed after the upgrade. Thanks.
Comment by Phong Trinh [ 13/Sep/22 07:59 PM ]
Please close this issue after upgraded to 11.0.28. Thanks!
Generated at Tue Apr 16 16:51:48 UTC 2024 using JIRA 189.