| << Back to previous view |
|
[QB-4277] Replace unbounded script class cache with bounded Guava Cache to resolve memory leak
|
|
| Status: | Closed |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | 14.0.40 |
| Fix Version/s: | 16.0.14 |
| Type: | Bug | Priority: | Major |
| Reporter: | Nguyen Danh Hung | Assigned To: | Robin Shen |
| Resolution: | Fixed | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Description |
|
Dear Mr. Robin Shen, Mr. Steve Luo,
Our collegue analyzed memory leak issue in ScriptInterpreter cache as below. Please consider this solution: fix: resolve Groovy ClassLoader memory leak in ScriptInterpreter cache Root cause: the Groovy ScriptInterpreter held compiled script classes in an unbounded Collections.synchronizedMap(ReferenceMap(HARD, SOFT)). Each class retains a strong reference back to the GroovyClassLoader that created it via Class.getClassLoader(). Because keys were HARD references, the classloaders stayed alive indefinitely, preventing the WeakHashMap entries in java.beans.ThreadGroupContext from being GC'd. This caused the heap to accumulate ~2.6 GB (WeakHashMap$Entry[262144]) in production. Changes: - Replace unbounded ReferenceMap with Guava Cache (maximumSize 500) to cap the number of live compiled classes and their classloaders - Add RemovalListener that on every eviction: + Calls Introspector.flushFromCaches(evicted) to remove the stale BeanInfo entry from ThreadGroupContext + Calls GroovyClassLoader.close() to release native resources and allow GC to reclaim the classloader, clearing the WeakHashMap entry - Replace non-atomic getIfPresent + put with cache.get(key, Callable) so concurrent threads sharing the same cache miss compile only once, preventing duplicate classloader instances from being created - Mark evaluate() parameters final to allow capture inside the Callable - Add imports: java.beans.Introspector, java.io.IOException, java.util.concurrent.Callable, java.util.concurrent.ExecutionException, com.google.common.cache.{Cache,CacheBuilder,RemovalListener,RemovalNotification} |
| Comments |
| Comment by Nguyen Danh Hung [ 01/Jun/26 02:53 AM ] |
|
Affects Version is 12.0.29
I chose the wrong version, please help me change it. Thank you |
| Comment by Robin Shen [ 03/Jun/26 02:36 AM ] |
| QB 16.0.14 uses SOFT key for the cache so that the script string can be GCed under memory pressure (previously only class itself is GCed). Also dynamic script generated by promotion (use build id as part of the script content) will not be cached in order not to increase the cache in a unpredictable manner. Guava cache is not used as it is not easy to decide how many scripts should be cached. |