|
|
|
The problem: When a service account queries with count=50 but has restricted permissions across a large configuration tree, the while-loop iterates many times — each loading count more Build entities (with
full StepRuntime graphs via lazy-load) into the StatefulPersistenceContext. There is: 1. No session.clear() between loop iterations — rejected entities remain in the persistence context 2. No upper bound on total iterations — maxReturnEntitiesViaRest ( 3. No eviction of permission-denied entities — builds that fail SecurityHelper.hasPermission() stay in session After the loop, accessing getStepRuntimes() and getRepositoryRuntimes() triggers lazy-load of PersistentBag collections, materializing the full entity graph while the session is still open. Reproduction scenario:
- Service account with limited permissions polls /rest/builds?configuration_id=X&count=50&status=SUCCESSFUL across many configurations - Each response carries ~1 MB due to large StepRuntime payloads - High polling frequency (e.g. every 5 minutes across 40+ configurations) compounds the issue - If permission filtering rejects many results, a single request can load tens of thousands of Build entities before the session closes Suggested fix:
Add session.clear() at the end of each while-loop iteration in BuildResource.query() after collecting permitted builds into a detached list, and add a hard cap on total entities loaded (not just the requested count). For example: Session session = SessionManager.getSession(); int totalLoaded = 0; int maxTotalEntities = 10000; // configurable cap while (true) { List result = BuildManager.instance.search(criteria, first, count); for (Build build : result) { if (!SecurityHelper.hasPermission(build.getConfiguration())) continue; builds.add(build); } totalLoaded += result.size(); session.clear(); // Evict loaded entities between iterations if (builds.size() < count && result.size() >= count && totalLoaded < maxTotalEntities) { first += result.size(); continue; } break; } Note: session.clear() would require re-attaching or copying the permitted builds to a detached collection before clearing, since cleared entities become detached. Heap dump evidence:
- Thread qtp55085102-38184087: retained 7.2 GB, accumulation point StatefulPersistenceContext, 51,995 Build objects, 227,253 PersistentBag, 8,359,018 StepRuntime - 4 additional threads with identical pattern (5.9 GB, 5.4 GB, 5.1 GB, 4.6 GB) - All 5 threads blocked at BuildResource.query() > AbstractEntityManager.searchEntities() > CriteriaImpl.list() > PostgreSQL socket read Related: | |||||||||||||||||||||||||||||||||||||||||||||||||||
After decompiling BuildResource.query(), we identified a permission-filtered pagination loop that accumulates Hibernate entities without eviction:
// BuildResource.query() — simplified
builds = new ArrayList();
while (true) {
List result = BuildManager.instance.search(criteria, first, count);
for (Build build : result) {
if (!SecurityHelper.hasPermission(build.getConfiguration())) continue;
builds.add(build);
}
if (builds.size() < count && result.size() >= count) {
first += result.size();
continue; // Next iteration loads more entities into same session
}
break;
}
The session lifecycle in DefaultServletContainer.service() opens the Hibernate session before the request and closes it only after Jersey serializes the response:
// DefaultServletContainer.service()
SessionManager.openSession();
try {
// ... authentication ...
super.service(request, response); // BuildResource.query() + serialization
} finally {
SessionManager.closeSession(); // Only closed after full serialization
}