|
|
|
Suggested fix:
Add session.clear() at the end of each while-loop iteration in BuildResource.query() after collecting permitted builds into a detached list, and add a hard cap on total entities loaded (not just the requested count). For example: Session session = SessionManager.getSession(); int totalLoaded = 0; int maxTotalEntities = 10000; // configurable cap while (true) { List result = BuildManager.instance.search(criteria, first, count); for (Build build : result) { if (!SecurityHelper.hasPermission(build.getConfiguration())) continue; builds.add(build); } totalLoaded += result.size(); session.clear(); // Evict loaded entities between iterations if (builds.size() < count && result.size() >= count && totalLoaded < maxTotalEntities) { first += result.size(); continue; } break; } Note: session.clear() would require re-attaching or copying the permitted builds to a detached collection before clearing, since cleared entities become detached. Reproduction scenario:
- Service account with limited permissions polls /rest/builds?configuration_id=X&count=50&status=SUCCESSFUL across many configurations - Each response carries ~1 MB due to large StepRuntime payloads - High polling frequency (e.g. every 5 minutes across 40+ configurations) compounds the issue - If permission filtering rejects many results, a single request can load tens of thousands of Build entities before the session closes The problem: When a service account queries with count=50 but has restricted permissions across a large configuration tree, the while-loop iterates many times — each loading count more Build entities (with
full StepRuntime graphs via lazy-load) into the StatefulPersistenceContext. There is: 1. No session.clear() between loop iterations — rejected entities remain in the persistence context 2. No upper bound on total iterations — maxReturnEntitiesViaRest ( 3. No eviction of permission-denied entities — builds that fail SecurityHelper.hasPermission() stay in session After the loop, accessing getStepRuntimes() and getRepositoryRuntimes() triggers lazy-load of PersistentBag collections, materializing the full entity graph while the session is still open. Root Cause Analysis:
After decompiling BuildResource.query(), we identified a permission-filtered pagination loop that accumulates Hibernate entities without eviction: // BuildResource.query() — simplified builds = new ArrayList(); while (true) { List result = BuildManager.instance.search(criteria, first, count); for (Build build : result) { if (!SecurityHelper.hasPermission(build.getConfiguration())) continue; builds.add(build); } if (builds.size() < count && result.size() >= count) { first += result.size(); continue; // Next iteration loads more entities into same session } break; } The session lifecycle in DefaultServletContainer.service() opens the Hibernate session before the request and closes it only after Jersey serializes the response: // DefaultServletContainer.service() SessionManager.openSession(); try { // ... authentication ... super.service(request, response); // BuildResource.query() + serialization } finally { SessionManager.closeSession(); // Only closed after full serialization } | |||||||||||||||||||||||||||||||||||||||||||||||||||
- Thread qtp55085102-38184087: retained 7.2 GB, accumulation point StatefulPersistenceContext, 51,995 Build objects, 227,253 PersistentBag, 8,359,018 StepRuntime
- 4 additional threads with identical pattern (5.9 GB, 5.4 GB, 5.1 GB, 4.6 GB)
- All 5 threads blocked at BuildResource.query() > AbstractEntityManager.searchEntities() > CriteriaImpl.list() > PostgreSQL socket read
Related:
QB-3717(maxReturnEntitiesViaRest) partially addresses this but only caps the requested count, not total loaded entities during permission filtering.