History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-3099
Type: Improvement Improvement
Status: Open Open
Priority: Major Major
Assignee: Robin Shen
Reporter: Robin Shen
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
QuickBuild

Separate build variables into a different table for query purpose

Created: 15/Dec/17 10:58 PM   Updated: 15/Dec/17 10:58 PM
Component/s: None
Affects Version/s: 7.0.30
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown


 Description  « Hide
The issue we run into is that we are reaching upwards of 200 builds per day, which makes it very probable that the search for a build that was triggered with a variable that matches a specific value will have to reach back several days before it finds what we're looking for. There's the obvious impact to the performance of the server when querying for these builds, but this can also take an extremely long time when you're querying for a list of N builds that match these parameters (and in some cases may have to walk the entire list of all builds ever done and still never find that many builds to list).
 
The purpose of this inquiry was to see if we could present a case where it makes sense for this feature to be natively supported in a future version of QuickBuild?
 
As best I can tell (from a cursory review of the QB source), the reason this is not already available is due primarily to the way that the variable keys/values are stored in the database, which seems to be in some sort of raw, serialized format that can easily be marshalled to/from a Java object. This is obviously not conducive to being able to filter / query using the database's back-end query language, so my estimation of the work required to support this would be something like this (simplified for the sake of brevity):
 
Alter the QuickBuild migrate / upgrade scripts to create a new database table:
This new table would have fields for build_id, variable_name, and variable_value
This new table could have fields indicating the hidden/secret status of the variable as well.
Alter the QuickBuild migrate / upgrade scripts to read the data out of the builds table in the serialized format and then insert it into the new table.
Alter the QuickBuild source to insert variable data into this new table instead of the old serialized format in the builds table.
Alter the QuickBuild REST interface to pull the data out of this table when a build object is being requested.
Given that this data is now being stored in a database-normalized form, it should be possible to allow filtering based on variable keys and values using table joins.
Performance concerns can be mitigated by making the table join optional based on whether the new variable-filtering query-param(s) exist in the URI, essentially allowing users to "pay for what they use".
 
This is something that I (and several industry peers of mine) would find extremely useful, enough such that I would even go so far as to offer to work on the feature and work on and submit a pull request for, with guidance from your architects / engineers (of course). There are a number of companies / products I know that base chunks of their development pipeline / toolchain on the QuickBuild REST interface, and this has universally been one of the bigger complaints we have.

 All   Comments   Work Log   Change History      Sort Order:
No changes have yet been made on this issue.