History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-1801
Type: New Feature New Feature
Status: Reopened Reopened
Priority: Major Major
Assignee: Unassigned
Reporter: Jonathan Oulds
Votes: 2
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
QuickBuild

QuickBuild should support the concept of resource pools

Created: 02/Oct/13 11:14 AM   Updated: 06/Sep/16 11:07 PM
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.24

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown


 Description  « Hide
A resource pool is a collection of resources that can be used by one or more agents. The number of resources available is defined by the pool and not the number of agents using the pool.

e.g. I have a resource pool called "Coverity Licenses" it has 5 units. I have 20 build agents 15 of which can use the "Coverity Licenses" resource pool. When a build step requires the "Coverity License" the build agent will draw from the pool, leaving only 4 licenses available.

This concept can be simulated at the moment by defining a machine that contains 5 units of the "Coverity License" resource and creating a parent step that encapsulates any build step that requires Coverity. However this is not ideal.

1- It assumes I know the difference between a node based and grde wide resource, e.g. If I change my license from 5 floating license to 10 licenses on specific machines I will need to rewrite all my configurations.
2- It messes up the process of transferring files to and from build agents, I now have to transfer files from the machine that did the checkout, to the machine that owns the "Coverity License" resource then to the machine that will perform the build.
3- Sometimes I may want to define a resource on a build agent. In this setup, I write a custom Node Selection script that checks for a list of resources plus the names of any resources that are defined as User Attributes on the node itself, e.g. I define a list of resources for a step "GCC" I also define a list of resources on a build agent "Win32 licenses" I join the two lists to form a set of resources that must be acquired to run a specifc step on a particular node. This may sound like a side case but it is actually a very powerful feature, see here for a full discussion:
http://forum.pmease.com/viewtopic.php?f=1&t=2843&p=10691#p10691

 All   Comments   Work Log   Change History      Sort Order:
Robin Shen [02/Oct/13 10:51 PM]
For issue 1: why all configurations have to be re-written when you change licenses from 5 to 10?

Jonathan Oulds [03/Oct/13 09:48 AM]
With the current system:

5 floating licenses:
* One machine with 5 instance of the Coverity resource.
> Step1: Master
>> Step2: Checkout
>> Step3: Sequential step that requires the Coverity resource (will only run on one machine)
>>> Step4: Build step runs on any machine.

10 Fixed licenses: i.e. only specifc machines can run a Coverity build.
* 10 machines have 1 instance of the Coverity resource.
> Step1: Master
>> Step2: Checkout
>> Step3: Build step requires the Coverity resource.

Becaue I change my Coverity license I have to modify my builds to remove the sequential step that acquires the Coverity resource. I also have to change the way resources are defined on my machines. Note that with the floating licences example if I want to copy files between machines I now need to perform an unnecessary copy to the machine owning the Coverity resource simply so that I can pass on the files to the actual machine that will perform the build.

After resource pools are implemented:
5 floating licenses:
* Define a resource pool that contains 5 instances of the Coverity resource.
* Allow all machines to draw from the Coverity resource pool.
> Step1: Master
>> Step2: Checkout
>> Step4: Build step runs on any machine that has the Coverity resource i.e. any of them as long as there are enough instances in the pool.

10 Fixed licenses: i.e. only specifc machines can run a Coverity build.
I have a choice, I can restrict the number of machines that can access the pool and up the instances in the pool to 10 or I can define a Coverity resource on each of the 10 machines that will be allowed to perform the Coverity build, in either case the steps look like this, i.e. unchanged.
> Step1: Master
>> Step2: Checkout
>> Step3: Build step requires the Coverity resource.

Currently I have to define my steps to work around the type of resource I wish to define, as you can see by extending the resource concept to include pooled resource this is no longer necessary.


Jonathan Oulds [03/Oct/13 10:10 AM]
As an extra:

Imagine I have have 5 floating licenses for Coverity and 2 floating licenses for AdvancedInstaller. I want to run a Coverity build and produce an MSI in the same build.

Currently I have two choices
* One machine with 5 instance of the Coverity resource.
* Another machine with 2 instances of the AdvancedInstaller resource.
> Step1: Master
>> Step2: Checkout
>> Step3: Sequential step that requires the Coverity resource.
>>> Step4: Sequential step that requires the AdvancedInstaller resource.
>>>> Step5: Build step runs on any machine.

As you can see lots of copying of files and a convuluted setup.

or:
* One machine with 5 instance of the Coverity resource and two instances of the AdvancedInstaller resource, effectively a license server.
> Step1: Master
>> Step2: Checkout
>> Step3: Sequential step that requires the Coverity resource and AdvancedInstaller resource.
>>> Step4: Build step runs on any machine.

Now imagine that I buy 3 fixed licenses of AdvancedInstaller, 5 licenses in total. This setup CANNOT be represented with the current resource model. So instead I give up my 2 floating licenses and buy 5 fixed licenses of AdvancedInstaller, now I have to go through my builds and change the resource requirements of all my builds.

With resource pools, I simply specify that my build step requires Coverity and AdvancedInstaller, and I am free to change the nature of my licenses at will, from fixed to floating to a mixture of both.

Change by Steve Luo [03/Sep/16 01:43 PM]
Field Original Value New Value

Change by Steve Luo [03/Sep/16 01:43 PM]
Fix Version/s 6.1.24 [ 11679 ]

Change by Robin Shen [06/Sep/16 11:07 PM]
Resolution Fixed [ 1 ]
Status Resolved [ 5 ] Reopened [ 4 ]