| << Back to previous view |
|
[QB-582] I'd like to be able to pass parameters to child steps without displaying them in the title
|
|
| Status: | Resolved |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major |
| Reporter: | Roger Perkins | Assigned To: | Robin Shen |
| Resolution: | Won't Fix | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Description |
|
I'd like to use the new parameter option to be able to pass parameters to child steps so I can reduce the number of overall unique steps i have. Right now I have to duplicate child steps up for different combinations, I can't really use the new parameter stuff to generate all the steps as I need fine control over which agents work runs on. I can see it would probably work right now if I just specified a single option per parameter, but then it starts displaying them in the title and creates very long titles on the step (which doesn't display well if you have 7ish steps running in parallel)
Could you maybe not display the parameters in the title if only one combination of all available steps exists? This would probably help me reduce some of the duplicates of my steps by a factor of 12 (where I can control what they do via parameter). |
| Comments |
| Comment by Robin Shen [ 29/Jun/10 03:27 AM ] |
|
parameter name and value needs to be appended step name even if there is only one option for the param since QuickBuild internally relies on this to get value of certain parameter for a certain step. It can not simply examine the step definition to get param value as scripts might be enclosed.
On the other hand, although specifying single option per parameter is totally valid, it does not make too much sense to me, since step will not be repeated in that case. I might be able to provide some suggestion if you can detail your step requirement across agents. |
| Comment by Roger Perkins [ 29/Jun/10 09:14 AM ] |
|
So right now I have a parallel step that runs 6 serial steps. These serial steps build a total of 12 exe's for 3 different platforms. Right now, due to not being able to re-use steps in 2.1 I have 12 copies of "Build Exe" which slight command line variations on (build.exe <PLATFORM> <CONFIG>). I need the 6 serial steps to go onto the same machines they ran on before to keep things running fast (a build from scratch could take ~10mins), its not something I can do using the parameters feature exposed.
However, if I could pass two values (be they parameters, variables as in the configuration) down from the serial step to it's children then I could have a single "Build EXE" step that is used multiple times and is much easier to maintain. I could use the new parameter feature to just pass parameters down to the children, but that bloats the name of the step, especially as I've got 7 steps running in parallel). If I could specify variables in the same was as I can in the configuration, that would then be passed down to the children such that I can query their value and expand then out then it would meet my needs. Right now the parameter code is the closest thing to that, just would be great not to have to see the parameter values on the step name. |
| Comment by Roger Perkins [ 29/Jun/10 09:15 AM ] |
| Just a quick follow up, I'm not after the step being repeated when I specify a single value for each parameter, I just want it to run once and be able to pass the value to it's children so I can make the steps that get run generic across 12 different executable compiles. |
| Comment by Robin Shen [ 30/Jun/10 04:52 AM ] |
|
Can I safely assume that you have to build exes for 3 different platforms, and for each platform you have to build exes for 4 different configs, so in total you have to build 12 different exes?
If yes, this can be greatly simplified with repeating parameters: you only have to define one serial step, and its repeating parameters can be specified like below for example: platform: platform1, platform2, platform3 config: config1, config2, config3, config4 This serial step is put into the parallel composition, and it will be repeated parallely for each combination of platform and config. The step calling "build.exe" is placed into the serial step to do the actual work and can be configured to execute command: build.exe ${params.get("platform")} ${params.get("config")} You mentioned that it is important to have the serial steps running on the same machine they ran on before. By mentioning "ran on before", I guess you have other corresponding serial steps preparing the workspace on all 3 platforms for 4 configs. They can also be repeated so that only one serial step is needed, assume it is named "prepare_step". To make sure the serial step calling build.exe running on the same machine as "prepare_step", you may specify node match condition of the serial step as: node.address == steps.get("master>path>to>prepare_step?os=" + params.get("os") + "&config=" + params.get("config")).node.address This does not make step title shorter, but can make step maintenance a lot easier since there are only two serial steps here (one is repeated for workspace preparation, and another is repeated to call the build.exe step). |
| Comment by Roger Perkins [ 30/Jun/10 09:41 AM ] |
|
I had thought about using the repeating stuff, but right now it's not a straight repeat for all 12 exe's. I have 6 builders that each first grab the latest code, then they each build two exe's. The exe steps aren't exactly the same right now for each of the 3 different platforms so can't be done with an easy repeat setup (although I might try and spend some time reworking that in the future). The steps also can't be run on the same machine at the same time (which I guess I could do with resource locking to stop multiple steps trying to run on the same pc). The other things is some of the bits take much longer than others, so I'd like to be able to manually distribute the load across the different machines to reduce time if possible.
Does the parameter stuff just work by going up the hierachy looking for a match to the parameter in the name of a step? Ideally I just want to be able to pass two configurable values down to the child steps so I can make them more generic. I don't really care about the repeat stuff right now as it's not straightforward for me to use in my current setup due to the steps not being the same. I can do this by just specifying one value for each parameter such that the children can pick them up, but then I'll be having "?platform=<X>?config=<Y>" tacked onto steps and they already wrap like crazy. Is there any way I can write a custom step of my own that will allow me to pass values down to child steps that they can then query? |
| Comment by Robin Shen [ 01/Jul/10 09:29 AM ] |
|
You may pass values from parent step to child step without using params API:
1. For each of the parent step, define the pre-execute script to store platform and config info into build object, for example: build.reports.put("platform", "platform1"); build.reports.put("config", "config1"); 2. In the child step, make sure of stored platform and config info: build.exe ${build.reports.get("platform")} ${build.reports.get("config")} |
| Comment by Robin Shen [ 01/Jul/10 09:37 AM ] |
| Or you can define "platform" and "config" variables, and set their values in pre-execute script of parent step, and then access them in child steps. |
| Comment by Roger Perkins [ 01/Jul/10 09:39 AM ] |
|
Will either method (the "reports" map or the variables) basically not overwrite the same values when set from the 6 parent steps running in parallel? I'm not up to speed on how values might be propogated when set in steps.
If that works that will be most excellent :) |
| Comment by Robin Shen [ 01/Jul/10 09:45 AM ] |
| No they won't overwrite each other. Both keep their own copy when steps are set to run in parallel. |