[#QB-4120] QuickBuild node went down, but it was showed as available on the server

QuickBuild

QuickBuild node went down, but it was showed as available on the server

Created: Wednesday 01:06 AM Updated: Thursday 01:51 AM

Component/s:

None

Affects Version/s:

12.0.29

Fix Version/s:

None

Original Estimate:

Unknown

Remaining Estimate:

Unknown

Time Spent:

Unknown

Description

« Hide

I have a QuickBuild configuration that allows to run many test cases based on the inputs, and one test case runs on one node at a time. The node/system is reverted after each test. I have a trigger to send a large number of test cases to the configuration, and they are waiting until the nodes to be available. I am having two issues as follows:
- QuickBuild server drops all of my test when all of the nodes in the resource disappeared.
- The nodes/test systems are down (while reverting) but they are still showed as active on QB server. I am thinking of increasing heartbeat frequency at nodes' level. Is it possible?
Any advice appreciated.
ptrinh

All

Comments

Work Log

Change History

Sort Order:

[ Permlink | « Hide ]

Robin Shen [18/Sep/24 01:42 AM]

Agent timeout can be configured in system setting. However, instead of changing agent timeout, please make sure to stop agent service gracefully before reverting the system so that agent has the chance to notify server of its down.

[ Permlink | « Hide ]

Phong Trinh [18/Sep/24 02:34 AM]

I am concerned that if I stop the service before reverting the system , there is chance that qb server drops the node before the reversion happens.

[ Permlink | « Hide ]

Robin Shen [18/Sep/24 03:45 AM]

You may run a command step to stop service, for instance:

# sleep a while to allow the build to finish
sleep 10
service buildagent stop

Make sure to untick the option "wait for finish" in advanced setting of the step.

[ Permlink | « Hide ]

Phong Trinh [18/Sep/24 09:20 PM]

Thank you very much for your suggestion. I am going to give it a try and will keep you informed.
Regarding my first issue: "QuickBuild server drops all of my test when all of the nodes in the resource disappeared." How do I avoid the server dropping the tests?
Some nodes have very heavy load and need more time to response to the server. Can I override the agent timeout at their level?

[ Permlink | « Hide ]

Robin Shen [19/Sep/24 01:51 AM]

What do you mean by "when all of the nodes in the resource disappeared"? Is this because all your nodes are running heavy load and QB server thinks they are dead? If so, what do you mean by "QB drops all tests"?