| << Back to previous view |
|
[QB-3617] There are two types error ( Read timed out and Error writing to server errors) that occur in many nodes of our quick build.
|
|
| Status: | Closed |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | 9.0.22 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical |
| Reporter: | Yoongeon Lee | Assigned To: | Robin Shen |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| File Attachments: |
Error Writting to Server.png
KOR-MACHINE05.png
linux machine error - timeout expired 300000.png
qbuild error.png
qbuild error1.png
|
| Description |
|
We found a serious problem last year, and we solved it by upgrading the qbuild server from 8.0.8 to 9.0.22. ( Issue number : [ Recently, we're facing same serious problem. There are two types error ( Read timed out and Error writing to server errors) that occur in many nodes of our quick build. We are still having a timeout issue between our build agents and the ci server. I have attached screenshots for your understanding. Please see this issue as soon as possible. this is a big issue for us now. - qbuild log "caused by: Error writing to server" "caused by: Read timed out" - qbuild agent TimeoutException: Idle timeout expired: 300001/300000 ms |
| Comments |
| Comment by Robin Shen [ 12/Sep/20 08:00 AM ] |
| The timeout indicates that server might be loaded. Is this happening when there are many builds running? |
| Comment by Robin Shen [ 12/Sep/20 08:06 AM ] |
| Also please check if you have builds publishing/downloading very large artifacts. |
| Comment by Yoongeon Lee [ 12/Sep/20 08:21 AM ] |
|
Thanks for the quick response, Robin
Builds happen multiple times on many nodes per day. Regarding large artifacts, I need to check with our admin engineer. and then let you know. |
| Comment by Yoongeon Lee [ 12/Sep/20 10:55 AM ] |
|
Hi Robin
our top engineer mentioned that there are a backup or VM snapshot rotation or similar long-lasting, high network demand transaction on our ci server. If the issue is caused by heavy downloads or backups, how should we fix the issue? For reference, the issue occurred two days ago, and we didn't make any change on server at that time. Please check if there were similar issues on other sites. |
| Comment by Robin Shen [ 12/Sep/20 11:12 AM ] |
| We will increase network timeout to see if it helps. |
| Comment by Robin Shen [ 14/Sep/20 11:40 PM ] |
|
Please upgrade QB server and agents experiencing timeout issue to below build to see if it helps:
https://build.pmease.com/build/5203 |