<< Back to previous view

[QB-3160] Upgrade Jetty to latest version to fix various 9.2 issues
Created: 27/Mar/18  Updated: 25/Jan/19

Status: Closed
Project: QuickBuild
Component/s: None
Affects Version/s: 8.0.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Robin Shen Assigned To: Robin Shen
Resolution: Duplicate Votes: 1
Remaining Estimate: Unknown Time Spent: Unknown
Original Estimate: Unknown


 Description   
After, QB upgrade to version 7 we started observing timeout issues during downloading large (>1G) build artifacts.

I see that this bug has been fixed in latest QB 8 version, however we don't want to upgrade our QB instances to this version.

What would you recommend to fix this issue, logs from QB console below:

 

--- console.log ---

jvm 1 | java.lang.RuntimeException: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | Caused by: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | java.lang.RuntimeException: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | Caused by: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

jvm 1 | 2018-03-27 02:26:24,607 WARN - Could not send response error 500: java.lang.RuntimeException: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30001/30000 ms

 

I tried to increase Jetty timeouts in Jetty but it didn't help. Would you recommend to upgrade Jetty in this QB version?

 

 

2nd issue is related with log rotating on Jetty

Seems that Jetty in version 9.2 is not able to rotate logs correctly, even after restart log is rotated only once.

 

Our Jetty configuration look like:

/infrastructure/qbs/current/conf/jetty.xml

 

<?xml version="1.0"?>

<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">

 

<Configure id="Server" class="org.eclipse.jetty.server.Server">

 

        <Get name="ThreadPool">

                <Set name="minThreads">256</Set>

                <Set name="maxThreads">1024</Set>

        </Get>

 

       <Ref id="SocketConnector">

                <Set name="acceptQueueSize">10000</Set>

        </Ref>

 

        <Ref id="Handlers">

                <Call name="addHandler">

                        <Arg>

                                <New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler">

                                        <Set name="requestLog">

                                                <New id="RequestLogImpl" class="org.eclipse.jetty.server.NCSARequestLog">

                                                        <Set name="filename">/infrastructure/qbs/logs/yyyy_mm_dd.request.log</Set>

                                                        <Set name="filenameDateFormat">yyyy_MM_dd</Set>

                                                        <Set name="retainDays">14</Set>

                                                        <Set name="append">true</Set>

                                                </New>

                                        </Set>

                                </New>

                        </Arg>

                </Call>

        </Ref>

 

</Configure>

 

 

<hostname>:~ # ll -h /infrastructure/qbs/logs/

total 3.7G

-rw-rw---- 1 *** *** 528M Mar 25 16:59 2018_03_25.request.log - log has been created after QB restart

-rw-rw---- 1 *** *** 1.8G Mar 27 02:36 2018_03_26.request.log - log has been truncated only once after restart

-rw-rw---- 1 *** *** 39M Mar 27 02:36 console.log

-rw-rw---- 1 *** *** 251M Mar 26 19:00 console.log.1

-rw-rw---- 1 *** *** 251M Mar 25 04:35 console.log.2

-rw-rw---- 1 *** *** 251M Mar 24 05:29 console.log.3

-rw-rw---- 1 *** *** 251M Mar 22 01:52 console.log.4

<hostname>:~ # date

Tue Mar 27 02:38:05 PDT 2018

 Comments   
Comment by Robin Shen [ 25/Jan/19 02:48 PM ]
Will be covered by QB-3230
Generated at Sat Apr 27 00:34:30 UTC 2024 using JIRA 189.