| << Back to previous view |
|
[QB-3299] Automated Database Bacup Causes issues with QB Ui
|
|
| Status: | Closed |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | 7.0.16 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major |
| Reporter: | Todd Scholl | Assigned To: | Robin Shen |
| Resolution: | Won't Fix | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Environment: |
Operating System Linux 4.4.114-94.14-default, amd64
OS User Name root JVM Java HotSpot(TM) 64-Bit Server VM 1.7.0_79, Oracle Corporation QuickBuild Version 7.0.16 - Fri May 12 09:57:15 EDT 2017 Total Memory 1.98 GB Used Memory 1.02 GB Configurations 7629 |
||
| File Attachments: |
console.log.zip
|
| Description |
|
Support,
On the morning of 11/22 at 1 am, the Quickbuild UI became non responsive. You could hit the sign in page but any attempt to login was met with a spinning browser. A look in the log files for the time showed only that the Automated DB backup had kicked off. In addition there were a number of messages indicating that jobs scheduled for 1 am we put in a "sleep" state. As we were unable to get to the UI even after waiting over an hour, we rebooted the system around 2 am. The UI came back at that time. We then turned off the auto backup function in the UI and this morning there was no issue. Can we get an RCA about what happened with our server during this time? Is there some tuning that needs to be done to allow the auto backup to work smoothly? |
| Comments |
| Comment by Todd Scholl [ 23/Nov/18 08:15 PM ] |
| I have attached a copy of the console for the time in question. Let me know if you need other data. |
| Comment by Robin Shen [ 23/Nov/18 10:44 PM ] |
|
Looks like there are too many audit log entries in your database. Please remain auto-backup off and configure less log entry preserve time in system setting. After one night, the audit log entry will be cleaned up and then enable auto-backup. Also if UI blocks next time, please run below command from QB server to get a stack trace:
/path/to/jdk/bin/jstack <QB JVM process id> (not the wrapper process id) |
| Comment by Robin Shen [ 23/Nov/18 10:46 PM ] |
| Also I noticed that memory of your QB server is too small to handle many database entries. Please increase it by editing property "wrapper.java.maxmemory" in "conf/wrapper.conf". I'd suggest to give it at least 6G for millions of database entries. |
| Comment by Todd Scholl [ 26/Nov/18 05:05 PM ] |
|
Robin,
Thank you for the info. Will work with my team on these recommendations. We have 16 GB of memory space on this server. Is there any down side to increasing the max memory further? say 12 - 14 GB? |
| Comment by Robin Shen [ 26/Nov/18 11:15 PM ] |
| Giving it too much memory will cause noticeable JVM pause when it garbage collects memory. I'd suggest 8G for maximum. |