History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-3183
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Unassigned
Reporter: AlSt
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
QuickBuild

Use different approach for truncate table (especially when using postgres)

Created: 24/May/18 09:17 AM   Updated: 05/Jul/18 05:33 AM
Component/s: None
Affects Version/s: 7.0.31
Fix Version/s: 8.0.10

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown


 Description  « Hide
Hi

we see a lot of problems lately because the measurements compression does a truncate of tables which actually takes a couple of hours.

Can you please implement a different truncation mechanism to avoid these long running truncate statements.

As far as I've seen there is no foreign key dependency in the "qb_measurement_datarXX" tables so it should be possible to just do the following steps in a single transaction:
1. lock qb_measurement_datarXX row exclusive via: LOCK TABLE qb_measurement_datarXX IN ROW EXCLUSIVE mode;
2. create a new table with a name like "qb_measurement_datarXX_new" with: CREATE TABLE qb_measurement_datarXX_new (LIKE qb_measurement_datarXX)
3. rename the table to something like "qb_measurement_datarXX_old" with: ALTER TABLE qb_measurement_datarXX RENAME TO qb_measurement_datarXX_old
4. rename the new table to "qb_measurement_datarXX" via: ALTER TABLE qb_measurement_datarXX_new RENAME TO qb_measurement_datarXX
5. commit the changes (unlocks the table and finishes the transaction)
6. start DROP TABLE qb_measurement_datarXX in the background

as creating a new table and the renaming just takes seconds and not hours this would not interrupt the operation for that long. And the DROP TABLE can just run out of process as it is not used in any way.
If there is some concern that the new table name could interrupt with something a generated name for just this purpose would also suffice.

And just to clarify what's meaning long execution time, the log line is as follows:
Finished truncating dead table [MeasurementDataR00], [0] rows removed in [5339615] millis

here 0 means that truncate is used because TRUNCATE basically does not return the exact amount of rows deleted. And it took around 1.5 hours to just execute the TRUNCATE.

 All   Comments   Work Log   Change History      Sort Order:
Change by Robin Shen [24/May/18 10:57 PM]
Field Original Value New Value
Assignee Robin Shen [ robinshine ] Steve Luo [ steve ]

Change by Steve Luo [23/Jun/18 02:45 AM]
Status Open [ 1 ] Resolved [ 5 ]
Assignee Steve Luo [ steve ]
Resolution Fixed [ 1 ]

Change by Steve Luo [23/Jun/18 02:45 AM]
Fix Version/s 8.0.10 [ 11804 ]