|
|
|
[
Permlink
| « Hide
]
Steve Luo [22/Nov/17 10:49 AM]
I have reproduced this issue and I will get it fixed in next patch release..
I have done a workaround on my end, nevertheless I can still see problems when specifying multiple elements in this monitorDisks (if there is no such drive, constraint fails, or when it is doubled etc). You should fix this in the code in my opinion, so that you never send the multiple same values to the database.
Please look at below page for details
http://wiki.pmease.com/display/QB70/Configuring+Disk+Monitoring no, even when specifying
monitdisk: /opt/buildagent/,C:/,D:/ it still fails but when I specify empty monitdisk it works back ok. and on my clear setup this is the only thing (except user) which I have changed and copied from our production env directly
Oh, maybe I know what is the problem - the config for plugin, after upgrade it is doubled
monitdisk: /opt/buildagent/,C:/,D:/,/opt/,/opt/quickbuild/ As you can see it tries to push the same values for example on disk.usage:
2017-11-21 15:11:51,368 [MeasurementPollingTask] ERROR com.google.common.eventbus.EventBus.QuickBuild Application - Could not dispatch event: Node: [quickbuild-master-dev:8810], beginTime: [Tue N ov 21 15:11:51 UTC 2017], endTime: [Tue Nov 21 15:11:51 UTC 2017] source=[quickbuild-master-dev:8810], metricName=[steps.waitingsTotal], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.rest.fiveMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] ----> source=[quickbuild-master-dev:8810], metricName=[disk.usage./], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.08526666755430508] source=[quickbuild-master-dev:8810], metricName=[builds.successfulRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[cpu.totalUsage], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.017573467674223343] source=[quickbuild-master-dev:8810], metricName=[buildrequests.processedTotal], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.download.oneMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[2.5812899986304503] source=[quickbuild-master-dev:8810], metricName=[disk.frees./opt/quickbuild], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[5.8174328832E10] source=[quickbuild-master-dev:8810], metricName=[builds.total], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[jvm.threadCount], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[89.0] source=[quickbuild-master-dev:8810], metricName=[web.upload.averageDuration], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.download.fifteenMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[4.900601942442881] source=[quickbuild-master-dev:8810], metricName=[memory.usage], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.5146279402453038] source=[quickbuild-master-dev:8810], metricName=[web.rpc.fiveMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[23.193595324912707] source=[quickbuild-master-dev:8810], metricName=[web.upload.fiveMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[steps.successfulOnNode], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[builds.failed], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.rpc.oneMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[32.33846350312917] source=[quickbuild-master-dev:8810], metricName=[web.download.fiveMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[9.038988781852984] source=[quickbuild-master-dev:8810], metricName=[jvm.gcTime], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.ui.fiveMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[3.14361176247683] source=[quickbuild-master-dev:8810], metricName=[web.ui.oneMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.49498770976847883] source=[quickbuild-master-dev:8810], metricName=[jvm.gcRuns], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.download.averageDuration], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[22.24468085106383] ----> source=[quickbuild-master-dev:8810], metricName=[disk.usage./], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.08526666755430508] source=[quickbuild-master-dev:8810], metricName=[network.txBytes], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[7.6817594E7] source=[quickbuild-master-dev:8810], metricName=[web.upload.oneMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[db.busyConnections], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[buildrequests.processingsTotal], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[steps.finishedOnNode], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[builds.averageWaitingDuration], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.rest.fifteenMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[jvm.memoryUsage], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.1732642578193656] source=[quickbuild-master-dev:8810], metricName=[web.rest.requests], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[disk.usage./opt/quickbuild], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.08526666755430508] source=[quickbuild-master-dev:8810], metricName=[web.rpc.fifteenMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[10.384748366379377] source=[quickbuild-master-dev:8810], metricName=[web.ui.fifteenMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[20.470406496636745] source=[quickbuild-master-dev:8810], metricName=[web.upload.fifteenMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.rest.oneMinuteRate], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[jetty.totalConnections], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[8.0] source=[quickbuild-master-dev:8810], metricName=[disk.usage./opt/quickbuild], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.08526666755430508] source=[quickbuild-master-dev:8810], metricName=[disk.frees./opt/quickbuild], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[5.8174328832E10] source=[quickbuild-master-dev:8810], metricName=[web.rest.averageDuration], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[buildrequests.submittedTotal], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.upload.requests], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.rpc.requests], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[182.0] source=[quickbuild-master-dev:8810], metricName=[jetty.usedConnections], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[3.0] source=[quickbuild-master-dev:8810], metricName=[db.numConnections], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[5.0] source=[quickbuild-master-dev:8810], metricName=[disk.frees./], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[5.8174328832E10] source=[quickbuild-master-dev:8810], metricName=[steps.runningsTotal], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[0.0] source=[quickbuild-master-dev:8810], metricName=[web.ui.requests], timestamp=[Tue Nov 21 15:11:51 UTC 2017], value=[9.0] We are using dockerized approach:
so everything is united under docker with Linux 4.4.0-83-generic #106~14.04.1-Ubuntu SMP Mon Jun 26 18:10:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux installed on it For QB 6.0.x it was working without problems, currently even when using clear DB, no agents connected , I can see that measurements from the master node are not working I have just started QB on my own mashine, with clean DB, and have the same problem:
java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74) at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) at com.google.common.eventbus.EventBus.dispatch(EventBus.java:313) at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296) at com.google.common.eventbus.EventBus.post(EventBus.java:267) at com.pmease.quickbuild.plugin.measurement.core.poller.MeasurementPollingTask.execute(MeasurementPollingTask.java:57) at com.pmease.quickbuild.plugin.measurement.core.poller.MeasurementPollingTask.run(MeasurementPollingTask.java:109) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) at java.lang.Thread.run(Thread.java:748) Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:72) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:211) at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:62) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3124) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3581) at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:104) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351) at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350) at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56) at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258) at com.pmease.quickbuild.entitymanager.impl.DefaultMeasurementDataManager.flushAndClear(DefaultMeasurementDataManager.java:86) at com.pmease.quickbuild.entitymanager.impl.DefaultMeasurementDataManager.save(DefaultMeasurementDataManager.java:113) at com.pmease.quickbuild.plugin.measurement.core.reporter.MeasurementServerReporter.save(MeasurementServerReporter.java:105) at com.pmease.quickbuild.plugin.measurement.core.reporter.MeasurementServerReporter.run(MeasurementServerReporter.java:76) at com.pmease.quickbuild.plugin.measurement.core.repor ok, I will try to reproduce it on our staging with a fresh DB, should I import the QB backup? or not?
We cannot do that - it is a production environment, and we need all of the data which is stored there already
I already tried:
- restart it, - clean tables, restore them, restart QB - start a fresh instance with the same DB connected Neither of those work After manually removing contraints from DB, I can see that most of the rows are pushed twice with the same value:
mysql> select * from QB_MEASUREMENT_DATAR03; +-------+----------------------------------------------------+-------------------------------------+---------------+---------------------------+ | QB_ID | QB_METRIC_NAME | QB_SOURCE | QB_TIMESTAMP | QB_VALUE | +-------+----------------------------------------------------+-------------------------------------+---------------+---------------------------+ | 103 | jvm.memoryUsage | quickbuild-agent-X:8811 | 1511272791582 | 0.06191427686063185 | | 104 | disk.usage./opt/buildagent | quickbuild-agent-X:8811 | 1511272791583 | 0.6052124435108935 | | 105 | disk.usage./opt/buildagent | quickbuild-agent-X:8811 | 1511272791583 | 0.6052124435108935 | | 106 | disk.usage./opt/buildagent | quickbuild-agent-X:8811 | 1511272791583 | 0.6052124435108935 | | 107 | cpu.totalUsage | quickbuild-agent-X:8811 | 1511272791583 | 0.0035431134379871783 | | 108 | disk.usage./ | quickbuild-agent-X:8811 | 1511272791583 | 0.6052124435108935 | | 109 | disk.frees./ | quickbuild-agent-X:8811 | 1511272791583 | 261964926976 | | 110 | disk.frees./opt/buildagent | quickbuild-agent-X:8811 | 1511272791583 | 261964926976 | | 111 | disk.frees./opt/buildagent | quickbuild-agent-X:8811 | 1511272791583 | 261964926976 | |