cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted

Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Hi everyone!

Currently, I am using ePO 5.3.3. I created a server task "Purge Server Task Log" to purge records older than x days. When I manually click "Run" to run task, it cannot purge and server task log return status fail. I checked orion.log on ePO server and got the following error

2019-01-23 17:44:38,148 ERROR [scheduler-TaskQueueEngine-thread-22] log.TaskLogPurgeService - SQL error when trying to purge log records: Transaction (Process ID 69) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
java.sql.SQLException: Transaction (Process ID 69) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:614)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:573)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.execute(JtdsPreparedStatement.java:787)
at sun.reflect.GeneratedMethodAccessor625.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.mcafee.orion.core.db.sqlserver.JtdsStatementRetryInvocationHandler.invoke(JtdsStatementRetryInvocationHandler.java:87)
at com.sun.proxy.$Proxy17.execute(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at com.mcafee.orion.task.log.TaskLogPurgeService.purgeRecordsBasedOnAge(TaskLogPurgeService.java:83)
at com.mcafee.orion.scheduler.command.PurgeTaskLogCmd.invoke(PurgeTaskLogCmd.java:96)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:999)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:968)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:945)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:437)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:474)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:383)
at com.mcafee.orion.scheduler.chainable.Chain.invoke(Chain.java:64)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runTask(ScheduledTaskManagerImpl.java:1577)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1548)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1502)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.execute(ScheduledTaskManagerImpl.java:1313)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:913)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:895)
at com.mcafee.orion.task.queue.TaskQueueEngine.access$1000(TaskQueueEngine.java:50)
at com.mcafee.orion.task.queue.TaskQueueEngine$3.run(TaskQueueEngine.java:864)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I am not got at database design and I don't understand the reason. I hope everyone can help me find out how to fix this error.

Thank you so much

1 Solution

Accepted Solutions
McAfee Employee cdinet
McAfee Employee
Report Inappropriate Content
Message 11 of 19

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

No, do not mess with that at all please.  I would highly recommend opening a ticket with McAfee so we can see what is causing the deadlocks.  There are blocking research queries we can run and also look at software versions that may have known issues with it.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

18 Replies
McAfee Employee hem
McAfee Employee
Report Inappropriate Content
Message 2 of 19

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

I suggest to open case with support. We may ask data as per KB60316 (can be accssed after logging into portal).

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?
McAfee Employee cdinet
McAfee Employee
Report Inappropriate Content
Message 3 of 19

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

You might want to check to see how fragmented the database is and run the reindex script that is in KB67184.  I would recommend to do this with all epo services off to start with, then use it in your regular maintenance.  If your db is highly fragmented, that can cause this, though there are other causes.

If that doesn't resolve, then definitely open a case with support.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Dear hem, Dear cdinet

I tried to defragment my ePO database by using script "RebuildReorganizeIndexes-V4.zip" attached on KB67184.

After defragmentation success, I manually run "Purge Server Task Log" again but task stuck at 75% and Duration: 16 hours 13 minutes. I looked into orion log of ePO server and I saw the error:

2019-01-24 17:33:10,554 ERROR [scheduler-InternalTask-thread-2] pip.TelemetryUploader - Could not get orion server properties for Iam CredentialsInvalid input.
2019-01-24 17:33:10,586 ERROR [scheduler-InternalTask-thread-2] pip.TelemetryUploader - Not able to fetch access token, please check if IAM server is reachable

Purger_Server_Task_Log.png

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Until now, my purge task finished but it failed. I checked orion.log and got the following error:

 

2019-01-25 13:53:05,044 ERROR [scheduler-TaskQueueEngine-thread-26] log.TaskLogPurgeService - SQL error when trying to purge log records: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_OrionSchedulerTaskLog_OrionSchedulerTaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionSchedulerTaskLogMT", column 'ParentId'.
java.sql.SQLException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_OrionSchedulerTaskLog_OrionSchedulerTaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionSchedulerTaskLogMT", column 'ParentId'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:614)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:573)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.execute(JtdsPreparedStatement.java:787)
at sun.reflect.GeneratedMethodAccessor713.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.mcafee.orion.core.db.sqlserver.JtdsStatementRetryInvocationHandler.invoke(JtdsStatementRetryInvocationHandler.java:87)
at com.sun.proxy.$Proxy17.execute(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at com.mcafee.orion.task.log.TaskLogPurgeService.purgeRecordsBasedOnAge(TaskLogPurgeService.java:83)
at com.mcafee.orion.scheduler.command.PurgeTaskLogCmd.invoke(PurgeTaskLogCmd.java:96)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:999)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:968)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:945)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:437)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:474)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:383)
at com.mcafee.orion.scheduler.chainable.Chain.invoke(Chain.java:64)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runTask(ScheduledTaskManagerImpl.java:1577)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1548)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1502)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.execute(ScheduledTaskManagerImpl.java:1313)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:913)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:895)
at com.mcafee.orion.task.queue.TaskQueueEngine.access$1000(TaskQueueEngine.java:50)
at com.mcafee.orion.task.queue.TaskQueueEngine$3.run(TaskQueueEngine.java:864)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

I tried to restart McAfee ePO service (Event Parser; Server and Application Server) and SQL server service. After that, I purge server task log again and it worked. I can purge server task log after x days.

But I don't think this is best solution. I hope anybody can give me the best suggesstion or I hope this can be fixed on the next update if this is a bug.

McAfee Employee cdinet
McAfee Employee
Report Inappropriate Content
Message 7 of 19

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Check KB's 84114 and 83384 to ensure the dbcleanup task is running properly.  That can cause all kind of issues and a restart of services temporarily clears things up.  The issues in those kb's can still occur.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution
  •  Following the KB84114: 

- When I running the second query: SELECT * FROM OrionTaskQueueMT WHERE TaskDescription LIKE '%DBCleanUp%'

The return result look like this:

Second_Query.png

- The first query: SELECT Id, UniqueKey, Enabled FROM OrionSchedulerTaskMT WHERE UniqueKey = 'DbCleanupTask'

Return the result:

First_Query.png

I think everything are correct. 

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

You are correct. Restart services is only temporary solution. I run purge task log and it's failed again. Orion.log on ePO server still show the error:

2019-01-29 10:57:59,554 ERROR [scheduler-TaskQueueEngine-thread-14] log.TaskLogPurgeService - SQL error when trying to purge log records: The DELETE statement conflicted with the REFERENCE constraint "FK_OrionTaskQueue_TaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionTaskQueueMT", column 'TaskLogID'.
java.sql.SQLException: The DELETE statement conflicted with the REFERENCE constraint "FK_OrionTaskQueue_TaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionTaskQueueMT", column 'TaskLogID'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

When I run query: SELECT * FROM OrionTaskQueueMT

I got the following result:

Query.png

There is a task with NodeUUID and TaskLogID are not equal NULL. Should I update that values to NULL?

ePO Support Center Plug-in
Check out the new ePO Support Center. Simply access the ePO Software Manager and follow the instructions in the Product Guide for the most commonly used utilities, top known issues announcements, search the knowledgebase for product documentation, and server status and statistics – all from within ePO.