Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi 1.0.0.2.0.0.0-579 PutSql to mysql db with MySQLNonTransientConnectionException

NiFi 1.0.0.2.0.0.0-579 PutSql to mysql db with MySQLNonTransientConnectionException

Explorer

HI

We are working with 3 NiFi nods of HDF. We import data to MySql with PutSql processor of NiFi. It is often that the PutSql processors report the below exceptions. The records don't lost since the PutSql processor put the flow files of unhandled to Relationship.SELF. I checked network that is normal state. And there is no errors at mysql server side.

Could you give me any advise what is the cause?

Thanks.

2017-05-04 13:40:58,029 INFO [Timer-Driven Process Thread-96] o.a.n.r.ambari.AmbariReportingTask AmbariReportingTask[id=3b80ba0f-a6c0-48db-b721-4dbc04cef28e] Successfully sent metrics to Ambari in 0 ms
2017-05-04 13:40:58,351 ERROR [Timer-Driven Process Thread-82] o.apache.nifi.processors.standard.PutSQL PutSQL[id=e9eb5b00-015a-1000-9a6e-08618ca8d49a] Failed to update database for [] due to com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.; routing to failure: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
2017-05-04 13:40:58,356 ERROR [Timer-Driven Process Thread-82] o.apache.nifi.processors.standard.PutSQL 
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
	at sun.reflect.GeneratedConstructorAccessor493.newInstance(Unknown Source) ~[na:na]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_92]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_92]
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[na:na]
	at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[na:na]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013) ~[na:na]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987) ~[na:na]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982) ~[na:na]
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927) ~[na:na]
	at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1665) ~[na:na]
	at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334) ~[na:na]
	at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:211) ~[na:na]
	at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:371) ~[nifi-standard-processors-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_92]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_92]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_92]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_92]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_92]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_92]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92]
3 REPLIES 3

Re: NiFi 1.0.0.2.0.0.0-579 PutSql to mysql db with MySQLNonTransientConnectionException

Explorer

any update? is this a NIFI bug?

Re: NiFi 1.0.0.2.0.0.0-579 PutSql to mysql db with MySQLNonTransientConnectionException

Super Mentor

@Paul Yang

This error seems to be more MySQL specific. Due to some reasons MySQL is not able to commit the reansactions. Looking at the MySQL logs of the same timestamp will give more details on this. Restarting MySQL can be a quick check to isolate the issue.

- Also checking the MySQL JDBC driver version (mysql-connector-java) is upto date or not also would be good to isolate the issue.

017-05-04 13:40:58,356 ERROR [Timer-Driven Process Thread-82] o.apache.nifi.processors.standard.PutSQL 
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
   at sun.reflect.GeneratedConstructorAccessor493.newInstance(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_92]
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_92]
   at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[na:na]
   at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[na:na]

.

Is this error occurring every time (very frequently) or intermittently?

Highlighted

Re: NiFi 1.0.0.2.0.0.0-579 PutSql to mysql db with MySQLNonTransientConnectionException

Explorer

@Jay SenSharma

This error is occurring intermittently. But It is frequently when triggered. I cannot found any error logs on MySql (master or slave) Server.

Don't have an account?
Coming from Hortonworks? Activate your account here