Member since
06-13-2017
45
Posts
2
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2018 | 01-24-2019 06:22 AM | |
3254 | 08-06-2018 12:05 PM |
12-11-2019
01:45 AM
@MattWho As your advice, I have created the said "state" folder under "conf", however, it raised the same error。
... View more
12-10-2019
01:53 AM
@MattWho I am running nifi 1.10 in windows10 with admin account and there is the said "state" directory: here is the full log about the s2s error: 2019-12-09 20:39:13,124 DEBUG [Timer-Driven Process Thread-10] org.apache.nifi.engine.FlowEngine A flow controller execution task 'java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@5a8bd0fa' has been cancelled. 2019-12-09 20:39:13,126 DEBUG [NiFi Web Server-17] org.apache.nifi.web.filter.TimerFilter GET http://localhost:8080/nifi-api/site-to-site/peers from 127.0.0.1 duration for Request ID null: 2 millis 2019-12-09 20:39:13,129 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@f1ae5b8 Unable to refresh Remote Group's peers due to null 2019-12-09 20:39:13,129 WARN [Http Site-to-Site PeerSelector] o.a.n.r.SiteToSiteBulletinReportingTask SiteToSiteBulletinReportingTask[id=ab75497c-016e-1000-c86b-2763551bcb01] org.apache.nifi.remote.client.PeerSelector@f1ae5b8 Unable to refresh Remote Group's peers due to null 2019-12-09 20:39:13,129 DEBUG [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector java.lang.NullPointerException: null at org.apache.nifi.remote.client.PeerSelector.persistPeerStatuses(PeerSelector.java:112) at org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:306) at org.apache.nifi.remote.client.http.HttpClient$2.run(HttpClient.java:86) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2019-12-09 20:39:13,132 DEBUG [Timer-Driven Process Thread-5] o.a.n.controller.tasks.ConnectableTask Triggering StandardPublicPort[id=db33781e-016e-1000-03bb-ceb910fd0a45,name=ErrorIn]
... View more
12-07-2019
03:01 AM
hi there,
I have created a S2S reporting task to the same instance of nifi:
After enabing it, the port "Bulletin" can get error message however the bulletin board raise warining the message continueously :
2019-12-06 20:35:13,274 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@7a616d8a Unable to refresh Remote Group's peers due to null
Why and any idea to fix it?
... View more
Labels:
- Labels:
-
Apache NiFi
06-06-2019
11:06 AM
after restart the ambari-agent service, i can start the zk , hdfs, mr, hive , spark2 now. thanks @Jay Kumar SenSharma
... View more
06-06-2019
10:37 AM
i also tried to use "ambari-admin-password-reset" to set admin password then login ambari with admin, then repeat the start service actions, but no luck to start hdp services..
... View more
06-06-2019
10:27 AM
While trying to install HDP 2.6.5 sandbox on windows 10 with wmare, after starting the vm with 4cpu and 10g ram, i can login the ambari but all service can't be started
I have tried "restart all" service or restart zookeeper via the account: raj_ops, but no luck.
... View more
Labels:
01-24-2019
06:22 AM
fixed after adding the edgenode into the datanode /etc/hosts
... View more
01-23-2019
03:51 PM
hi there, Below spark-sql comand failed to run on an edgenode with only spark-client installed spark-sql --executor-cores 2 --num-executors 2 -e 'select profitcenter, count(1) from default.tableX group by profitcenter' --master yarn --name spark_sql_test1 --verbose --queue default The error log is: 19/01/23 19:18:24 ERROR ApplicationMaster: Failed to connect to driver at emr-header-1.cluster-92503:35638, retrying ...
19/01/23 19:18:24 ERROR ApplicationMaster: Uncaught exception:
org.apache.spark.SparkException: Failed to connect to driver!
at org.apache.spark.deploy.yarn.ApplicationMaster.waitForSparkDriver(ApplicationMaster.scala:672)
at org.apache.spark.deploy.yarn.ApplicationMaster.runExecutorLauncher(ApplicationMaster.scala:532)
at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$runImpl(ApplicationMaster.scala:347)
at org.apache.spark.deploy.yarn.ApplicationMaster$anonfun$run$2.apply$mcV$sp(ApplicationMaster.scala:260)
at org.apache.spark.deploy.yarn.ApplicationMaster$anonfun$run$2.apply(ApplicationMaster.scala:260)
at org.apache.spark.deploy.yarn.ApplicationMaster$anonfun$run$2.apply(ApplicationMaster.scala:260)
at org.apache.spark.deploy.yarn.ApplicationMaster$anon$5.run(ApplicationMaster.scala:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1727)
at org.apache.spark.deploy.yarn.ApplicationMaster.doAsUser(ApplicationMaster.scala:814)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:259)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:839)
at org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:869)
at org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
19/01/23 19:18:24 INFO ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: Uncaught exception: org.apache.spark.SparkException: Failed to connect to driver!)
19/01/23 19:18:24 INFO ShutdownHookManager: Shutdown hook called
End of LogType:stderr emr-header-1.cluster-92503 is the hostname of the edgenode.. The command works fine if change it to local[*] Would some one has any ideas? Thanks Forest
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
08-30-2018
07:38 AM
thanks your hints @euricana . it really caused by server time is 5 min earlier than host time
... View more
08-30-2018
07:29 AM
The same program runs succefully on an edgenode with hive2 client.
... View more