Member since
05-24-2017
12
Posts
2
Kudos Received
0
Solutions
05-17-2018
02:23 PM
I also came across this error. I created the kafka_server_jass.conf manually and put it under directory /usr/hdf/current/kafka-broker/config/. And then In Ambari kafka-env template, add the path to file kafka_server_jass.conf to environment variable KAFKA_OPTS like this: export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/hdf/current/kafka-broker/config/kafka_server_jaas.conf" With this settings, kafka broker can start up.
... View more
05-17-2018
02:02 PM
2 Kudos
I think your problem is similar to this one https://community.hortonworks.com/questions/120978/hdp-kafka-service-setup-with-saslplain-not-working.html I did what @Jeff Groves said, it worked. You might have a try.
... View more
12-15-2017
03:01 PM
I am sorry for the last additional picture, because it's the same as the second one. I tried to delete it but failed. If you know how to, please tell me. Thanks!
... View more
12-15-2017
02:53 PM
Hello I wrote a python program to retrieve the status of process group with this API:/flow/process-groups/{id}/status. I found that some connection status in one of nodes is exactly the same with the one in aggregated status. There are 21 nodes in cluster. I think the sum of all the nodes status equals the one in aggregated status, this is what I mean consistent status. For example, the value of bytesQueued in aggregated status should equals the sum of all the value of bytesQueued in the separate node. Could this be a bug or did I have something wrong? Any suggestions? Thanks advanced! This is the connection status in aggregated status: This is the node(bj1175) status: the connection status is the same as the one in aggregated status(last picture) This is the other node(bj1228) status:
... View more
Labels:
- Labels:
-
Apache NiFi
12-15-2017
02:30 PM
@dhieru singh do u mean this [nifi@sz1314 bin]$ export HADOOP_CLIENT_OPTS="-Xmx2048m"
[nifi@sz1314 bin]$ export HADOOP_CLIENT_OPTS="-Xmx2048m"
[nifi@sz1314 bin]$ ./nifi.sh dump
... View more
12-15-2017
02:27 PM
@dhieru singh thanks very much. So it's some kind of feature of PublishKafka 🙂
... View more
12-03-2017
03:57 PM
Hi @Bryan Bende, I encounter this issue again. Unfortunately, I failed to run dump with OOM: [nifi@sz1314 bin]$ ./nifi.sh dump
Java home: /usr/local/jdk1.8.0_45
NiFi home: /usr/hdf/current/nifi
Bootstrap Config File: /usr/hdf/current/nifi/conf/bootstrap.conf
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:421)
at java.lang.StringBuilder.append(StringBuilder.java:136)
at org.apache.nifi.bootstrap.RunNiFi.dump(RunNiFi.java:707)
at org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:232)
Do I need to make a tuning to some setting of NiFi to run dump successfully? Thanks!
... View more
11-17-2017
01:46 PM
@Bryan Bende, thanks very much for your replay. It's a pity that I've already restarted the NiFi node, so I can't get where it was stuck now. When I encounter this problem next time, I'll do what you suggested and post the thread dump here. Thanks!
... View more
11-15-2017
03:39 AM
Hello, I stopp a PublishKafka processor on the Web UI. The processor is still running in 2 hosts of the cluster. I try to find which are the 2 hosts by dragging the connection to the PublishKafka processor to another processor, I get the following message: It shows me that host bj1233 can not fulfill this request, so I check the log in host bj1233. There are two related lines: 2017-11-15 10:05:23,532 INFO [NiFi Web Server-547070] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10
2017-11-15 10:05:23,532 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PublishKafka_0_10[id=6c643549-32fa-1f10-bad1-7a65935d7466] to run It makes me confused: the StandardProcessScheduler has stopped the process, but the UI shows that the processor is running. Did I misunderstand it? How to handle this situation? Any suggestions? Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
05-24-2017
11:13 AM
It's solved by copying file hdfs-site.xml and core-site.xml from directory /etc/hadoop/conf/ to conf directory of NiFi.
... View more