Member since
08-15-2017
1
Post
0
Kudos Received
0
Solutions
08-16-2017
12:28 AM
Hi, I'm stumped by an issue where the NiFi service enters a suspended state whenever I attempt to interact with Hive via beeline --- any help is greatly appreciated. I have NiFi and CDH running in a docker container (I've tried two different published containers and get the same result). If I substitute the database call with a call to mysql, I don't run into any issues. I'm also able to run standard bash commands in the processor (I used an ExecStreamCommand processor fed from a GenerateFlowFile processor; I've also tried the ExecuteProcess processor). I'm able to run this simple command /usr/bin/beeline -u jdbc:hive2://localhost:10000/default --fastConnect=true --silent=true -e "SHOW TABLES;" directly from the CLI. However, if I wrap it in a script (/root/test.sh) and call it from the processors, NiFi locks up completely: 7806 pts/0 T 0:00 /bin/sh /opt/nifi-1.2.0.3.0.0.0-453/bin/nifi.sh start
7808 pts/0 Tl 0:01 /usr/lib/jvm/jdk1.8.0_40/bin/java -cp /opt/nifi-1.2.0.3.0.0.0-453/conf:/opt/nifi-1.2.0.3.0.0.0-453/lib/bootstrap/* ...
7825 pts/0 Tl 1:47 /usr/lib/jvm/jdk1.8.0_40/bin/java -classpath /opt/nifi-1.2.0.3.0.0.0-453/./conf:/opt/nifi-1.2.0.3.0.0.0-453/./lib/javax.servlet-api-3.1.0.jar:...
8001 pts/0 T 0:00 /bin/bash /root/test.sh
8013 pts/0 Tl 0:01 /usr/lib/jvm/jdk1.8.0_40/bin/java -Xmx256m -Dhadoop.log.dir=/usr/lib/...
8014 pts/0 T 0:00 tee -a /tmp/out
8198 pts/0 T 0:00 sh -c stty -icanon min 1 -icrnl -inlcr -ixon < /dev/tty
8199 pts/0 T 0:00 stty -icanon min 1 -icrnl -inlcr -ixon
I've turned on debugging in the log4j files of hive, and there is no hint of a request being received. Furthermore, the log files for NiFi (also debug mode) simple stall when the processes are suspended. I've experimented with this in Centos 6 and Ubuntu 14.04 containers. The problem could well lie with the containerization, but perhaps someone has encountered something similar in their efforts and can steer me in the right direction. Many thanks!
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
-
Docker