Unable to fetch data from hive table using Apache NiFi (ExecuteSQL processor)

[#1]

Hi,

I am trying to fetch data from hive table (ExecuteSQL) and load into file system(PutFile). I have kept all required hive

jar in lib folder of nifi. However getting Error message in ExecuteSQL processor. Please let me know if i have done any mistake anywhere and help me to complete the job.

[#2 · Accepted solution]

If you are using HDF 1.2, unfortunately the ExecuteSQL processor doesn't work with hive, yet. The Hive processor is on the roadmap and may be included in the next release.

[#3]

Just an update - the 'SelectHiveQL' has been added as part of Nifi 0.7

[#4]

There must be a new nifi processor 'SelectHiveQL' that queries from hive. Also, there is a processor now to insert or update data directly to hive 'PutHiveQL'

[#5]

HI @hduraiswamy, i need to know the steps involved in upgrading the HDF nifi(version 0.6.0.1.2.0.1-1) to to 0.7 version. My current version doesn't have 'SelectHiveQL' and 'PutHiveQL' processors. Pls share...it helps lot

[#6]

@Iyappan Gopalakrishnan

Follow the below steps:

• save your hdf flow files to xml templates
• download the nifi 0.7 from apache nifi downloads site (https://nifi.apache.org/download.html)
• unzip the file, edit the port (if you would like) and start nifi
• import the templates

If this answer and comment is helpful, please upvote my answer and/or select as best answer. Thank you!!

[#7]

Hi @hduraiswamy

Thanks for your replay

• My issue is how to start 0.7 nifi. I'm having HDF 0.6.0.1.2.0.1-1 version and downloaded from Hortonworks.
• If i need to start my Nifi is \bin\run-nifi.bat
• Here(0.7) not having same file structure of 0.6.0.1.2.0.1-1
• Please explain briefly

Thanks

Iyappan

[#8]

@Iyappan Gopalakrishnan

• Download the nifi-0.7.0-bin.zip file from the downloads link https://nifi.apache.org/download.html
• After that, if you unzip the file, you will see the folder structure similar to this one below:

• Then based on the OS, you can either use 'bin/run-nifi.bat' for windows or 'bin/nifi.sh start' for mac/linux. More details on how to start nifi is here https://nifi.apache.org/docs/nifi-docs/html/getting-started.html#starting-nifi
• You can tail the logs from logs/nifi-app.log (to see if it starts properly)
• OPTIONAL: By default, nifi starts on port 8080 - but if you see any port conflict or want to start this on a different port, you can change that by editing the file 'conf/nifi.properties', search for 8080 and update the port number.

If you like the answer, please make sure to upvote or accept the answer.

[#9]

Thanks @hduraiswamy

I made mistake on download itself now its working fine......Thanks once again...

[#10]

---
- name: Check and start https service
hosts: your_server_hostname
become: true

tasks:
- name: Check https service status
systemd:
name: httpd
state: started
enabled: true
register: service_status

- name: Start https service if stopped
systemd:
name: httpd
state: started
loop_control:
label: "Starting https service on {{ inventory_hostname }}"
when: service_status.changed | default(false)

[#11]

Here's what the playbook does:

1. The hosts line specifies the hostname of the server you want to check and start the httpd service on.
2. become: true ensures that the playbook runs with root privileges, which is necessary to start the service.
3. The tasks section of the playbook contains two tasks.
4. The first task checks the status of the httpd service using the systemd module. If the service is not started, the state parameter sets the service to started. The enabled parameter ensures that the service will start automatically when the system boots up. The register parameter stores the result of this task in a variable called service_status.
5. The second task starts the httpd service using the systemd module, but only if the service_status.changed variable from the first task shows that the service was not already started. The when parameter specifies this condition.
6. The loop_control parameter adds a label to the loop, which will be used to identify the server where the https service is being started.