NIFI Install

[#1]

We are currently using sqoop and Hive sql to ingest data.Can we use NIFI to ingest data into HDFS and load data into Hive tables.Is it possible to Install NIFI on a windows Server and schedule jobs to load data into HDFS and Hive?

[#2]

@vishal dutt

NiFi has many processor components to include those capable of writing to both Hive and HDFS.

NiFi can be installed on Windows Server. NiFi itself is only dependent on Java 8. So pretty much anywhere you can run Java 8, you can run NiFi.

Thank you,

Matt

[#3]

@vishal dutt

The HDF version of NiFi comes with some modifications to some processor components to make sure they are compatible with the specific versions of services running in specific HDP installs.

-

You don't need a license to download and use HDF, but installing HDF on top of your HDP may affect your HDP support license.

-

So I recommend keeping those install separate.

-

Thanks,

Matt

[#4]

You can use NiFi to fetch data from RDBMS and store it in HDFS/Hive. There can be multiple examples you can have online for that. That's a tried and tested pattern used by many organizations.

Regarding your requirement to configure NiFi on the windows server, what is the driving factor for that? Not that NiFi can't be installed, but just curious about your use case.

[#5]

We have HDP license , but NIFI complete install(on the cluster ) would require HDF license. We have to ingest files into HDFS and Incrementally pull data from sqlserver and load to Hive. Currently we are using custom sqoop and hive sql i wanted to look at the option of using NIFI standalone on windows server to Perform data Ingestion & scheduling .