@Hariprasanth Madhavan There are lot of ways to insert data into `HiveORC` table from NiFi. Method1: Using PutHiveStreaming Processor: Create transactional table and then feed the avro data to PutHivestreaming table. As HiveStreaming processor converts the avro format data into ORC format and regards to all delta files you can use major compaction to create one base file. Method2: ConvertAvroToORC in NiFi and store into HDFS: Use ConvertAvroToORC processor to convert the avro format data into ORC format. Store the data into HDFS and create an External hive table pointing to the same HDFS directory. Method3: Create Avro table and load from Avro table to ORC table: Based on the avro file we are having in NiFi we can create avro tables dynamically based on avro.schema. Create an orc table and after storing the avro data into HDFS use PutHiveQL processor to run insert into ORC table select * from Avro table Refer to this link for more details regards to create avro table dynamically. - If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues. ... View more

@Hariprasanth Madhavan Nifi on start , tries to launch a bootstrap process which tries to connect to ports available on the server(host). In your case, it was trying to connect to port(42511) in first instance of startup and port(34170) in second instance of start up. But ports seems to have blocked, hence you are getting connection refused. Please check your firewall settings. ... View more

Please check out below KB articles https://community.hortonworks.com/articles/46258/iot-example-in-apache-nifi-consuming-and-producing.html https://community.hortonworks.com/articles/178747/mqtt-with-apache-nifi.html Note: Please upvote and accept this answer if you found it useful ... View more

@Hariprasanth Madhavan PutHiveQL processor is used to: Executes a HiveQL DDL/DML command (UPDATE, INSERT, e.g.). The content of an incoming FlowFile is expected to be the HiveQL command to execute. - If you want to insert data into hive table directly then use PutHiveStreaming processor instead of PutHiveQL. Puthivestreaming processor expects the incoming data in AVRO format and table needs to Transactional enabled, so based on the KafkaConsumer format of data use ConvertRecord processor to Convert the source data into AVRO format then feed the Avro data into PutHiveStreaming processor. Flow: 1.ConsumeKafka 2.ConvertRecord //convert the outgoing flowfile into AVRO format 3.PutHiveStreaming Refer to this link for hive transactional tables and this link for ConvertRecord processor usage. - If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues. ... View more