Support Questions

aliyesami · ‎03-07-2018

please see the attached error , i am feeding it JSON data but its complaining about AVRO format.

Shu_ashu · ‎03-08-2018

@Sami Ahmad

Output of QueryDatabaseTable processor are always in avro format , so you need to use PutHiveStreaming processor after Querydatabasetable processor.

As PutHiveStreaming processor expects incoming data to be in avro format and we are getting incoming data from querydatabasetable in avro format.

Flow:-

1.QueryDatabasetable
2.PutHiveStreaming
3.LogAttribute

please refer to below links regarding table creation of puthivestreaming processor

https://community.hortonworks.com/questions/59411/how-to-use-puthivestreaming.html
https://community.hortonworks.com/articles/52856/stream-data-into-hive-like-a-king-using-nifi.html

View solution in original post

aliyesami · ‎03-20-2018

db-record.zipi think its the data issue , I am uploaded a queue entry please take a look.

the table ddl is as follows :

CREATE TABLE purchase_acct_orc (
acct_num BIGINT,
pur_id BIGINT,
pur_det_id BIGINT,
product_pur_product_code STRING,
prod_amt FLOAT,
accttype_acct_type_code STRING,
acctstat_acct_status_code STRING,
emp_emp_code STRING,
plaza_plaza_id STRING,
purstat_pur_status_code STRING
)
PARTITIONED BY (pur_trans_date TIMESTAMP)
CLUSTERED BY(acct_num) INTO 5 BUCKETS
STORED AS ORC
TBLPROPERTIES ("transactional"="true")

aliyesami · ‎03-21-2018

why do I have "LogAttribute" with QueryDatabasetable processor? its showing a red block on the corner which normally indicates issues I think.

from QueryDatabaseTable there are two relations one going to PutHiveStreaming and one going to LogAttribute . why do I need two relationships for success ?
(please see attached)

capture.jpg

aliyesami · ‎03-08-2018

puhive-after-querydatabase.jpg

I was told by the HW engineer who setup this initial setup that putHiveStreaming processor is not able to write into hive table cause the data is not in JSON format and I need to convert it to JSON first .. ?

please see the attached error when I put HiveStreaming after the QueryDatabase processor.

TimothySpann · ‎03-08-2018

What is the Hive table DDL?

It needs to be bucketed and an ORC file.

See my example:

https://community.hortonworks.com/articles/174538/apache-deep-learning-101-using-apache-mxnet-with-h...

TimothySpann · ‎03-08-2018

should be Avro with a schema.

Get a document or example code from an engineer

aliyesami · ‎03-12-2018

my hive table is bucketed and in ORC format.

so why am I getting error ? and what does this error means?

Cloudera Community

Support Questions

NIFI putHiveStreaming processor error