Member since
01-03-2018
89
Posts
3
Kudos Received
0
Solutions
06-12-2018
11:36 AM
Yes, the issue was resolved, as we were missing to mention, select query in SelectHiveQL.
... View more
06-02-2018
09:25 AM
Followed the same but facing issues. GetFetchTable_Propoerties SelectHiveQL_Properties PutHiveStreaming_Properties Errors are SelectHiveQL and PutHiveStreaming
... View more
06-01-2018
08:06 PM
Bit more info; Source table, is simple Hive External table Target table, is Hive Partitioned+Clustered with Bucketing and ORC format.
... View more
06-01-2018
07:47 PM
1 Kudo
Thanks for quick response but as per this processors details, its not recommended for production, plus it will not help to do CDC (Change Data Capture). - GenerateFlowFile 1.2.0.3.0.2.0-76 org.apache.nifi - nifi-standard-nar This
processor creates FlowFiles with random data or custom content.
GenerateFlowFile is useful for load testing, configuration, and
simulation. There is an other processor: GenerateTableFetch, it will do the same, asks for database name and tables names via Controller Services, but without any query and will also help in incremental data pull, which is great to have for CDC. Next and last is processor: PutHiveQL, which does not ask fpr target database or table name. So how to push data into target hive tables using source processor as GenerateTableFetch. Please advice.
... View more
06-01-2018
06:49 PM
1 Kudo
Hi, How to load Hive managed table from Hive external table using NiFi? We have used NiFi --> Processor --> SelectHiveQL to pull data from Hive External table. One of its property i.e. Output Format have only 2 options Avro & CSV, we selected Avro. If we use CSV, it ask all the ingredients to generate a CSV, which we don't want to, as we need to load into Hive, so we selected Avro. Secondly, we are used NiFi --> Processor --> PutHiveQL to load into Hive, by giving all the required details, but it give error: Can't recognise Avro format. Looking forward. Cheers.
... View more
Labels:
- Labels:
-
Apache NiFi
05-29-2018
09:44 PM
Found the missing pieces. Go to Configuration (not configure) and added all Controller Services related to Avro and CSV (if you are loading from source). Issue resolved.
... View more
05-29-2018
09:44 PM
Thanks. You response helped, to start thinking in the right direction.
... View more
05-29-2018
04:57 PM
Hi, We are loading CSV file (1 rows for testing purpose) into Hive using NIFI. Processor: ConvertCSVToAvro is unable to find Records Schema generated by Processor: InferAvroSchema. InferAvroSchema --> CSV Header Definition is manually entered.
Is there any other way, we ignore Avro and directly load CSV or any other format of source files into Hive Table? ------- Any yes, we have also tried to ignore Processor: InferAvroSchema and use Processor: ConvertCSVToAvro by creating xyz.avsc file manually, but then we are with error: unable to find xyz.avsc file. inferavroschema.jpeg convertcsvtoavro.jpeg Looking forward.
... View more
Labels:
- Labels:
-
Apache NiFi
05-10-2018
05:53 PM
Upgraded the Firefox to latest version using following command: yum update firefox ------------------ Another error came in the process i.e. Failed to start Flow Service due to: java.net.BindException: Address already in use (Bind failed) Fixed it by changing NiFi protocol port from 9988 to 9989 Issue closed.
... View more
05-10-2018
04:07 AM
Firefox: 17.0.10 NiFi: 1.2.0 (HDF 3.0.2) HDP: 2.6.3.0 ------------------------------------- TypeError: URL is not a constructor
... View more
- « Previous
- Next »