About Raj_B

Raj_B · ‎03-09-2017

thanks @Artem Ervits, I'll give it a shot

Raj_B · ‎03-08-2017

thanks @Artem Ervits, but it seems comparing the source and target (by joining them in the same query, for change detection) is not doable in Sqoop's free-form query.

Raj_B · ‎03-07-2017

Thanks @Matt Burgess, how about Sqoop, any options in Sqoop ?

Raj_B · ‎03-07-2017

Hi All, We've some relational tables where there are no audit columns or no max value columns, the way we're figuring out changes in the table is to compare the entire table in source with the corresponding relational table in target and using checksum determine which rows in source have changed or new. This is done via a traditional ETL tool that works with relational databases. If your goal is to do incremental loads from this relational table to a Hive table, can this be done in NiFi or Sqoop and how ? Thanks for your suggestions.

Raj_B · ‎03-07-2017

Hi All, I'm getting this error below while writing data to a Hive table using NiFi's PutHiveStreaming processor. NiFi version is 1.0.0 and HDP is 2.3.4.0; I saw another thread where this error occurs in a Windows environment, but our NiFi and HDP are hosted on Linux. Thanks. 2017-03-07 10:24:09,234 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Timer-Driven Process Thread-9] ERROR org.apache.nifi.processors.hive.PutHiveStreaming - PutHiveStreaming[id=015a1000-2600-129d-1dc2-b20f16685c5c] PutHiveStreaming[id=015a1000-2600-129d-1dc2-b20f16685c5c] failed to process session due to com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient

Raj_B · ‎03-01-2017

Ok great, thanks for letting me know @Matt Clarke

Raj_B · ‎03-01-2017

Any takers ? @Bryan Bende @Timothy Spann @Matt Clarke @Matt Burgess @Andrew Grande

Raj_B · ‎03-01-2017

Awesome, @Jeff Storck, that did the trick. Thank you.

Raj_B · ‎03-01-2017

thanks @Jeff Storck, I removed the hive nar file from the working directory (in my local install it was at ./work/nar/extensions/) and ensured that it was not in the lib directory, but still I couldn't start NiFi. In the log I still the same error as before.

Raj_B · ‎02-28-2017

Yes, we're using ListenTCP; I agree with your recommendation. Our Data Producer is not able to send data to multiple IPs, they can send it to just one IP; so, we're exploring an external load balancer appliance option that sits in front of NiFi Cluster, but I am having this Site-to-Site design as a backup option to the external load balancer (in our case some custom coding needs to be done on our Data Producer side to make load balancing work, so the site-to-site is just a backup in case we have trouble making it work)

Online	Offline
Last Visited	‎08-19-2020 03:25 PM

Member Since	‎04-29-2016 04:49 PM
Last Visited	‎08-19-2020 03:25 PM
Posts	192
Kudos received	20

Cloudera Community

Re: Does NiFi evaluate processor properties (with ...

Re: NiFi's GetHDFS processor with Cron schedule no...

Re: Incremental data loads from relational databas...

Re: Incremental data loads from relational databas...

Re: Incremental data loads from relational databas...

Incremental data loads from relational database ta...

Unable to instantiate org.apache.hive.hcatalog.com...

Re: How to integrate NiFi with Atlas, for metadata...

Re: How to integrate NiFi with Atlas, for metadata...

Re: NiFi startup error

Re: NiFi startup error

Re: Looking for feedback on NiFi Cluster setup