Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

import a large table from hive in nifi using SelectHiveQl

import a large table from hive in nifi using SelectHiveQl

New Contributor

I want to import large table in nifi from hive . I am trying using SelectHiveQL but the processor gets stuck as the data volume is around 280 million rows .

1 REPLY 1
Highlighted

Re: import a large table from hive in nifi using SelectHiveQl

Contributor

Hi,

You want to extract those entire 280 millions in a single call ? not a good idea ! I suggest you loop thru it using the table partition + bucket keys(for better query response on Hive).

What is you table construct ?

Based on this you can run SelectHiveQL in parallel or sequentially thru a loop.

A once extract won`t do.

Thx

Don't have an account?
Coming from Hortonworks? Activate your account here