Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Which tool to schedule Hive queries?

Solved Go to solution

Which tool to schedule Hive queries?

Rising Star

Hi,

I would like to create/update Hive tables let's say once per hour.

Which tool should i use to schedule the execution of my hql script?

Thank you in advance

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Which tool to schedule Hive queries?

Hi @Lubin Lemarchand, for simple, single script, independent jobs you can also use cron. For jobs consisting of 2 or more scripts you can use Oozie which is the scheduling tool of choice in Hadoop ecosystem. Initially, Oozie didn't support Hive but now it does together with Pig, MR, Sqoop, Spark and other, so-called "actions" or steps. Long term, it's definitely worth learning more about Oozie: quick start.

6 REPLIES 6

Re: Which tool to schedule Hive queries?

New Contributor

why can't you use oozie time based workflow.

Re: Which tool to schedule Hive queries?

Rising Star

Well, most of the tutorials i found on Oozie were written 2 or 3 years ago so i was wondering if it was still the recommended tech for this kind of things.

Re: Which tool to schedule Hive queries?

Guru

You can also use falocn with hive action.

Re: Which tool to schedule Hive queries?

Hi @Lubin Lemarchand, for simple, single script, independent jobs you can also use cron. For jobs consisting of 2 or more scripts you can use Oozie which is the scheduling tool of choice in Hadoop ecosystem. Initially, Oozie didn't support Hive but now it does together with Pig, MR, Sqoop, Spark and other, so-called "actions" or steps. Long term, it's definitely worth learning more about Oozie: quick start.

Re: Which tool to schedule Hive queries?

Mentor

you also invested time in learning nifi, nifi has a cron based scheduler, would be interesting to see it leveraged for this use case. Look under scheduling tab https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Configuring_a_Processor

Re: Which tool to schedule Hive queries?

Rising Star

Yes i use it to request the API i'm interested in once a week (except for the twitter streaming api of course).

Don't have an account?
Coming from Hortonworks? Activate your account here