Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to determine whether a hive script fails?

avatar
Super Collaborator

I have a shell script that calls a series of hive .sql scripts.

If a hive operation fails, I want the shell script to stop and exit.

How can I accomplish this?

Does "hive -f" have return codes that I can check in bash?

1 ACCEPTED SOLUTION

avatar

Hi @Zack Riesland,

Hive shell activity is like other bash programs. You can check the status of the previous command with $? -- unsuccessful queries/jobs will return a non-zero exit code.

[root@dn0 /]# hive -e "show databas;"

Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root":hdfs:hdfs:drwxr-xr-x

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)




....

[root@dn0 /]# echo $?

1




View solution in original post

5 REPLIES 5

avatar
Master Guru

@Zack Riesland many ways to do this, but .02 easiest with oozie. Oozie has a hive action where you can specify the hsql you want to run decision points after each step, ie failure do this, success do that. It is simple workflow. OBTW, the new workflow manager is available in 2.5 where you can orchestrate this workflow via UI

Another option is NiFi. You can execute series of hive scripts and make decision points after each script based on if it failed or succeeded.

avatar
Super Collaborator

Thanks @Sunile Manjee

I have actually re-written a lot of this with nifi, and it certainly provides a lot of flexibility.

But for now, I also have to maintain this older, script-based version of this process.

Any ideas about how to accomplish this with scripts?

avatar
Master Guru

@Zack Riesland to use scripts I would go with oozie.

avatar

Hi @Zack Riesland,

Hive shell activity is like other bash programs. You can check the status of the previous command with $? -- unsuccessful queries/jobs will return a non-zero exit code.

[root@dn0 /]# hive -e "show databas;"

Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root":hdfs:hdfs:drwxr-xr-x

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)




....

[root@dn0 /]# echo $?

1




avatar
Super Collaborator

Perfect. Thanks!