Support Questions
Find answers, ask questions, and share your expertise

How to determine whether a hive script fails?

Super Collaborator

I have a shell script that calls a series of hive .sql scripts.

If a hive operation fails, I want the shell script to stop and exit.

How can I accomplish this?

Does "hive -f" have return codes that I can check in bash?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to determine whether a hive script fails?

Hi @Zack Riesland,

Hive shell activity is like other bash programs. You can check the status of the previous command with $? -- unsuccessful queries/jobs will return a non-zero exit code.

[root@dn0 /]# hive -e "show databas;"

Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root":hdfs:hdfs:drwxr-xr-x

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)




....

[root@dn0 /]# echo $?

1




View solution in original post

5 REPLIES 5

Re: How to determine whether a hive script fails?

Super Guru

@Zack Riesland many ways to do this, but .02 easiest with oozie. Oozie has a hive action where you can specify the hsql you want to run decision points after each step, ie failure do this, success do that. It is simple workflow. OBTW, the new workflow manager is available in 2.5 where you can orchestrate this workflow via UI

Another option is NiFi. You can execute series of hive scripts and make decision points after each script based on if it failed or succeeded.

Re: How to determine whether a hive script fails?

Super Collaborator

Thanks @Sunile Manjee

I have actually re-written a lot of this with nifi, and it certainly provides a lot of flexibility.

But for now, I also have to maintain this older, script-based version of this process.

Any ideas about how to accomplish this with scripts?

Re: How to determine whether a hive script fails?

Super Guru

@Zack Riesland to use scripts I would go with oozie.

Re: How to determine whether a hive script fails?

Hi @Zack Riesland,

Hive shell activity is like other bash programs. You can check the status of the previous command with $? -- unsuccessful queries/jobs will return a non-zero exit code.

[root@dn0 /]# hive -e "show databas;"

Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root":hdfs:hdfs:drwxr-xr-x

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)




....

[root@dn0 /]# echo $?

1




View solution in original post

Re: How to determine whether a hive script fails?

Super Collaborator

Perfect. Thanks!