Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

error_message : You have exceeded your daily reque

avatar
Expert Contributor

Hi, while running a Hive SQL group by query on few million rows I get a message in the results mixed with the output:

error_message : You have exceeded your daily reque

(xxxxxx row(s) affected)

results : []

this is mixed with some actual data output. Anyone has seen this error message?

1 ACCEPTED SOLUTION

avatar
Super Guru
If you are using CDH, you should already have parquet-tools command line available. If not, you can get it from here:
https://github.com/apache/parquet-mr/tree/master/parquet-tools

You just need to run:

parquet-tools cat /path/to/parquet-file.parq

Please note that the path is on local file system, not in HDFS.

And then you can grep the output to search for string "error_message".

View solution in original post

15 REPLIES 15

avatar
Super Guru
You can give that a try, however, I believe hive staging directory should not affect the result.

Have you used parquet-tools to analyze the data in the parquet file?

avatar
Expert Contributor

No I am not sure how to use these tools. 

avatar
Super Guru
If you are using CDH, you should already have parquet-tools command line available. If not, you can get it from here:
https://github.com/apache/parquet-mr/tree/master/parquet-tools

You just need to run:

parquet-tools cat /path/to/parquet-file.parq

Please note that the path is on local file system, not in HDFS.

And then you can grep the output to search for string "error_message".

avatar
Expert Contributor

Thanks will try that and see.

avatar
Expert Contributor

You are absolutely right! The fifth file 000004_0 in the parquet hive table directory had one string that matched the row value in the beeline SQL output. I am sure the other strings will also be in the data. So it finally confirms that everything is working fine without any errors in hive. Thanks for all your help as this really confused me but I did learn couple of new things so thanks again!!

 

$ parquet-tools cat 000004_0 | grep '(7256823 row(s) affected)'
triptype = (7256823 row(s) affected)

 

 

avatar
Super Guru
Glad that we identified the issue!