Created 10-05-2018 03:27 PM
As part of a script I'm writing, I'm wanting to get the HDFS location of a list of Hive schemas that's passed in via a text file. The best way I can think to do this is to run a beeline command in a loop that performs a `describe schema` command and extracts the HDFS location of each schema from this output. However, this will require me to authenticate every time I run this command which is inefficient. Is there a better way to programtically get the HDFS locations of a list of Hive schemas?
Created 10-05-2018 04:04 PM
You can put all your sql commands in a file and run the file using beeline.
Ex queries.sql has below statements
describe formatted table1; describe formatted table2;
You can run the queries.sql like below
beeline -u "{url}" -f queries.sql
.
Please "Accept" the answer if this helps.
Created 10-05-2018 04:04 PM
You can put all your sql commands in a file and run the file using beeline.
Ex queries.sql has below statements
describe formatted table1; describe formatted table2;
You can run the queries.sql like below
beeline -u "{url}" -f queries.sql
.
Please "Accept" the answer if this helps.
Created 10-08-2018 03:12 PM
Thanks @Aditya Sirna, I think this will get me what I need. What I'm ultimately trying to get is the HDFS location so I can use it in the script I'm writing. Is performing a describe table and then grepping the output the best way to do this?
Created 10-08-2018 03:32 PM
You can grep for the location. I am not able to think of other solution for now.
Created 10-08-2018 03:39 PM
Hey Josh:
you could do:
SHOW CREATE TABLE mytable;
and then look for the keyword LOCATION in the output.
When i run that in my sql client, the hdfs path is the next line. You can also look for a line that starts with
'hdfs://
If you want to use this with the information @Aditya Sirna provided, you could have a file with multiple statements like:
SHOW CREATE TABLE mytable;
SHOW CREATE TABLE mytabl1;
SHOW CREATE TABLE mytabl2;
and then filter for lines that start with hdfs. I haven't found a way to get JUST the Location of a table.
Hope that helps.
Thanks!
Regards