question Re: Hive - TBLPROPERTIES in Support Questions

Hive - TBLPROPERTIES

reachkarthikt — Wed, 10 Feb 2016 17:06:42 GMT

Hello, DW guy learning hadoop.

I wanted to understand what are the full list of TBLPROPERTIES that are predefined in Hive and available for use? Couldnt find it in the Hive Manual. Thanks for the help

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 17:09:18 GMT

@KARTHIK THYAGARAJAN

See this https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

The TBLPROPERTIES clause allows you to tag the table definition with your own metadata key/value pairs. Some predefined table properties also exist, such as last_modified_user and last_modified_time which are automatically added and managed by Hive. Other predefined table properties include:

TBLPROPERTIES ("comment"="table_comment")
TBLPROPERTIES ("hbase.table.name"="table_name") – see HBase Integration.
TBLPROPERTIES ("immutable"="true") or ("immutable"="false") in release 0.13.0+ (HIVE-6406) – see Inserting Data into Hive Tables from Queries.
TBLPROPERTIES ("orc.compress"="ZLIB") or ("orc.compress"="SNAPPY") or ("orc.compress"="NONE") and other ORC properties – see ORC Files.
TBLPROPERTIES ("transactional"="true") or ("transactional"="false") in release 0.14.0+, the default is "false" – see Hive Transactions.
TBLPROPERTIES ("NO_AUTO_COMPACTION"="true") or ("NO_AUTO_COMPACTION"="false"), the default is "false" – see Hive Transactions.
TBLPROPERTIES ("auto.purge"="true") or ("auto.purge"="false") in release 1.2.0+ (HIVE-9118) – see Drop Table and Drop Partitions.

Re: Hive - TBLPROPERTIES

reachkarthikt — Wed, 10 Feb 2016 17:41:14 GMT

Thanks @Neeraj Sabharwal

But yes, i did see this already.

I am looking for a place where i can find the full list.

For example I learnt from a tutorial that tblproperties ("skip.header.line.count"="1") is available from 0.13 onwards

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 17:48:37 GMT

@KARTHIK THYAGARAJAN Hmm 🙂 Need to find a git to see if someone has shared it

hive.file.max.footer 
 Default Value: 100 
 Max number of lines of footer user can set for a table file. 
skip.header.line.count 
 Default Value: 0 
 Number of header lines for the table file. 
skip.footer.line.count 
 Default Value: 0 
 Number of footer lines for the table file.

Re: Hive - TBLPROPERTIES

aervits — Wed, 10 Feb 2016 18:22:23 GMT

@KARTHIK THYAGARAJAN

You can read the source code https://github.com/apache/hive/tree/master/ql/src/java/org/apache/hadoop/hive/ql

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 18:29:02 GMT

@KARTHIK THYAGARAJAN Here you

https://github.com/apache/hive/search?utf8=%E2%9C%93&q=TBLPROPERTIES

Now, you can be creative and keep running your own search 😉

Re: Hive - TBLPROPERTIES

reachkarthikt — Wed, 10 Feb 2016 18:36:36 GMT

@Neeraj Sabharwal - thank you! did you try to share a link? in 'go' - it doesnt work

Re: Hive - TBLPROPERTIES

reachkarthikt — Wed, 10 Feb 2016 18:38:05 GMT

@Artem Ervits - this is the way then, Thank you! - I will explore further now.

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 18:39:09 GMT

@KARTHIK THYAGARAJAN Yes 🙂 see this https://github.com/apache/hive/search?utf8=%E2%9C%93&q=TBLPROPERTIES

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 18:40:36 GMT

@KARTHIK THYAGARAJAN I do believe that you can consolidate or run more searches.

FYI: You can do the same in future for any of the components

Re: Hive - TBLPROPERTIES

reachkarthikt — Wed, 10 Feb 2016 18:50:45 GMT

this option surely helps! Thanks a lot @Neeraj Sabharwal

Re: Hive - TBLPROPERTIES

nsabharwal — Wed, 10 Feb 2016 18:55:46 GMT

Please accept the best answer to close the loop @KARTHIK THYAGARAJAN

Re: Hive - TBLPROPERTIES

wzheng — Sat, 16 Apr 2016 00:58:35 GMT

As far as I understand, tblproperties is just a list of key/value pairs. You can specify whatever you want on it (even if it's not predefined, e.g. 'weather'='warm'). But of course there must be code logic that interprets your tblproperties and do something, otherwise it's useless.