About dchiguruvad

dchiguruvad · ‎07-11-2016

@Parinita Kothari Yes you can do that too.. Create a Hive paritioned table and set hive.exec.dynamic.partition.mode=nonstrict in a case of dynamic -Import table from DB to the partitioned table here you need not specificy partition-keys and partition-values , Just -table <tblname> --hcatalog-table <tblname> is enough..

dchiguruvad · ‎07-07-2016

@Parinita Kothari To import into Multi-key partitioned Hive table , you can make use of --hcatalog-table -table <tblname> --hcatalog-table <tblname> --hcatalog-partition-keys k1,k2 --hcatalog-partition-values 'v1,v2' eg: -table DW_FACT_PUNTER_TEST_TEMP --hcatalog-table DW_FACT_PUNTER_TEST_TEMP --hcatalog-partition-keys TIME_YEAR,TIME_MONTH --hcatalog-partition-values '2014-04-11,2014-APR'

dchiguruvad · ‎07-07-2016

@Dinesh E This depends on the file system we specify in the path(if you dont specify any file system it will take local file system path) generic: hive.aux.jars.path=file:///hdp/apps/2.3.4.0-3485/hive/auxjars/* HDFS specific: hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/* hope this clears your doubt, if so accept the answer

dchiguruvad · ‎07-05-2016

@Dinesh E You can set this in hive config - so that every Hive/beeline session takes this path hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/* Yes you can use HDFS path here.(recommended to to Use HDFS path... if you go for local path it should also be present in the node where HS2 is running) https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration Alternate to it you can also set this: added this to the script hive-env.sh HIVE_AUX_JARS_PATH=<path of jar location>

dchiguruvad · ‎07-01-2016

@Simran Kaur In sqoop --hive import --warehouse directory is the temporary HDFS directory location which collects the imported data finally it moves the data (metadata of files) to hive.warehouse.dir (generally /apps/hive/warehouse- as we specify in our hive-site.xml)

dchiguruvad · ‎06-23-2016

@alain TSAFACK Ambari Hive Views provide this feature (Upload Table) where you can directly upload CSV file into an ORC Hive table.( It takes care internally the 2 step process to create ORC table)

dchiguruvad · ‎06-19-2016

@Vijay Parmar If this helps you in solving your problem set .. pls Vote or accept the comment.

dchiguruvad · ‎06-16-2016

@Roman Boyko By chance is the pgBouncer (lightweight connection pooler for PostgreSQL) installed along with yout Postgress Install. If so set this param ignore_startup_parameters = extra_float_digits in pgbouncer.ini (/etc/pgbouncer/pgbouncer.ini) http://lists.pgfoundry.org/pipermail/pgbouncer-general/2011-July/000830.html

dchiguruvad · ‎06-16-2016

@Steven Castelein Hiveserver2 in secure environment by default Authentication mode uses plain SASL. You can disable it either by setting in In hive-site.xml: hive.server2.authentication= 'NOSASL' Or To Use SASL: (https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-Integrity/ConfidentialityProtection) Integrity/Confidentiality Protection Integrity protection and confidentiality protection (beyond just the default of authentication) for communication between the Hive JDBC driver and HiveServer2 are enabled (Hive 0.12 onward, see HIVE-4911). You can use the SASL QOP property to configure this. This is only when Kerberos is used for the HS2 client (JDBC/ODBC application) authentication with HiveServer2. hive.server2.thrift.sasl.qop in hive-site.xml has to be set to one of the valid QOP values ('auth', 'auth-int' or 'auth-conf'). You can connect via below url jdbc:hive2://<m/c HS2>:10001/default;principal=<hive princiapl>?transportMode=http;httpPath=cliservice;auth=kerberos;sasl.qop=auth-int (if auth-int is set)

dchiguruvad · ‎06-15-2016

@suresh krish Try this alter table <table_name> > set serde 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH SERDEPROPERTIES ('field.delim' = '\001'); I guess hive internally converts '\001' --> \u0001

Online	Offline
Last Visited	‎06-26-2023 05:47 PM

Member Since	‎04-12-2016 10:40 AM
Last Visited	‎06-26-2023 05:47 PM
Posts	46
Kudos received	74

Cloudera Community

Re: Split string by SemiColon not working in Beeli...

Re: Sqoop Import - Now what

Re: sqoop insert into partitioned table

Re: Setting Hive Auxiliary jars path to HDFS locat...

Re: How does --hive import work

Re: sqoop insert into partitioned table

Re: sqoop insert into partitioned table

Re: Setting Hive Auxiliary jars path to HDFS locat...

Re: Setting Hive Auxiliary jars path to HDFS locat...

Re: How does --hive import work

Re: import csv data into hive table orc format

Re: How can I automate a process in Hive?

Re: sqoop fails with "Unsupported startup paramete...

Re: Cannot connect 'Streaming Data Ingest' to secu...

Re: How to use /u0001 Delimiter