I am thinking of solution for the jira.. This needs to be implemented in code. There is no config to do this for now. ... View more

Concatenation depends on which files are chosen first. The ordering of the files not deterministic with CombineHiveInputFormat, since grouping happens at hadoop layer Concatenation will split or combine files based on orc file size > or < maxSplitSize. for eg., say if you have 5 files.. 64MB, 64MB, 64MB, 64MB, 512MB & mapreduce.input.fileinputformat.split.minsize=256mb this can result in 2 files 256MB, 512MB.. or it may result in 3 files 256MB, 256MB, 256MB. I raised a jira for the same Easy solution for this would be to add a path filter to skip files > maxSplitSize. ... View more

I am glad the problem is resolved. Both 1) http://nexus-private.hortonworks.com/nexus/content/groups/public/ 2) http://repo.hortonworks.com/content/groups/public/ Will resolve this issue as mentioned in my previous link to refer hortonworks public repo. Please accept the answer ... View more

@Gaurang Shah You are using hortonworks.extrepo (http://repo.hortonworks.com/content/repositories/releases, releases=true, snapshots=true), This repo dont have dependent jars, can you please use http://nexus-private.hortonworks.com/nexus/content/groups/public/ This repo has all the required dependencies eg., 1) org.mortbay.jetty:jetty-util:jar:6.1.ca26.hwx http://nexus-private.hortonworks.com/nexus/content/groups/public/org/mortbay/jetty/jetty-util/6.1.26.hwx/ ... View more

Use hortonworks public repo & try with below maven dependency to compile custom UDF with specific HDP version. <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-exec</artifactId> <version>1.2.1000.2.6.4.0-91</version> </dependency> Hortonworks maven public repo can be referred from this link ... View more

Any fat jar added for UDF ? Can you enable verbose to see which jar is getting loaded ? ... View more

One possible reason i could think is you have hive 1.x serde jar in LLAP classpath. ie., In Hive 1.x, return type of ColumnProjectionUtils.getReadColumnNames is List<String> In Hive 2.x, return type of ColumnProjectionUtils.getReadColumnNames is String[] There is a method return type mismatch which is throwing NoSuchMethodError. ... View more

AFAIK, Phoenix Handlers are compiled with Hive 1.x dependency, @Sergey Soldatov, are we supporting Phoenix Storage handlers in HS2Interactive / Hive 2.x ? ... View more

varchar requires length of the column to be mentioned in hive. Can you try below query create external table default.val_deriv_orc (deriv varchar(10), deriv_descr varchar(15))stored as orcfile; if you are not sure about the max length of the column data, its better to use string datatype. varchar will automatically remove the column data more than specified length. 0: jdbc:hive2://localhost:10000> create external table default.val_deriv_orc (deriv varchar(4) ,deriv_descr varchar(5))stored as orcfile; 0: jdbc:hive2://localhost:10000> insert into default.val_deriv_orc values('abcdef','abcde'); 0: jdbc:hive2://localhost:10000> select * from default.val_deriv_orc; +----------------------+----------------------------+--+ | val_deriv_orc.deriv | val_deriv_orc.deriv_descr | +----------------------+----------------------------+--+ | abcd | abcde | +----------------------+----------------------------+--+ ... View more

As @gopal said, HS2 (Interactive) has better SQL Support, Hive CLI always use Hive1.x, whereas HS2 (Interactive) use Hive 2.x If you are using beeline, u can connect to both HS2 & HS2 (Interactive) based on url provided. No need to reload the data. If above SQL needs to be supported in HDP 2.6.0 Hive 1.x, then i think you might need HIVE-15211 & HIVE-16885. ... View more

@Carlton Patterson pls try the below create statement for acorn_category_frequency table creation Create table acorn_category_frequency ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/sampleout/' as select acorn_category, count(*) as acorn_categorycount from geography group by acorn_category<br> ... View more

In secure setup, to view active sessions, open queries, last max history queries tabs, keytab & principal passed for HS2 Interactive UI should be admin user. Admin Users are configured using "hive.users.in.admin.role", spengo prinicipal should be one of the admin users specified in this property. You can refer the link for HS2 Interactive UI configs ... View more

pls try to use below command for creating HiveSampleIn Table CREATE EXTERNAL TABLE HiveSampleIn ( anonid INT, eprofileclass INT, fueltypes STRING, acorn_category INT, acorn_group STRING, acorn_type INT, nuts4 STRING, lacode STRING, nuts1 STRING, gspgroup STRING, ldz STRING, gas_elec STRING, gas_tout STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/samplein/' tblproperties ("skip.header.line.count"="2"); ... View more

CREATE EXTERNAL TABLE HiveSampleIn ... LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/samplein/'; -- Getting terminated at here tblproperties ("skip.header.line.count"="2"); -- This might be throwing ParseException ... View more

You can use pre / post execution hook in hive. hive.exec.pre.hooks - Java class implementing ExecuteWithHookContext interface, this will be invoked before the execution of each statement hive.exec.post.hooks - Java class implementing ExecuteWithHookContext interface, this will be invoked after the execution of each statement ... View more