@Daniel Müller When merging the Hive ORC files, instead of matching the files wrt Block size, the files are merge as per the ORC stripe size. The property which controls this is hive.merge.orcfile.stripe.level. When the property is set to true, the merge happens at stripe level and when set to false, the files are merge at file level. Parameters which affect the file level merge are: hive.merge.tezfiles=true hive.merge.mapfiles=true hive.merge.size.per.task=256000000 hive.merge.smallfiles.avgsize=16000000 For more details refer link. Also, there are some known limitations related to concatenation. Do observe the behaviour and file count when the concatenate is run in say 5 iterations. ... View more

@wei mou Seems like the privilege issue at the MySQL end. Please try running the below command: grant all privileges on *.* to 'root'@'hadoopS4' using password 'xxx'; flush privileges; ... View more

@Anjali Shevadkar It seems like you are hitting "Dag submit failed due to Invalid TaskLaunchCmdOpts defined for Vertex Map 1 : Invalid/conflicting GC options found". Please check and share the values of the following parameters "tez.am.launch.cmd-opts" and "hive.tez.java.opts" they should not be conflicting. Specially the GC options. Remove +UseParallelGC from either of the properties to address the issue. This is because "-XX:+UseG1GC and -XX:+UseParallelGC" Should never be used together." ... View more

@Sambasivam Subramanian Please refer below links for adding nodes to existing cluster: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_administration/content/ref-4303e343-9aee-4e70-b38a-2837ae976e73.1.html https://www.youtube.com/watch?v=jiCI4yIkUlc ... View more

@Anjali Shevadkar Were you able to verify with the permissions? ... View more

@Anjali Shevadkar I checked internally and below policy works: beeline> !connect jdbc:hive2://:2181,:2181,:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Connecting to jdbc:hive2://:2181,:2181,:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 Enter username for jdbc:hive2://:2181,:2181,:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: hive Enter password for jdbc:hive2://:2181,:2181,:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: **** Connected to: Apache Hive (version 1.2.1000.2.6.5.0-292) Driver: Hive JDBC (version 1.2.1000.2.6.5.0-292) 0: jdbc:hive2://xxx:> show databases; +----------------+--+ | database_name| +---------------+--+ | default| | testdb | +----------------+--+2 rows selected (0.362 seconds) ... View more

@Rajendra Manjunath You can use Metron 0.4.2 with HDP 2.6.5 including Kafka 0.10.1 and Storm 1.1.0. ... View more

@Anjali Shevadkar Can you share the screenshot of the Ranger policy and also, error snippet from Hiveserver2 log? ... View more

@Anjali Shevadkar Do you have Ranger policies defined for Hive? If yes, check if the login user has use/select on the databases. Also, you could check the 'show databases' from Hive Cli as it by-passes the Ranger. ... View more