About chrsvarma

chrsvarma · ‎03-14-2016

Execution engine from MR to TEZ

chrsvarma · ‎03-14-2016

Just upgraded HDP from 2.2 to 2.3.4. & default execution engine to MR to TEZ. table we are querying has 2.4Billion rows select * from lowes_clickstream.clickstream_mobile limit 100; Error: java.io.IOException: java.lang.IndexOutOfBoundsException: toIndex = 993 (state=,code=0) 0: jdbc:hive2://hadoop-sa:8443/default>

chrsvarma · ‎02-05-2016

@Neeraj Sabharwal yea...we did follow with support and they said it is a know issue...i posted the comments below..alll we need to do is change the DB engine for all the tables which are MyISAM to InnoDB. thanks for your response though..

chrsvarma · ‎02-05-2016

This is known issue for MYSQL DBs and we need to convert all the tables to use InnoDB engine. You can use the below SQL to generate the commands to alter the tables to InnoDB engine. SELECT concat('ALTER TABLE ',TABLE_NAME,' ENGINE=InnoDB;') FROM Information_schema.TABLES WHERE TABLE_SCHEMA = 'ambaridb' AND ENGINE = 'MyISAM' AND TABLE_TYPE='BASE TABLE'; Once we get the output , remove the ' | ' and run the alter commands. Once we run alter commands, commit and exit mysql. and than run ambari-server upgrade.

chrsvarma · ‎02-04-2016

@Artem Ervits no it is not

chrsvarma · ‎02-03-2016

@Artem Ervits Error executing query: ALTER TABLE cluster_version ADD CONSTRAINT FK_cluster_version_cluster_id FOREIGN KEY (cluster_id) REFERENCES clusters (cluster_id) clusters table has one row and cluster_version table is empty. that was the reason it is not able to add FK

chrsvarma · ‎02-03-2016

@Artem Ervits HW doc doesnt say anything about that... If your current Ambari version is 1.6.1 or below, you must upgrade the Ambari Server version to 1.7 before upgrading to version 2.2.

chrsvarma · ‎02-03-2016

ambari upgrade upgrading ambari-server from 1.7 to 2.2 with mysql 5.6 while running ambari-server upgrade it failed with org.apache.ambari.server.AmbariException: Cannot add foreign key constraint any work around, i couldn't even create the FK manually either.. is it a known issue?

chrsvarma · ‎01-27-2016

@Neeraj Sabharwal this is a beautiful future...but we are hitting a bug with Hive settings in Ambari...Support is working on a case...in the mean is there any way where i can make my group as default group in Ambari?

chrsvarma · ‎01-27-2016

@Neeraj Sabharwal i understand what you are saying...but our requirement is below... Mast2 , HS2 port 10001 Authentication None - users connect through Knox 8443 Mast1, HS2 port 10000, i can stop the instance and deploy HS2 it through Ambari...if i do that can each hiveserver2 in ambari can have individual settings? because on Mast1 i would like to enable HS2 with LDAP authentication, if we can do that that will make my work easy...i can simply enable Ranger authorization in Hive settings in ambari which will control both.

chrsvarma · ‎01-27-2016

@Neeraj Sabharwal how do i add another hive-plugin from ambari? if i install new hiveserver2 through ambari , can i make different settings for each hiveserver2? live one can get no authentication and other with AD authentication?

chrsvarma · ‎01-27-2016

i got 2.3.4 installed and ranger-hive-plugin is installed on one of the master node where hiveserver is running on port 10001 i would like to enable ranger-hive-pluin on another master node where hiveserver is running on port 10000. is there any way to enable plugin on master node1 in Ambari? i did try to edit install.properties of hive in mast1 and enabled it and bounced hive which did not help. hiveserver2 running on mast1 is not registered with Ambari-- just FYI

chrsvarma · ‎01-25-2016

I'm new to Spark and appreciate if i can get some answers. How can i enable spark authentication against LDAP in a non kerberized environment? Looks like spark is connecting to Metastore directly...how can we force it to connect to hiveserver2? Is there any way to suppress all the info it prints when we start/exit spark-sql? every session i start it start using 4040 +1 and http & sparkdriver with randon ports, is there any way we can force spark to use subs et of ports instead of random ports?

chrsvarma · ‎01-21-2016

thank you @Deepesh

chrsvarma · ‎01-21-2016

thank you @Jonas Straub

chrsvarma · ‎01-21-2016

@Neeraj Sabharwal @Predrag Monodic @Jonas Straub @Artem Ervits thanks for looking at the post. I know this is strange issue. I did clear cache and did all the possible things..nothing helped...i opened a ticket with support and we learned it as a BUG. support forwarded this to Dev team to assist. there is a weird workaround for this issue. to see the settings...i have to click on yarn first go to yarn config and click on hive which shows hive settings...yes it does work only if i follow above method.

chrsvarma · ‎01-21-2016

is there any command which can give us the list of components that are registered with znode in real time? i want to see what are the hiveservers that are registered with zookeeper...etc

chrsvarma · ‎01-20-2016

@Kuldeep Kulkarni it is simple, we had the same problem and i simply edited hive.distro file /usr/hdp/2.2.0.0-2041/hive/bin/hive.distro go to this line and comment it and add below string --- if [ "$SERVICE" = "" ] ; then if [ "$SERVICE" = "" ] && [ "$USER" = "xxxxxxxx" ] ; then if [ "$SERVICE" = "" ] ; then xxxxxxx - you can use your shared id /service id let me how did it go..

chrsvarma · ‎01-20-2016

upgraded from HDP 2.2 to HDP 2.3.4 and i couldn't see anything under hive where memory and other important parameters display in ambariscreen-shot-2016-01-20-at-53419-pm.png

chrsvarma · ‎01-14-2016

it happens when Ambari is upgraded to 2.2 this is a known issue -https://issues.apache.org/jira/browse/AMBARI-13325 Disable bucket cache by erasing the values from these propeerties from ambari - and restart the Hbase service. - hbase.bucketcache.ioengine - hbase.bucketcache.percentage.in.combinedcache - hbase.bucketcache.size

chrsvarma · ‎01-14-2016

@Enis value is set to 4G isn't big enough? we hardly use hbase

chrsvarma · ‎01-14-2016

not sure what parameter i have to change Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2484) ... 5 more Caused by: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)

chrsvarma · ‎12-30-2015

we got ambari 2.1.2 and we are able to send snmptraps to externals monitoring servers. however our monitoring folks says they don't see complete message to format it on their side. is it related to https://issues.apache.org/jira/browse/AMBARI-13205... bug?

chrsvarma · ‎12-30-2015

@bsaini i think i figured out the issue...it is a bug when we upgrade ambari database from Mysql 5.1 to later...as of now only option is to upgrade ambari to 2.2 or rollback the changes. thanks for taking time in answering my question...appreciate it!!! https://na9.salesforce.com/articles/en_US/Issue/Ambari-log-reports-You-have-an-error-in-your-SQL-syntax-check-the-manual-that-corresponds-to-your-MySQL-server-version-for-the-right?popup=true

chrsvarma · ‎12-30-2015

I had a tough time with Ambari 1.7 pointing to Mysql 5.6 on remote host. AS is using all the available ports on the source to make a connection to remote host to port 3306. At one time i saw 28000 ports being used by ambari to connect. this is killing us because it even used 50070 at one time, and i had tough time to figure it out. Finally i stopped ambari-server and all is well. Did anyone face this issue? if so any idea on fix? apart from downgrading Mysql or upgrade ambari to 2.1.2?

chrsvarma · ‎12-28-2015

thanks Scott..but this is sad that it doesn't support AD...

chrsvarma · ‎12-28-2015

thanks for swift response..

chrsvarma · ‎12-28-2015

i'm trying to sqoop (Linux server) data from Sql server...i was able to do it with local account on sql server, but i want to try with Windows authentication

chrsvarma · ‎12-21-2015

i got Mysql 5.1.73 on RH and would like to upgrade to 5.6

chrsvarma · ‎12-16-2015

users connect to Hive through Knox uses AD credentials...integrated HDFS with AD groups... now HDFS is not able to recognize local user groups.

Online	Offline
Last Visited	‎01-02-2018 09:29 AM

Date Registered	‎12-09-2015 03:37 PM
Last Visited	‎01-02-2018 09:29 AM
Total Messages Posted	115
Total Kudos Received	43

Re: insert overwrite directory - beeline

Re: Master failed to complete initialization after...

Re: centrify express user principal

Re: hive metastore authentication

Re: Kerberos principal should have 3 parts: hive

Re: java.lang.IndexOutOfBoundsException

java.lang.IndexOutOfBoundsException

Re: org.apache.ambari.server.AmbariException: Cann...

Re: org.apache.ambari.server.AmbariException: Cann...

Re: org.apache.ambari.server.AmbariException: Cann...

Re: org.apache.ambari.server.AmbariException: Cann...

Re: org.apache.ambari.server.AmbariException: Cann...

org.apache.ambari.server.AmbariException: Cannot a...

Re: additional ranger hive plugin

Re: additional ranger hive plugin

Re: additional ranger hive plugin

additional ranger hive plugin

spark authentication ldap

Re: zookeeper register components

Re: zookeeper register components

Re: hive settings missing in ambari

zookeeper register components

Re: How to disable hive shell for all users (Hive ...

hive settings missing in ambari

Re: Hbase region servers crash with Out of Memory

Re: Hbase region servers crash with Out of Memory

Hbase region servers crash with Out of Memory

Ambari 2.1.2 snmptrap

Re: Ambari with mysql 5.6

Ambari with mysql 5.6

Re: sqoop sqlserver with windows authentication

Re: sqoop sqlserver with windows authentication

sqoop sqlserver with windows authentication

mysql upgrade

HDFS user group mapping with AD