About SK1

SK1 · ‎02-17-2016

@Neeraj Sabharwal: Thanks a lot for your testing. I see you have tested it for a unix user(neeraj) who is part of a unix group(hdpadmin).Which is working fine for me. But my requirement is we have some users where they don't connect to server,they directly use some tools (like Aqua Data Studio or SQL client or Teradata client) and we validate them by login to our cluster by their LDAP(active directory) with jdbc string or though beeline. And when they submit their job then they have to set property mapred.job.queue.name and run their jobs. So my point is can we configure CS view for ldap or AD groups as well ? I tried it for groups but getting below error. But when I tried for user specific then it is working(as expected) g:adhdpadm:batch ERROR : Failed to execute tez graph. org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1455533826426_0025 to YARN : Failed to submit application application_1455533826426_0025 submitted by user saurkuma reason: No groups found for user saurkuma u:saurkuma:batch: working

SK1 · ‎02-15-2016

Hi @Neeraj Sabharwal: when configured ad group mapping then I don't defined any user mapping.And yes user saurkuma is a part of adhdpadm group. My first point is does it supports ldap groups(Active directory) or not ? I checked with local unix groups and found them working.

SK1 · ‎02-15-2016

@Neeraj Sabharwal: Yes I am a part of the AD group(adhdpadm) and when I configure u:saurkuma:default then it is working fine but when I do g:adhdpadm:default then it is failing with above error.

SK1 · ‎02-15-2016

Hello @Neeraj Sabharwal: Thanks for the above explanation. I have configured CS view which is working fine for local unix group and users. But when I configured for Ldap or AD group it does not work and fail with below error. org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1455533826426_0018 to YARN : Failed to submit application application_1455533826426_0018 submitted by user saurkuma reason: No groups found for user saurkuma

SK1 · ‎02-13-2016

@Zaher Mahdhi: Also you can refer below article. http://www.hadoopadmin.co.in/planning-of-hadoop-cluster/

SK1 · ‎02-13-2016

@Tech Guy, Can you restart hdfs service once again and wait for sometime until namenode comeout from safe mode. Some time it stay back in safe mode because of DN is down or hdfs space is full or many under replicated/corrupted files. so also run hadoop dfsadmin -report command to get above info. Also as Neeraj suggested you can leave safe mode forcefully by running below command. hdfs dfsadmin -safemode leave.

SK1 · ‎02-13-2016

If a pool's minimum share is not met for some period of time, the scheduler optionally supports preemption of jobs in other pools. The pool will be allowed to kill tasks from other pools to make room to run. Preemption can be used to guarantee that "production" jobs are not starved while also allowing the Hadoop cluster to also be used for experimental and research jobs. In addition, a pool can also be allowed to preempt tasks if it is below half of its fair share for a configurable timeout (generally set larger than the minimum share preemption timeout). When choosing tasks to kill, the fair scheduler picks the most-recently-launched tasks from over-allocated jobs, to minimize wasted computation. Preemption does not cause the preempted jobs to fail, because Hadoop jobs tolerate losing tasks; it only makes them take longer to finish. for more details on configuration and understanding on preemption please go through below link http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_yarn_resource_mgt/content/preemption.html

SK1 · ‎02-12-2016

Can you check mysql connectivity and past logs here. Can you also paste output of the following command. SHOW GRANTS FOR 'hive'; And as Neeraj suggested please grant all privileges to hive user on host like below commend. GRANT ALL PRIVILEGES ON *.* TO 'hive'@'<hive_host>' IDENTIFIED BY PASSWORD '<password>'

SK1 · ‎02-12-2016

HI, I have a requirement where we have to assign our CS queues based on ad groups. For example our ad users are using cluster and running jobs under defined queue but I want that is there any way to configure their AD group with queue so that each member of that queue will go only to a specific queue.

SK1 · ‎02-12-2016

Is there a good way to convert curl api json output to csv ? Actually when I do curl -X GET "http://lxhdpmasttst001.lowes.com:8088/ws/v1/cluster/scheduler" it gives output to json and I want to load it to hive or want to convert into csv so that everyone can read it easily.

SK1 · ‎02-12-2016

@rajdip chaudhuri Can you please paste /var/log/hadoop/hdfs/<latest_NN>.log and /var/log/ambari-server/ambari-server.log files content. Also please check whether hdfs and ambari service is running.

SK1 · ‎02-11-2016

When you upgrade your stack from 2.2 to 2.3 then you will have issue using hue or hive due to a bug. In hive config file you may have wrong value in templeton.libjars property = /usr/hdp/${hdp.version}/zookeeper,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar/zookeeper.jar So change it to correct one. Actually this is a bug which occurs with upgrade from HDP 2.2 to 2.3. correct value is below /usr/hdp/${hdp.version}/zookeeper/zookeeper.jar,/usr/hdp/${hdp.version}/hive/lib/hive-common.jar

SK1 · ‎02-11-2016

@Artem Ervits Sorry if I could not explain my question. Actually I don't have any issue by creating widgets, but when I export into csv for 1 hr then it would be on 1 min interval but as I go for one day or week or year then this interval increase to 1 hr. So my requirement is to have 30 sec interval only for 1 month or one year also. I tried to follow below URL but could not get clear idea . https://cwiki.apache.org/confluence/display/AMBARI/Configuration

SK1 · ‎02-10-2016

@Rahul Pathak I have ambari Version2.2.0.0 and hdp stack 2.3.4.0. I tried for other metrics for Resourcemanager but getting some weird error. Problem! Unable to compile query predicate: Invalid Query Token: token='(', previous token type=VALUE_OPERAND.

SK1 · ‎02-10-2016

HI @Rahul Pathak I am selecting from drop down only as below . but it is giving me same error.

SK1 · ‎02-10-2016

@Rahul Pathak @Neeraj Sabharwal : I am trying to configure grafan with ambari and trying to create graph but it is throwing me an error. So can you please point me to solution. Problem! The requested resource doesn't exist: ServiceComponentHost not found, clusterName=HDPTST, serviceName=AMBARI_METRICS, serviceComponentName=METRICS_COLLECTOR, hostName=test.com

SK1 · ‎02-10-2016

Hello Everyone, In ambari widgets we get data for every 1 minute if we extract for a 1 hr but if we go for one day or more then this time interval keep on increasing. But my requirement is to get a widget for Capacity queue utilization every 30 sec. So can you please help me to get it done. Thanks in advance.

SK1 · ‎02-10-2016

Hello Hadi, It seems your ambari-agent is not running. So please restart ambari agent on all the nodes. $ ambar-agent start or $ ambari-agent restart

SK1 · ‎02-10-2016

Hi Vinod, You can use following steps to apply patch on your server. 1. Download patch to local or server. 2. You can use below command to apply it under ambari home dir. $ patch -p0 -i /path/to/AMBARI-14466.patch

SK1 · ‎02-10-2016

Hello Sasikumar, Please use username :azure and password : azure.

SK1 · ‎02-09-2016

@Rupinder Singh Some time by pressing ctrl+c does not kill jobs, so it always better to know job status via command and if it is running then kill by command. yarn application -status jobid yarn application -kill jobid (if running). Also do you have hue installed in your env ? if yes can you try with hue as well. Also put your log of yarn jobs.

SK1 · ‎02-08-2016

@Artem Ervits @Neeraj Sabharwal @Rahul Pathak : Thanks for all your help and now I am able to create widgets for Queue utilization, but still have one question. When I export data for one day then it gives me for every minutes metrics but when I do for one month then it goes to 1 hr interval and even for one year it goes to 1 day interval. So my question is can we export data every 1 minute interval for one year or one month.

SK1 · ‎02-08-2016

Hi @kang hua Can you restart hive metastore and HS2 and then do the following. Is your View server is separate and Hiveserver is separate ? If yes can you please check whether you are able to make connection to HS2 from View server. Also run service check through ambari by clicking on hive service and then action. Also goto to some server CLI where you can run some hive query manually just to cross check. If still error persist then paste logs here .

SK1 · ‎02-08-2016

So what could be the reason of failure @peeyush. any idea.

SK1 · ‎02-08-2016

Thanks @peeyush I cross checked and found the below : What I see is there was no value passed for this variable and for other variable as well. But my point is how it is getting success for next times. Does falcon create separate coordinator for each job and every-time ?

SK1 · ‎02-08-2016

@peeyush Just forgot to mentioned that after some failure if I resubmit process and feed entities it is running successfully. But getting failed many times with no arguments value in configuration.

SK1 · ‎02-08-2016

@peeyush I ma using below falcon version(0.6.0.2.2.0.0-2041). And there are some confidential info in entity so unfortunately I can not share that one. But I can tell you that if I delete feed and process entity and then resubmit again that time it is working.And note that I am not a tall using this parameter in my entity as well as job.properties in oozie. So I want to know from where falcon takes this value

SK1 · ‎02-08-2016

We are running hive jobs with oozie and falcon, it was running fine but all of sudden it started failing with below error. EL_ERROR: variable [feedInstancePaths] cannot be resolved, Also we are not declaring or assigning this variable anywhere in our code. sitecatalyst-kpis-generation-daily-process] JOB[0011048-151208005457707-oozie-oozi-W] ACTION[0011048-151208005457707-oozie-oozi-W@succeeded-post-processing] ELException in ActionStartXCommand javax.servlet.jsp.el.ELException: variable [feedInstancePaths] cannot be resolved at org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106)

SK1 · ‎02-08-2016

Divakar: Thanks for your consideration, but as I have mentioned above problem has been resolved by running build/env/bin/hue syncdb --noinput

SK1 · ‎02-08-2016

Yes,you are right @artem Evits we should use views but for that we should have a dedicated server which would manage views,otherwise it may create overload on our ambari servers and we do some changes and restart ambari then users also may get impacted.

Online	Offline
Last Visited	‎07-08-2020 01:11 PM

Date Registered	‎05-29-2017 06:13 AM
Last Visited	‎07-08-2020 01:11 PM
Total Messages Posted	408
Total Kudos Received	121

Re: select * from table is not returning any rows ...

Re: Falcon feed throwing an error during feed sche...

Re: we are getting below error/exception while exe...

Re: bin/solr status throwing an error "clusterstat...

Re: How to solve buffer memory issue in Solr

Re: How to assigned capacity scheduler queue based...

Re: How to assigned capacity scheduler queue based...

Re: How to assigned capacity scheduler queue based...

Re: How to assigned capacity scheduler queue based...

Re: Best cluster configuration

Re: Connection refused while accessing HDFS Files ...

Re: Preemption on Fair queue

Re: Hive metastore issue in HDp2.3.4.0

How to assigned capacity scheduler queue based on ...

Is there a good way to convert curl api json outpu...

Re: Issue while using Hive View in Ambari console

templeton.libjars property changed its value after...

Re: how to get a widget for Capacity queue utiliza...

Re: YARN Capacity Scheduler Queue utilisation grap...

Re: YARN Capacity Scheduler Queue utilisation grap...

Re: YARN Capacity Scheduler Queue utilisation grap...

how to get a widget for Capacity queue utilization...

Re: How can I start my Ambari heartbeat?

Re: setpgid(2296, 0) failed - [Errno 13] Permissio...

Re: unable to login to ambari with the default adm...

Re: org.apache.pig.backend.hadoop.executionengine....

Re: How to add Scheduler widgets to RM in Ambari U...

Re: ambari hive view timeout ,why ?

Re: Falcon jobs are failing with EL_ERROR: variabl...

Re: Falcon jobs are failing with EL_ERROR: variabl...

Re: Falcon jobs are failing with EL_ERROR: variabl...

Re: Falcon jobs are failing with EL_ERROR: variabl...

Falcon jobs are failing with EL_ERROR: variable [f...

Re: Hue configuration Error | Beewax | File Brows...

Re: Hue configuration Error | Beewax | File Brows...