About jyadav

jyadav · ‎06-02-2017

@awhitter Thanks a lot. Ambari repo file permission was wrong.

jyadav · ‎06-01-2017

Hi Team, While trying to install standalone Ambari server on one of the existing edge node. Below is the error I got every time when I ran ambari-server setup command. SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? ERROR: Unexpected error 'getpwuid(): uid not found: 55025' ERROR: Exiting with exit code 1. REASON: Failed to create user. Exiting. Also, I don't see UID "55025" in the /etc/passwd and wondering from where it's picking this UID. Thanks

jyadav · ‎05-30-2017

@Constantin Stanca There was no error in hive/hdfs/yarn etc. during that time frame, even I tried to restart all the services but didn't helped. But when I restarted Ambari server the issue got resolved, don't know how this exception is related to ambari restart 🙂

jyadav · ‎05-30-2017

Hi, I'm getting below error while running select query in hive view. I checked proxy settings and evrything looks fine, this is a custom hive view on secured cluster and SSL hadoop. Can anyone suggest where to look?. java.lang.NullPointerException at org.apache.ambari.view.hive2.resources.jobs.JobService.getOne(JobService.java:139) at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)

jyadav · ‎01-17-2017

Hi, I'm trying to integrate JasperSoft with Hiveserver2(HDP-2.4) but getting error while testing hive connection. However I checked the class and it looks like that class is already available on below location. /opt/jasperreports-server-cp-6.3.0/apache-tomcat/lib/libfb303-0.9.0.jar /opt/jasperreports-server-cp-6.3.0/apache-tomcat/webapps/jasperserver/WEB-INF/lib/libfb303-0.9.0.jar 2017-01-17 15:23:42,360 ERROR SystemErrorController,http-nio-8080-exec-7:86 - Internal server error javax.servlet.ServletException: java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:418) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:708) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:88) at com.jaspersoft.jasperserver.api.security.csrf.JSCsrfGuardFilter.doFilter(JSCsrfGuardFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ..... at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368)

jyadav · ‎12-18-2016

Thanks @nyakkanti I also had similar issue and used one of the custom port available in Port forwarding.

jyadav · ‎09-23-2016

Hi @Smart Solutions Along with that property, lets try to send SIGTERM signal to the spark driver and rest it will ensure that application get stopped gracefully. You will see graceful shutdown logs in driver logs. Node where Driver is running. ps -ef | grep spark | grep <Spark Driver Name> kill -SIGTERM PID

jyadav · ‎09-21-2016

can you please share the minimal reproducible steps so that i can reproduce this locally? Also kindly mentioned the HDP version.

jyadav · ‎09-21-2016

Thanks @Ayub Pathan for the help, it worked with below syntax as per your suggestion. RULE:[1:$1](4[0-9]*)s/^4/d/

jyadav · ‎09-21-2016

Hi @bandhu gupta Can you please share the hiveserver2 logs?

jyadav · ‎09-20-2016

Hi @Josh Persinger Are you able to get result with same table structure but in case of hive managed table? Issue may be related to the delimiter or actual file format.

jyadav · ‎09-20-2016

Hi @Smart Solutions Is this behavior same even if you mention below param in application context? sparkConf.set(“spark.streaming.stopGracefullyOnShutdown","true")

jyadav · ‎09-20-2016

Hi @Ayub Pathan I'm still not able to convert HDFS auth rule i.e RULE:[1:$1](4[0-9]{7,8})s/^4/d/ into atlas auth rule. Ideally same should work.

jyadav · ‎09-16-2016

Thanks @Constantin Stanca But wondering why Ambari 2.2.2 has new feature (Grafana) even it was a maintainence release.

jyadav · ‎09-16-2016

Hi, I'm trying to implement auth rule in atlas by coping a rule from hdfs auth rules i.e RULE:[1:$1](4[0-9]{7,8})s/^4/d/ however this rule syntax doesn't seems to be working for atlas though same works for HDFS. Please help.

jyadav · ‎09-16-2016

Thanks @Mats Johansson so as per doc (<major>.<minor>.<maintenance> ) if I'm using Ambari 2.2.1 and want to upgrade to 2.2.2 then that will be a maintenance release upgrade with no new feature addition, right?.

jyadav · ‎09-15-2016

Hi, Can somebody explain about Ambari version naming standard? Looking similar to HDP https://community.hortonworks.com/questions/41422/question-on-hdp-versioning.html

jyadav · ‎09-13-2016

I found out the issue, actually we enabled the spnego for atlas therefore it was checking the spnego.service.keytab file. Somehow user "atlas" got removed from the application group therefore didn't had read permission on spnego.service.keytab file. Once we given read permission issue got resolved. Thanks

jyadav · ‎09-13-2016

After enabling kerberos we are seeing below errors in atlas application log. WARN - [3e0a832b-bc1f-43bf-ab70-fe616747cf1a:] ~ Authentication exception: GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid argument ( 400) - Cannot find key of appropriate type to decrypt AP REP - RC4 with HMAC) (AuthenticationFilter:586) Below is the klist output. klist -kte atlas.service.keytab Keytab name: FILE:atlas.service.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 4 01/01/70 01:00:00 atlas/xxx (DES cbc mode with CRC-32) 4 01/01/70 01:00:00 atlas/xxx (DES cbc mode with RSA-MD5) 4 01/01/70 01:00:00 atlas/xxx (ArcFour with HMAC/md5) 4 01/01/70 01:00:00 atlas/xxx (AES-256 CTS mode with 96-bit SHA-1 HMAC) 4 01/01/70 01:00:00 atlas/xxx (AES-128 CTS mode with 96-bit SHA-1 HMAC) Checked kerberos kvno number for that principal from KDC and keytab file and both are same. Please guide.

jyadav · ‎09-09-2016

can you please show me the output of below command? hadoop fs -ls hdfs://<servicename>/

jyadav · ‎09-09-2016

Hi @Farhad Heybati Instead of cluster name I believe you need to mentioned namenode nameserviceID which you mentioned while configuring NN HA.

jyadav · ‎09-06-2016

please accept the answer if issue got resolve or let us know if you are still facing same issue.

jyadav · ‎09-06-2016

sorry my bad. hadoop fs -chown root:root /user/root

jyadav · ‎09-06-2016

can you please try to run these two hadoop commands from hdfs user?. hadoop fs -mkdir /user/root hadoop fs -chown /user/root

jyadav · ‎09-06-2016

Direcotry name should be "/user/root" not "/usr/root"

jyadav · ‎09-06-2016

From hdfs user: hadoop fs -chown root:root /user/root and then please run the hive shell again from root user.

jyadav · ‎08-11-2016

Hi @Bala Vignesh N V The above approach is pretty good and work very well when you having small number of files but what if you have thousands or millions of files in directories? In that case its better to use Hadoop Mapreduce framework to do same job on large files but in less time. Below is an example to count lines using mapreduce. https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-count-number-of-lines-in-a-file-using-map-reduce-framework

jyadav · ‎08-06-2016

Hi @Rajib Mandal Workflow /lib is an optional thing and shouldn't cause job failure, however as per the message you posted it is a warning not error. 2016-08-05 12:49:50,403 WARN LiteWorkflowAppService:523 - SERVER[xxxx00691239901.aaa.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] libpath [hdfs://xxxx00691239901.aaa.com:8020/user/bigframe/generic/oozie_dir/lib] does not exist If your workflow is not running fine then probably we need to look deeper and find the exact error exception.

jyadav · ‎08-05-2016

I would highly suggest you to upgrade your ambari server with latest release. As a workaround restart the Ambari (server and agents) as well as Ambari Metrics and see if that resolve the issue.

jyadav · ‎08-05-2016

Hi @sankar rao This seems to be an known issue with Ambari, please refer below bug for more info and see if version matches. https://issues.apache.org/jira/browse/AMBARI-12931

Online	Offline
Last Visited	‎06-02-2017 09:42 PM

Date Registered	‎02-02-2016 09:29 AM
Last Visited	‎06-02-2017 09:42 PM
Total Messages Posted	583
Total Kudos Received	517

Re: Ambari release versioning

Re: Atlas application log showing below error

Re: failed to start hive from root

Re: corrupted block issue..i have 100+ corrupted b...

Re: What's causing ClassNotFound: RangerHiveAuthor...

Re: Error installing standalone ambari server

Error installing standalone ambari server

Re: hive view NullPointerException

hive view NullPointerException

Error connecting hive through jaspersoft

Re: Spark 2 HDP 2.5 sandbox Spark2HistoryServer UI...

Re: killed Spark Streaming job in YARN cluster mod...

Re: Can a hive external table read .deflate files?

Re: Atlas kerberos Auth rules not working

Re: Hive Server2 service getting stopped

Re: Can a hive external table read .deflate files?

Re: killed Spark Streaming job in YARN cluster mod...

Re: Atlas kerberos Auth rules not working

Re: Ambari release versioning

Atlas kerberos Auth rules not working

Re: Ambari release versioning

Ambari release versioning

Re: Atlas application log showing below error

Atlas application log showing below error

Re: Solr configuration with Standby NameNode

Re: Solr configuration with Standby NameNode

Re: failed to start hive from root

Re: failed to start hive from root

Re: failed to start hive from root

Re: failed to start hive from root

Re: failed to start hive from root

Re: How to find no of lines in all the files in a ...

Re: oozie failing with libpath under the workflow ...

Re: corrupted block issue..i have 100+ corrupted b...

Re: corrupted block issue..i have 100+ corrupted b...