About bigdatacn

bigdatacn · ‎06-16-2016

I've configured hue on hdp2.4.2. and refer below docs. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_installing_manually_book/content/configure_hdp_support_hue.html and and encountered below issues. anyone can help this Traceback (most recent call last): File "/usr/lib/hue/desktop/core/src/desktop/views.py", line 355, in _get_config_errors error_list.extend(validator(user)) File "/usr/lib/hue/desktop/libs/liboozie/src/liboozie/conf.py", line 80, in config_validator message=_('Oozie Share Lib not installed in default location.'))) File "/usr/lib/hue/desktop/core/src/desktop/lib/conf.py", line 658, in validate_path if path is None or not fs.exists(path): File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 217, in exists return self._stats(path) is not None File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 205, in _stats raise ex WebHdfsException: RemoteException: User: httpfs is not allowed to impersonate hdfs (error 500) [16/Jun/2016 09:10:34 +0000] supervisor ERROR Exception in supervisor main loop Traceback (most recent call last): File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main wait_loop(sups, options) File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 424, in wait_loop time.sleep(1) File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 221, in sig_handler raise SystemExit("Signal %d received. Exiting" % signum) SystemExit: Signal 15 received. Exiting [16/Jun/2016 02:10:47 +0000] webhdfs ERROR Failed to determine superuser of WebHdfs at http://etl01.prod2.everstring.com:14000/webhdfs/v1/: RemoteException: User: httpfs is not allowed to impersonate hue (error 500)

bigdatacn · ‎05-31-2016

I tried hadoop.http.staticuser.user=yarn in core-site.xml. it doesn't work on hdp2.4.2. When I submit jobs with that setting. it will show some logs like below. For more detailed output, check application tracking page:http://master02.com:8088/cluster/app/application_1464688798017_0006Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_e20_1464688798017_0006_02_000001 Exit code: 15 Stack trace: ExitCodeException exitCode=15: at org.apache.hadoop.util.Shell.runCommand(Shell.java:576) at org.apache.hadoop.util.Shell.run(Shell.java:487) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

bigdatacn · ‎05-30-2016

I've enabled yarn ranger plugin under ambari.also configured yarn.admin.acl with "dr.who". but after I submit job, I found I can't access logs. It shows below: User [dr.who] is not authorized to view the logs for container_e18_1464594716018_0007_01_000001 in log file [host1_45454] No logs available for container container_e18_1464594716018_0007_01_000001 anyone has same problems?thanks in advance.

bigdatacn · ‎05-04-2016

it's worked Thanks

bigdatacn · ‎04-28-2016

@Artem Ervits thanks, I will try it.

bigdatacn · ‎04-28-2016

Is there any docs for this purpose? or any blog can be referred? Thanks a lot.

bigdatacn · ‎03-23-2016

I found our resource manager UI access quite slowly. sometimes it will take over several mins for show all jobs. I found some exception from resource log as below, btw I found resource manager permgen size default size is 20MB and it's usage over 99.3%. is there any solution for this? ###part resource manger logs########################### I/O exception (java.net.ConnectException) caught when processing request: Connection refused

bigdatacn · ‎03-03-2016

@Neeraj Sabharwal sure

bigdatacn · ‎03-02-2016

@Neeraj Sabharwal I've fixed by myself. by setting SYNC_LDAP_USER_SEARCH_FILTER to "uid=*"

bigdatacn · ‎03-02-2016

@Neeraj Sabharwal thanks, btw, I encountered another issues when sync LDAP user/groups. Can you help on this? Thanks. The error messages: 02 Mar 2016 06:38:09 INFO LdapUserGroupBuilder [UnixUserSyncThread] - LdapUserGroupBuilder initialization completed with -- ldapUrl: ldaps://52.17.129.212:636, ldapBindDn: cn=admin,dc=abc,dc=com, ldapBindPassword: ***** , ldapAuthenticationMechanism: simple, userSearchBase: ou=people,dc=abc,dc=com, userSearchScope: 2, userObjectClass: person, userSearchFilter: -, extendedSearchFilter: (&(objectclass=person)(-)), userNameAttribute: uid, userSearchAttributes: [uid, memberof] 02 Mar 2016 06:38:09 ERROR UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 300000 milliseconds. Error details: javax.naming.directory.InvalidSearchFilterException: Missing 'equals'; remaining name 'ou=people,dc=abc,dc=com' at com.sun.jndi.ldap.Filter.encodeSimpleFilter(Filter.java:330) at com.sun.jndi.ldap.Filter.encodeFilter(Filter.java:146) at com.sun.jndi.ldap.Filter.encodeFilterList(Filter.java:741) at com.sun.jndi.ldap.Filter.encodeComplexFilter(Filter.java:657) at com.sun.jndi.ldap.Filter.encodeFilter(Filter.java:104) at com.sun.jndi.ldap.Filter.encodeFilterString(Filter.java:74) at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:547) at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1985) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1847) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1772) at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:386) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:356) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:339) at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:267) at com.xasecure.ldapusersync.process.LdapUserGroupBuilder.updateSink(LdapUserGroupBuilder.java:195) at com.xasecure.usergroupsync.UserGroupSync.run(UserGroupSync.java:59) at java.lang.Thread.run(Thread.java:745)

Online	Offline
Last Visited	‎10-24-2019 04:29 AM

Member Since	‎11-17-2017 12:37 AM
Last Visited	‎10-24-2019 04:29 AM
Posts	55
Kudos received	14

Cloudera Community

Re: How to install spark2 on rpm installed cluster

Re: ambari Datanode stale status

about hue access HDFS ha

Re: HDP2.4.2 Spark jobs log can't be access

HDP2.4.2 Spark jobs log can't be access

Re: we want to integrate hbase in hue on HDP2.2

Re: we want to integrate hbase in hue on HDP2.2

we want to integrate hbase in hue on HDP2.2

HDP 2.2 resource manager UI open slowly

Re: ranger usersync connect to ldap failed

Re: ranger usersync connect to ldap failed

Re: ranger usersync connect to ldap failed