About sen_ke

sen_ke · ‎01-22-2019

@Jay Kumar SenSharma Thanks for your reply, i deleted some logs to recover Namenode service status to running (service can't start when log space full), and my logs over 50GB per year, so i will study log4j extra to save space usage.

sen_ke · ‎01-21-2019

Hi, there are many log files in /var/log/hadoop/hdfs, and file name like hdfs-audit.log.2018-06-01, and some file name like hadoop-hdfs-namenode-hostname are very large. may i delete it all? (this host running namenode, ranger, datanode flume and node manager.) and if any configure can setting those log as auto cycling or auto delete? Thanks

sen_ke · ‎10-24-2018

@Jay Kumar SenSharma i'm tried rebalance hdfs, but failed and show 0 moved / 0 left / 0 being processed, stderr: /var/lib/ambari-agent/data/errors-1216.txt (can not find this log) stdout: /var/lib/ambari-agent/data/output-1216.txt (can not find this log) looks no work, if need any action before rebalance?

sen_ke · ‎10-22-2018

@Jay Kumar SenSharma Thanks for reply, so it only rebalance manually right?

sen_ke · ‎10-22-2018

Hi, i added 2 new data node to hdfs cluster and in the same rack, will data auto move to new data node? my old data node space are 10T per node, new data node space are 20T per node, when old data node space full, new data node only 50% usage right? or will hdfs put more data to large data node? or i need manually excute hdfs balancer when old data node full? Thanks.

sen_ke · ‎07-25-2018

@Adi Jabkowsky yes, i'm using ambari manager, and it's look easy, just i have many file maybe need wait a long time to replicate data, thanks for reply.

sen_ke · ‎07-25-2018

Hi everyone, i want decommision datanode and keep other service (prevent disk space not enough), in the server have Hive metastore/hiveserver2/flume/node manager service, i want keep these services. how to do? may i also remove node manager? Thanks.

sen_ke · ‎10-23-2017

@Vishal Gupta in my case, i'm change WEBHDFS service config from {{webhdfs_service_urls}} to http://My_NameNode:50070/webhdfs in xml and your command URL should use "ui" or "adminui" instead "default"

sen_ke · ‎10-16-2017

oh! i think i solved this problem, after add user1, i restart knox all service, and start DEMO LDAP, and DEMO LDAP looks no restart, so i stop DEMO LDAP then restart again, it's worked! thanks @Aditya Sirna

sen_ke · ‎10-16-2017

@Aditya Sirna 2017-10-16 09:40:15,499 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=user1,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: user1 2017-10-16 09:40:15,509 INFO hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(203)) - Could not login: org.apache.shiro.authc.UsernamePasswordToken - user1, rememberMe=false (10.243.91.58) 2017-10-16 09:40:15,509 ERROR hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(205)) - Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user uid=user1,ou=people,dc=hadoop,dc=apache,dc=org]

Online	Offline
Last Visited	‎01-22-2019 07:28 AM

Member Since	‎09-20-2017 06:44 AM
Last Visited	‎01-22-2019 07:28 AM
Posts	22
Kudos received	1

Cloudera Community

Re: rest api via knox only admin can access

Re: May i delete /var/log/hadoop/hdfs files

May i delete /var/log/hadoop/hdfs files

Re: will data auto move to new data node?

Re: will data auto move to new data node?

will data auto move to new data node?

Re: decommision datanode and keep other service

decommision datanode and keep other service

Re: knox responsed 404 not found when use file vie...

Re: rest api via knox only admin can access

Re: rest api via knox only admin can access