Member since
02-08-2016
793
Posts
669
Kudos Received
85
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1156 | 06-30-2017 05:30 PM | |
1560 | 06-30-2017 02:57 PM | |
1311 | 05-30-2017 07:00 AM | |
1540 | 01-20-2017 10:18 AM | |
2495 | 01-11-2017 02:11 PM |
08-29-2017
06:34 PM
@Hugo Felix This is different issue. Your database seems to be inconsistent. Can you please open a new thread and attach /var/log/ambari-server/ambari-server-check.database.log so that i can check and if possible provide you solution.
... View more
07-24-2017
11:44 AM
@czounaidou Filenames needs to be corrected - Existing - Modified /etc/hst/conf/capture-levels.json ---> /etc/hst/conf/capture_levels.json /etc/hst/conf/anonymization-rules.json ---> /etc/hst/conf/anonymization_rules.json
... View more
06-30-2017
05:30 PM
1 Kudo
@Mahendra More Do you have "Enable Group Search First" enabled? Sometime i have seen it creates problem. Also make sure your usersync and groupsync are enabled under Ambari->Ranger->RangerUserinfo
... View more
06-30-2017
02:57 PM
@chennuri gouri shankar Do you have kerberos enabled in cluster ? if so you need to use user principal and password created for the principal in HDFS repository username and password. Also use same in HDFS Ranger configs. Please check link below - https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/hdfs_plugin_kerberos.html
... View more
06-30-2017
01:46 PM
@chennuri gouri shankar Can you please attach screenshot and xa_portal error log
... View more
06-28-2017
05:14 PM
Also try same with other browser, just to make sure its not browser specific issue.
... View more
06-28-2017
05:14 PM
@Mohammad Hamdan Can you check if you have DB inconsistency ? You can try restarting ambari-server and see if it gives any warning/error with DB inconsistency.
... View more
06-28-2017
04:16 PM
1 Kudo
SYMPTOM: While performing HDP upgrade [HDP-2.3.0 upgrade to HDP-2.5.3] there was error for which we had to start ambari server. But ambari server startup failed with below error - ERROR: Caused by: Exception [EclipseLink-116] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DescriptorException
Exception Description: No conversion value provided for the value [UPGRADED] in field [host_version.state].
Mapping: org.eclipse.persistence.mappings.DirectToFieldMapping[state-->host_version.state]
Descriptor: RelationalDescriptor(org.apache.ambari.server.orm.entities.HostVersionEntity --> [DatabaseTable(host_version)])
at org.eclipse.persistence.exceptions.DescriptorException.noFieldValueConversionToAttributeValueProvided(DescriptorException.java:1066)
at org.eclipse.persistence.mappings.converters.ObjectTypeConverter.convertDataValueToObjectValue(ObjectTypeConverter.java:226)
at org.eclipse.persistence.mappings.converters.EnumTypeConverter.convertDataValueToObjectValue(EnumTypeConverter.java:141)
at org.eclipse.persistence.mappings.foundation.AbstractDirectMapping.getObjectValue(AbstractDirectMapping.java:616)
at org.eclipse.persistence.mappings.foundation.AbstractDirectMapping.valueFromRow(AbstractDirectMapping.java:1220)
at org.eclipse.persistence.mappings.DatabaseMapping.readFromRowIntoObject(DatabaseMapping.java:1539)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:462)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:1005)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:899)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:852)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:735)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:689)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:805)
at org.eclipse.persistence.queries.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:962)
at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:573)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1175)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:904)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1134)
ROOT CAUSE: There was manual entry in ambari db table "host_version" where the 'state' was modified to "UPGRADED" as shown below which cause the issue- ambari=> select * from host_version;
id | repo_version_id | host_id | state
----+-----------------+---------+---------
1 | 1 | 1 | CURRENT
2 | 1 | 3 | CURRENT
3 | 2 | 2 | UPGRADED
RESOLUTION: Steps followed for resolution - 1. Took ambari DB backup 2. Check the latest repo_version_id using below command - select (repo_version_id, version) from repo_version ;
row
---------------
(1, 2.3.0.1-15)
(2, 2.3.5.3-8)
3. Check for latest version and repo_version_id from above command and modified respective repo_version_id in "host_version" table pointing the status to CURRENT and made rest of the entries to INSTALLED. So the output should look like below - ambari=> select * from host_version;
id | repo_version_id | host_id | state
----+-----------------+---------+---------
1 | 1 | 1 | INSTALLED
2 | 1 | 3 | INSTALLED
3 | 2 | 2 | CURRENT
4. Restart ambari-server.
... View more
- Find more articles tagged with:
- ambari-server
- Hadoop Core
- hdp-upgrade
- Issue Resolution
Labels:
06-28-2017
03:50 PM
1 Kudo
SYMPTOM: During upgrade from HDP 2.4.3 to HDP2.6.0 the express upgrade failed on the step "Yarn NodeMangers / Restarting NodeManger." ERROR: 2017-06-22
15:02:22,791 INFO recovery.ZKRMStateStore
(ZKRMStateStore.java:runWithRetries(1227)) - Exception while executing a ZK
operation. org.apache.zookeeper.KeeperException$AuthFailedException:
KeeperErrorCode = AuthFailed for /rmstore at
org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
ROOT CAUSE: The issue was with property Ambari-Yarn-Configs-"yarn-resourcemanager-zk-address" was set to localhost:2181 RESOLUTION: Modified address to "{zk-server-hostname}:2181" resolved the issue. After which node managers were able to start successfully.
... View more
- Find more articles tagged with:
- Hadoop Core
- hdp-upgrade
- Issue Resolution
- nodemanager
- YARN
Labels:
05-30-2017
07:00 AM
1 Kudo
@Pravin Bhagade
Hi praveen, it seemse you are performing some wrong step. I tested it on my local system and it works as expected. Will send you the video link of the steps performed.
... View more
04-04-2017
01:40 PM
2 Kudos
@rahul gulati Can you make sure your TEST connection works successfully for Ranger HIVE repository?
... View more
03-06-2017
12:57 PM
@Marco Silva Can you pass me output of below command - Login to ambari DB and execute - ambari=> select * from clusters ;
... View more
02-22-2017
06:11 PM
1 Kudo
@Colin Cunningham This seems to be a BUG- Pls refer - https://issues.apache.org/jira/browse/AMBARI-13345 https://hortonworks.jira.com/browse/BUG-45151 [Hortonworks internal jira for reference]
... View more
02-22-2017
08:57 AM
@Sachin Ambardekar Can you uninstall ambari agent on the datanode3 and install ambari agent with same version as ambari-server on datanode3 [ie. 2.4.2.0-136] Ambari agent version output should look something like - [ambari@datanode3 ~]$ ambari-agent --version
2.4.2.0-136
... View more
02-22-2017
07:06 AM
@rahul gulati Check below things - 1. /etc/ambari-agent/conf/ambari-agent.ini has entry pointing to ambari server -> server=<ambari_host> 2. Check if iptables and selinux are stopped and disabled. 3. Try restarting the agent $ambari-agent restart 4. Check if there is issue with /etc/hosts file. Wrong/incorrect intry in /etc/hosts file can create such issue.
... View more
02-22-2017
07:03 AM
PLease check if you have same repo file for ambari in /etc/yum.repos.d/ folder. You can copy ambari repo file from ambari server node to datanode3 and reinstall ambari agent package. yum remove ambari-agent yum install ambari-agent This should resolve the issue.
... View more
02-22-2017
07:01 AM
@Sachin Ambardekar
Can you make sure ambari server version and agent version are same -using below command - ambari-server --version -- [ambari server] ambari-agent --version -- [datanode3] Below error clearly states its version mismatch issue - ERROR 2017-02-21 22:29:57,286 Controller.py:155 - Cannot register host with non compatible agent version, hostname=datanode3.localdomain, agentVersion=2.1.1, serverVersion=2.4.2.0
... View more
02-20-2017
05:12 PM
@Marco Silva Can you do "select * from upgrade" table and check if any data exist in that table? Also try executing below command - --select distinct(request_id,status) from host_role_command where status IN ('PENDING', 'HOLDING_FAILED','ABORTED');
--update host_role_command set status = 'ABORTED' where status IN ('PENDING', 'HOLDING_FAILED') and request_id = 455;
... View more
02-18-2017
11:24 AM
@Joel Patterson You can ignore the crc check errors. Those will not impact on ldap startup. For "ldap_sasl_interactive_bind_s: Invalid credentials (49" it seems there is either wrond DN or password. Can you please check and confirm.
... View more
02-16-2017
05:03 PM
Can you make sure port[50010] are open? Chcek if you are able to connect to 50010, try writing some sample data to remote cluster from existing node.
... View more
02-16-2017
01:58 PM
@hardik desai
Can you check the contents from the file from namenode machine ? $cat /etc/hadoop/conf/dfs.exclude.
... View more
02-16-2017
01:14 PM
@Maher Hattabi
what is the error you are hitting? Can you paste the error ? Also what is the size of data you are loading in hadoop ?
... View more
02-16-2017
11:53 AM
@jzhang From the error its suspected that you have not properly defined principal name in configs - Can you please revisit the configs again "Kerberos principal should have 3 parts: hive" It should be - "hive/_HOST@REALM" Check properties below in HIVE configs - hive.server2.authentication.kerberos.principal
hive.metastore.kerberos.principal
... View more
02-16-2017
10:15 AM
Try to check the "dfs.hosts.exclude" property of your "hdfs-site.xml" file.
... View more
02-16-2017
10:14 AM
@hardik desai Are the DN's functioning properly ? Can you check if the datanodes are in exclude list ? Suspected from below error from logs - There are 3 datanode(s) running and 3 node(s) are excluded in this operation.
... View more
02-15-2017
02:58 PM
@rahul gulati Can you make sure the output of "$hostname" and "$hostname -f" are same?
... View more
02-15-2017
01:09 PM
@Maher Hattabi Please check link below - https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/WebHDFS.html For examples also refer - https://dzone.com/articles/hadoop-rest-api-webhdfs https://bighadoop.wordpress.com/2013/06/02/hadoop-rest-api-webhdfs/
... View more
02-15-2017
11:08 AM
@jaime
@Jaime Garcia No Problem. Will be happy to help you there. You can refer below link to start/stop services from ambari ui - https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_Ambari_Users_Guide/content/_starting_and_stopping_all_services.html For ranger UI you can hit - http://<ranger_host>:6180/
... View more
02-15-2017
09:26 AM
@Jaime Garcia 1. It seems that Ranger admin service is down. make sure you always restart ranger admin service 1st before starting any other services [For those services for which enabled plugin] 2. It might be case your "amb_ranger_admin" password is not in sync and you might see Alerts in Ambari UI wrt "Ranger Admin Password Check" as shown below - You can resolve the "Ranger Admin Password Check" issue using below steps - 1. Login to Ranger UI->Settings->users/Groups->select "amb_ranger_admin" user and click on change password - Save and now login to AmbariUI->Services->Ranger->Configs->Advance and drop same password for "amb_ranger_admin" user- Save and Restart Ranger service. Now try restarting Hbase service now. Let me know if that works.
... View more