Member since
03-27-2017
20
Posts
1
Kudos Received
0
Solutions
10-03-2020
09:50 AM
curl -u *********:************** -H "X-Requested-By:ambari" -i GET http://bathdi-pp-ne-petra-hive-prod-02.azurehdinsight.net:8080/api/v1/clusters/cl1/components?fields=host_components/HostRoles/state Getting error : curl: (6) Could not resolve host: GET
... View more
08-03-2020
07:23 AM
Hi @deepesh1, Please find the output: /usr/hdp/current/hive-server2/bin/schematool -dbType mysql -initSchema -verbose SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Initializing the schema to: 3.1.1000 Metastore connection URL: jdbc:mysql://fgpoc.ambarisvr.com/ Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: ambari_user Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. Starting metastore schema initialization to 3.1.1000 Initialization script hive-schema-3.1.1000.mysql.sql Connecting to jdbc:mysql://fgpoc.ambarisvr.com/ Connected to: MySQL (version 5.7.31-0ubuntu0.16.04.1) Driver: MySQL Connector/J (version mysql-connector-java-8.0.21 (Revision: 33f65445a1bcc544eb0120491926484da168f199)) Transaction isolation: TRANSACTION_READ_COMMITTED 0: jdbc:mysql://fgpoc.ambarisvr.com/> !autocommit on Autocommit status: true 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */ No rows affected (0.01 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET NAMES utf8 */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40103 SET TIME_ZONE='+00:00' */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */ No rows affected (0 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */ No rows affected (0.004 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */ No rows affected (0.002 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET @saved_cs_client = @@character_set_client */ No rows affected (0.002 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> /*!40101 SET character_set_client = utf8 */ No rows affected (0.001 seconds) 0: jdbc:mysql://fgpoc.ambarisvr.com/> CREATE TABLE IF NOT EXISTS `BUCKETING_COLS` ( `SD_ID` bigint(20) NOT NULL, `BUCKET_COL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `INTEGER_IDX` int(11) NOT NULL, PRIMARY KEY (`SD_ID`,`INTEGER_IDX`), KEY `BUCKETING_COLS_N49` (`SD_ID`), CONSTRAINT `BUCKETING_COLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 Error: No database selected (state=3D000,code=1046) Closing: 0: jdbc:mysql://fgpoc.ambarisvr.com/ Schema initialization FAILED! Metastore state would be inconsistent! Underlying cause: java.io.IOException : Schema script failed, errorcode 2 org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent! at org.apache.hadoop.hive.metastore.tools.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:66) at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.run(MetastoreSchemaTool.java:446) at org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) Caused by: java.io.IOException: Schema script failed, errorcode 2 at org.apache.hive.beeline.schematool.HiveSchemaTool.execSql(HiveSchemaTool.java:105) at org.apache.hive.beeline.schematool.HiveSchemaTool.execSql(HiveSchemaTool.java:83) at org.apache.hadoop.hive.metastore.tools.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:62) ... 8 more *** schemaTool failed ***
... View more
11-15-2017
12:08 PM
Generally, If there are too many small stripes with many columns or few columns with huge value in string column, overhead of storing metadata(column stats) might be higher than protobuf size. i.e., In Orc, index data will help to filter out stripes using min/max of the column. For String Column, min/max & sum of the lengths of the values are recorded in metadata. -- Min/Max of string column is higher than protobuf maxLimit which is throwing this exception. Whereas for Binary Column, total length of all binary columns are recorded in metadata. -- This will not exceed protobuf maxLimit & will not have min/max values to filter the stripes. If column values are huge & query will not have filters on such huge columns, its better to use binary data type as it stores less information in metadata.
... View more
05-16-2018
02:25 PM
This might help to other people. This seems like a bug : https://issues.apache.org/jira/browse/AMBARI-21687 The bug is fixed in 2.6.0.0 branch only ( not fixed in 2.4.x.x or 2.5.x.x versions of ambari). So, The available workaround is to add the node via 'Admin' user or use the ambari 2.6.x
... View more
06-30-2017
10:13 AM
@krajguru Thanks !!
... View more
06-30-2017
10:09 AM
1 Kudo
@Pankaj Degave
You can use the below call to get only the required fields mentioned in Ranger UI. curl -o ranger.query --negotiate -u : -X GET "http://<ambari-infra-solr-instance-hostname>:8886/solr/ranger_audits_shard1_replica1/select?q=*%3A*&fq=evtTime%3A%5B2017-06-11T10%3A44%3A00Z+TO+NOW%5D&fl=policy,evtTime,reqUser,repo,resource,resype,access,result,enforcer,cliIP,cluster,event_count&sort=evtTime+desc&start=0&rows=307600&wt=csv&version=2"
Depending on what all logs you want to pull adjust the evtTime, the above query pulls all the audit records, change the evtTime to the timestamp of the first record in ranger.
... View more
06-15-2017
07:10 AM
@hari Kishore javvaji Dr.who is the default username/static for hadoop core property : "hadoop.htttp.staticuser.user" The description say's : The user name to filter as, on static web filters while rendering content. An example use is the HDFS web UI (user to be used for browsing files)
See this: [1] http://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/core-default.xml Hope this answered your question.
... View more