i need some help with this error??!!!! hive> (base) hadoop@sajanraj:~$ schematool -dbType mysql -initSchema --verbose SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/apache-hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://192.168.5.249:3306/hive?createDatabaseIfNotExist=true Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: hiveuser Wed Jun 12 14:34:03 IST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Starting metastore schema initialization to 3.1.0 Initialization script hive-schema-3.1.0.mysql.sql Connecting to jdbc:mysql://192.168.5.249:3306/hive?createDatabaseIfNotExist=true Wed Jun 12 14:34:03 IST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Connected to: MySQL (version 5.7.26-0ubuntu0.18.04.1) Driver: MySQL Connector Java (version mysql-connector-java-5.1.45 ( Revision: ${revinfo.commit} )) Transaction isolation: TRANSACTION_READ_COMMITTED 0: jdbc:mysql://192.168.5.249:3306/hive> !autocommit on Autocommit status: true 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */ No rows affected (0.002 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET NAMES utf8 */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40103 SET TIME_ZONE='+00:00' */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */ No rows affected (0.005 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */ No rows affected (0.002 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @saved_cs_client = @@character_set_client */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = utf8 */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> CREATE TABLE IF NOT EXISTS `BUCKETING_COLS` ( `SD_ID` bigint(20) NOT NULL, `BUCKET_COL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `INTEGER_IDX` int(11) NOT NULL, PRIMARY KEY (`SD_ID`,`INTEGER_IDX`), KEY `BUCKETING_COLS_N49` (`SD_ID`), CONSTRAINT `BUCKETING_COLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 No rows affected (0.007 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = @saved_cs_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @saved_cs_client = @@character_set_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = utf8 */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> CREATE TABLE IF NOT EXISTS `CDS` ( `CD_ID` bigint(20) NOT NULL, PRIMARY KEY (`CD_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = @saved_cs_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @saved_cs_client = @@character_set_client */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = utf8 */ No rows affected (0 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> CREATE TABLE IF NOT EXISTS `COLUMNS_V2` ( `CD_ID` bigint(20) NOT NULL, `COMMENT` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, `TYPE_NAME` MEDIUMTEXT DEFAULT NULL, `INTEGER_IDX` int(11) NOT NULL, PRIMARY KEY (`CD_ID`,`COLUMN_NAME`), KEY `COLUMNS_V2_N49` (`CD_ID`), CONSTRAINT `COLUMNS_V2_FK1` FOREIGN KEY (`CD_ID`) REFERENCES `CDS` (`CD_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 No rows affected (0.002 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = @saved_cs_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET @saved_cs_client = @@character_set_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = utf8 */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> CREATE TABLE IF NOT EXISTS `DATABASE_PARAMS` ( `DB_ID` bigint(20) NOT NULL, `PARAM_KEY` varchar(180) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, `PARAM_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, PRIMARY KEY (`DB_ID`,`PARAM_KEY`), KEY `DATABASE_PARAMS_N49` (`DB_ID`), CONSTRAINT `DATABASE_PARAMS_FK1` FOREIGN KEY (`DB_ID`) REFERENCES `DBS` (`DB_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> /*!40101 SET character_set_client = @saved_cs_client */ No rows affected (0.001 seconds) 0: jdbc:mysql://192.168.5.249:3306/hive> CREATE TABLE `CTLGS` ( `CTLG_ID` BIGINT PRIMARY KEY, `NAME` VARCHAR(256), `DESC` VARCHAR(4000), `LOCATION_URI` VARCHAR(4000) NOT NULL, UNIQUE KEY `UNIQUE_CATALOG` (`NAME`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 Error: Table 'CTLGS' already exists (state=42S01,code=1050) Closing: 0: jdbc:mysql://192.168.5.249:3306/hive?createDatabaseIfNotExist=true org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !! Underlying cause: java.io.IOException : Schema script failed, errorcode 2 org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !! at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:594) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567) at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.io.IOException: Schema script failed, errorcode 2 at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1226) at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1204) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:590) ... 8 more *** schemaTool failed *** ... View more

@Joshua Adeleke There is obviously something else going on within your system that is affecting leader election. When you start you NiFi, do you see a leader election/Cluster coordinator count down timer running? Is your NiFi having trouble talking to your Zookeeper? Looks like you are having timeout issues talking to your zookeeper. I still don't understand why you are running your NiFi as a 1 node cluster if all you want is a single standalone instance of NiFi. A NiFi configured as a standalone instance does not need zookeeper and also does not perform election of cluster coordinator or primary node. Setting the following property in your nifi.properties and restarting will make you NiFi a truly standalone instance: nifi.cluster.is.node=false Matt ... View more

you just copy the query generated in any one of the output flowfile of the ReplaceText processor and execute that in Oracle DB.If it is executed successfully in DB here also will execute. ... View more

Try below command and let me know if it works for you. It worked for me. hdfs dfs -chmod -R /tmp/hive , Try below command and let me know if it worked. hdfs dfs -chmod -R 777 /tmp/hive ... View more

Thank you @mclark @Timothy Spann @Andrew Grande. I eventually only needed to save and import the old templates. I forgot to mention that the HDF 2.0 had already been installed on the new server. ... View more

Hi All, even i too need one simple example how to set mail alerts with business logic when something went wrong in NIFI. if possible share one sample link. I am very new to NIFI, now i need this requirement to my current support project, Thanks in advance. ... View more

@David Kjerrumgaard I found out the issue was with my csv file. It had to do with the column names i was ingesting in the csv file. Also, the turning the Obtain Generated Keys property to false worked as well in my case. ... View more

Thank you @Ryan Cicak. I checked the syslog and it gave an hint of issues in the hdfs staging directory. I plan to use the yarn user to run but i would think any user should be able to run it. I can launch it successfully in CLI with user HDFS. Job init failed org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://server:8020/user/hdfs/.staging/job_1469778114081_4925/job.splitmetainfo ... View more