Member since
01-19-2017
3682
Posts
633
Kudos Received
373
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1732 | 06-04-2025 11:36 PM | |
| 2166 | 03-23-2025 05:23 AM | |
| 1035 | 03-17-2025 10:18 AM | |
| 3966 | 03-05-2025 01:34 PM | |
| 2710 | 03-03-2025 01:09 PM |
09-23-2019
11:28 PM
Hbase shell command not executed /usr/local/Hbase# ./bin/hbase shell Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.jruby.java.invokers.RubyToJavaInvoker (file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar) to method java.lang.Object.registerNatives() WARNING: Please consider reporting this to the maintainers of org.jruby.java.invokers.RubyToJavaInvoker WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release ArgumentError: wrong number of arguments (0 for 1) method_added at file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:10 method_added at file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:129 Pattern at file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:2 (root) at file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:1 require at org/jruby/RubyKernel.java:1062 (root) at file:/usr/local/Hbase/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:42 (root) at /usr/local/Hbase/bin/../bin/hirb.rb:38
... View more
09-23-2019
06:54 AM
Just note for HDP version - 3.1 , the spark version is - Apache Spark 2.3.2 and not 2.4
... View more
09-21-2019
10:04 AM
@budati For this case define your avro schema(with one field) to read incoming flowfile with some delimiter that doesn't exist in flowfile. So that whole row will be read as string then we can filter out the records by using not like (or) using regex operator in apache calicite. Select * from flowfile where col1 not like 'SKIP' Now output flowfile will not having any records that have SKIP in them and this solution will work dynamically for any number of columns.
... View more
09-13-2019
05:51 AM
@kvinod Good then close the thread. It's always a good idea to have an external database to host not only the CM/Ambari database but also Oozie, hive, ranger etc in a multitenant architecture and maybe backup the database host periodically to avoid having your backend databases all over the cluster. 20 cents advice !
... View more
09-12-2019
03:47 AM
Thanks a lot!! I did the same with your msql settings which you had mentioned!! And it works fine
... View more
09-10-2019
02:31 PM
1 Kudo
@ranger There are three modes for Hive Metastore deployment: Embedded Metastore Not recommended for production. Local Metastore This mode allows us to have many Hive sessions i.e. many users can use the metastore at the same time. It's achieved by using any JDBC compliant like MySQL. In this case, the javax.jdo.option.ConnectionURL property is set to jdbc:mysql://host/dbname? createDatabaseIfNotExist=true, and javax.jdo.option.ConnectionDriverName is set to com.mysql.jdbc.Driver. The JDBC driver JAR file for MySQL must be on Hive’s classpath, Remote Metastore In this mode, metastore runs on its own separate JVM, not in the Hive service JVM. If other processes want to communicate with the metastore server they can communicate using Thrift Network APIs here you have the ability to have one more metastore servers in this case to provide High availability. having said that it seems you are trying to use embedded Metastore. What I advice you to do is create one as the root through the Ambari ui, it will ask you for the DBName and Host which would be where you installed the MySQL database else pre-create the metastore. The Hive database must be created before loading the Hive database schema that explains why you are getting the startup error. Using Hive with MySQL On the Ambari Server host, stage the appropriate MySQL connector for later deployment. On the Ambari Server host, Download the MySQL Connector/JDBC driver from MySQL. Runambari-server setup --jdbc-db=mysql --jdbc-driver=/path/to/mysql/mysql-connector-java.jar Confirm that mysql-connector-java.jar is in the Java share directory. ls /usr/share/java/mysql-connector-java.jar Make sure the .jar file has the appropriate permissions - 644. Execute the following command: ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar Create a user for Hive and grant it permissions. using the MySQL database admin utility: # mysql -u root -p
CREATE USER '[HIVE_USER]'@'localhost' IDENTIFIED BY '[HIVE_PASSWORD]';
GRANT ALL PRIVILEGES ON *.* TO '[HIVE_USER]'@'localhost';
CREATE USER '[HIVE_USER]'@'%' IDENTIFIED BY '[HIVE_PASSWORD]';
GRANT ALL PRIVILEGES ON *.* TO '[HIVE_USER]'@'%';
CREATE USER '[HIVE_USER]'@'[HIVE_METASTORE_FQDN]' IDENTIFIED BY '[HIVE_PASSWORD]';
GRANT ALL PRIVILEGES ON *.* TO '[HIVE_USER]'@'[HIVE_METASTORE_FQDN]';
FLUSH PRIVILEGES; Where [HIVE_USER] is your desired Hive user name, [HIVE_PASSWORD] is your desired Hive user password and [HIVE_METASTORE_FQDN] is the Fully Qualified Domain Name of the Hive Metastore host. Create the Hive database. The Hive database must be created before loading the Hive database schema. # mysql -u root -p
CREATE DATABASE [HIVE_DATABASE] Where [HIVE_DATABASE] is your desired Hive database name. After the above step then in the Ambari UI when you reach the Hive Metastpre configuring stage use the same credentials and the "test" should succeed and that should fire up when you start all the HDP components HOpe that helps
... View more
08-30-2019
01:49 AM
@irfangk1 If its an HDP cluster then I assume you are using Ambari for managing the HDF cluster, you will need to first prepare the 2 new hosts see the Prepare the EnvironmentCloudera document Then add the hosts to the cluster see Add host to cluster Thereafter add HDF to these 2 new nodes it follows the same procedure as adding HDF Services on an existing HDP Cluster hTH
... View more
08-25-2019
12:20 PM
1 Kudo
@Manoj690 Going through your logs I can see that the Namenode is in SAFE MODE, and in this case it won't allow you to change the status of any file in the cluster including the logs. 2019-08-22 12:31:01,376 [server.Accumulo] INFO : Attempting to talk to zookeeper 2019-08-22 12:31:01,681 [server.Accumulo] INFO : ZooKeeper connected and initialized, attempting to talk to HDFS 2019-08-22 12:31:01,946 [server.Accumulo] WARN : Waiting for the NameNode to leave safemode 2019-08-22 12:31:01,946 [server.Accumulo] INFO : Backing off due to failure; current sleep period is 1.0 seconds 2019-08-22 12:31:02,950 [server.Accumulo] WARN : Waiting for the NameNode to leave safemode 2019-08-22 12:31:02,950 [server.Accumulo] INFO : Backing off due to failure; current sleep period is 2.0 seconds 2019-08-22 12:31:04,954 [server.Accumulo] WARN : Waiting for the NameNode to leave safemode To resolve the issue can you do the following as hdfs user $ hdfs dfsadmin -safemode get Safe mode is OFF The above is the desired output but if you get ON then proceed like below First backup your FS Image & Edits $ hdfs dfsadmin -saveNamespace Then exit the safemode $ hdfs dfsadmin -safemode leave Once successful then revalidate $ hdfs dfsadmin -safemode get This time it should be off and you can now successfully restart the failed services from Ambari everything should succeed HTH
... View more
08-18-2019
01:27 AM
@ray_teruya If you found this answer addressed your question, please take a moment to log in and click the "kudos" link on the answer. That would be a great help to Community users to find the solution quickly for these kinds of errors.
... View more