Member since
03-06-2020
392
Posts
50
Kudos Received
34
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
422 | 07-23-2024 10:52 PM | |
811 | 05-16-2024 12:27 AM | |
2041 | 05-01-2024 04:50 AM | |
988 | 03-19-2024 09:23 AM | |
1026 | 03-04-2024 05:33 AM |
08-06-2024
10:06 PM
Hi @Supernova Can you try this? SELECT a.currency as currency, SUM(coalesce(a.ColA, 0) + coalesce(a.ColB, 0) + coalesce(a.ColC, 0) + coalesce(b.Col1, 0) + coalesce(b.Col2, 0) + coalesce(b.Col3, 0)) as sales_check FROM db.sales a INNER JOIN db.OTHER_sales b ON a.currency = b.currency WHERE a.DateField = '2024-06-30' AND b.DateField = '2024-06-30' GROUP BY a.currency; Regards, Chethan YM
... View more
08-06-2024
10:04 PM
Hi @Marks_08 To insert into a managed table from an external table created with HBase structure in CDP, you need to ensure that Hive can properly connect to HBase. This typically involves making sure that the Hive service is configured to access HBase correctly. One common issue is that the necessary configuration files, such as hbase-site.xml, are not accessible to Hive, leading to connection issues. Here’s what you can do to address this: 1. Copy hbase-site.xml to Hive Configuration Directory You need to copy the hbase-site.xml file to the Hive configuration directory. This file contains the necessary configuration for Hive to connect to HBase. sudo cp /etc/hbase/conf/hbase-site.xml /etc/hive/conf/ 2. Verify HBase Configuration Ensure that the hbase-site.xml file contains the correct configuration and points to the correct HBase nodes. The key configurations to check are: hbase.zookeeper.quorum hbase.zookeeper.property.clientPort These settings should correctly point to the Zookeeper quorum and client port used by your HBase cluster. 3. Restart Hive Service After copying the hbase-site.xml file, you might need to restart the Hive service to ensure it picks up the new configuration. 4. Check Hive and HBase Connectivity Make sure that the Hive service can properly communicate with HBase by running a simple query that accesses HBase data through Hive. Regards, Chethan YM
... View more
08-06-2024
10:00 PM
Hi @Supernova , Can you try this below? SELECT Currency, CASE WHEN COALESCE(SUM(ColA), 0) + COALESCE(SUM(ColB), 0) = COALESCE(SUM(ColC), 0) THEN 0 ELSE COALESCE(SUM(ColA), 0) + COALESCE(SUM(ColB), 0) - COALESCE(SUM(ColC), 0) END AS sales_check FROM Sales GROUP BY Currency; Regards, Chethan YM
... View more
07-23-2024
10:56 PM
@rizalt Yes, the Key Version Numbers (KVNO) of different principals can indeed be different. Each principal in Kerberos can have its own KVNO, which is an identifier that increments each time the key for that principal is changed. Reference: https://web.mit.edu/kerberos/www/krb5-latest/doc/user/user_commands/kvno.html#:~:text=specified%20Kerberos%20principals Regards, Chethan YM
... View more
07-23-2024
10:52 PM
1 Kudo
Hi @therealsrikanth You can follow this if you do not need CM or ambari. Step 1: Install Hadoop Download Hadoop: Download the latest stable release of Hadoop from the Apache Hadoop website. tar -xzf hadoop-3.3.4.tar.gz sudo mv hadoop-3.3.4 /usr/local/hadoop 2. Configure Hadoop Environment Variables: Add the following lines to your .bashrc or .profile file. export HADOOP_HOME=/usr/local/hadoop export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin 3. Edit Configuration Files: Edit the core configuration files in $HADOOP_CONF_DIR. core-site.xml: <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master-node:9000</value> </property> </configuration> hdfs-site.xml: <configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:///usr/local/hadoop/hdfs/namenode</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:///usr/local/hadoop/hdfs/datanode</value> </property> </configuration> mapred-site.xml: <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration> yarn-site.xml: <configuration> <property> <name>yarn.resourcemanager.address</name> <value>master-node:8032</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration> 4. Format the NameNode: hdfs namenode -format 5. Start Hadoop Services: start-dfs.sh start-yarn.sh Step 2: Install Zookeeper: 1. Download and Extract Zookeeper: https://downloads.apache.org/zookeeper tar -xzf apache-zookeeper-3.8.1-bin.tar.gz sudo mv apache-zookeeper-3.8.1-bin /usr/local/zookeeper 2. Configure Zookeeper: Create a configuration file at /usr/local/zookeeper/conf/zoo.cfg tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=master-node1:2888:3888 server.2=master-node2:2888:3888 server.3=slave-node1:2888:3888 3. Start Zookeeper: /usr/local/zookeeper/bin/zkServer.sh start Step 3: Install HBase: 1. Download HBase: Download the latest stable release of HBase from the Apache HBase website. tar -xzf hbase-2.4.16-bin.tar.gz sudo mv hbase-2.4.16 /usr/local/hbase 2. Configure HBase: hbase-site.xml <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://master-node:9000/hbase</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>master-node1,master-node2,slave-node1</value> </property> </configuration> 3. Start HBase: /usr/local/hbase/bin/start-hbase.sh Step 4: Verify Installation: Check the Hadoop services using the web interfaces: NameNode: http://master-node:9870 ResourceManager: http://master-node:8088 HBase: http://master-node:16010 Additional Resources Apache Hadoop Documentation Apache Zookeeper Documentation Apache HBase Documentation Regards, Chethan YM
... View more
06-11-2024
10:29 PM
1 Kudo
Hi @rizalt The error is because you have not provided keytab path here the command should look like below: > klist -k example.keytab To create the keytab you can refer any of below steps: $ ktutil
ktutil: addent -password -p myusername@FEDORAPROJECT.ORG -k 42 -f
Password for myusername@FEDORAPROJECT.ORG:
ktutil: wkt /tmp/kt/fedora.keytab
ktutil: q Then kinit -kt /tmp/kt/fedora.keytab myusername@FEDORAPROJECT.ORG Note: Replace the username and REALM as per your cluster configurations. Regards, Chethan YM
... View more
05-30-2024
10:24 PM
Hi @jayes May i know how you are exporting the table into HDFS? what is the command?
... View more
05-16-2024
12:27 AM
2 Kudos
Hi @d_liu_ The error shows that problem with the network connectivity between the Hive server (ip-172-19-36-68) and the HDFS namenode (ip-172-19-36-94.ap-southeast-2.compute.internal) on port 8020. So as per your comment above HDFS node does not exist in your cluster right? May i know how you are running this query like from beeline or Hue or from any third party tool? Search the above host in all the configuration files(hive-site.xml,hive-env.sh,hive.metastore.uris etc..) Try to search the above host name at client side from where you are running the query and there is a chances if you have any custom scripts or config files causing the issue, so check if you have any custom scripts to run the jobs. Regards, Chethan YM
... View more
05-15-2024
02:46 AM
1 Kudo
@hadoopranger 1. Verify if the JDBC connection string is valid and correct. 2. Try to use the latest JDBC driver version that is available. 3. Check the HS2 servers that you are connecting are accepting the connections and in good health. 4. Check if beeline is working or not. 5. If all correct enable the JDBC driver trace level logging and get the more details on the error. https://docs.cloudera.com/documentation/other/connectors/hive-jdbc/2-6-15/Cloudera-JDBC-Driver-for-Apache-Hive-Install-Guide.pdf Regards, Chethan YM
... View more
05-15-2024
02:34 AM
1 Kudo
Hello @hadoopranger Do provide the complete error stack trace from hue log and error screenshot from Hue UI to check the same. Try restarting impala, hue and see if any difference. Regards, Chethan YM
... View more