Member since
03-06-2020
406
Posts
56
Kudos Received
36
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
921 | 11-21-2024 10:40 PM | |
885 | 11-21-2024 10:12 PM | |
2708 | 07-23-2024 10:52 PM | |
2026 | 05-16-2024 12:27 AM | |
6743 | 05-01-2024 04:50 AM |
03-04-2024
02:06 AM
1 Kudo
@Shivakuk When you replace a disk in an HDFS cluster, especially if it's a DataNode disk, the Hadoop system should handle data replication and rebalancing automatically. This means that once the new disk is added and the DataNode is back online, HDFS will redistribute the data across the cluster to maintain the configured replication factor. If data was wiped during or after the disk replacement process, it's critical to investigate why this occurred and take measures to prevent data loss in the future. Ensure that proper backup and recovery procedures are in place, and consider implementing data mirroring or replication to minimize the risk of data loss due to hardware failures. Regards, Chethan YM
... View more
02-21-2024
01:44 AM
1 Kudo
Hi, The error message you've provided indicates a problem with the agent's ability to send heartbeats to the master. This can occur due to various reasons, such as network issues, firewall settings, or misconfigurations. Check the master server is available and reachable, check the network/firewall settings in the system, re-check the agent config files regarding the hostnames, port numbers etc... Regards, Chethan YM
... View more
02-20-2024
11:30 PM
Hi @Timo , In Apache Hadoop, the directories where HDFS DataNodes and YARN NodeManagers store their data and logs are typically configured using the "dfs.datanode.data.dir" and "yarn.nodemanager.local-dirs" properties respectively. To prevent HDFS DataNodes and YARN NodeManagers from writing data to the root-vg directory when disks fail, you should ensure that these properties are configured correctly to point to directories on the healthy disks or storage volumes. -> Configure HDFS DataNode Data Directories:Set the "dfs.datanode.data.dir" property in "hdfs-site.xml" to specify the directories where HDFS DataNodes should store their data. Make sure to list the directories on the healthy disks or storage volumes. -> Configure YARN NodeManager Local Directories: Set the "yarn.nodemanager.local-dirs" property in "yarn-site.xml" to specify the directories where YARN NodeManagers should store their local data and logs. Again, ensure that these directories are on the healthy disks or storage volumes. Regards, Chethan YM
... View more
02-06-2024
06:13 AM
Hi @Sokka , I think its possible, can you try the below? """<workflow-app name="Workflow" xmlns="uri:oozie:workflow:0.5"> <start to="hive-04f4"/> <kill name="Kill"> <message>Error al realizar la acción. Mensaje de error [${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <!-- First Hive action to read data --> <action name="hive-04f4" cred="hive2"> <hive2 xmlns="uri:oozie:hive2-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <jdbc-url>jdbc:hive2://host:10000/default</jdbc-url> <script>${wf:appPath()}/hive-04f4.sql</script> <!-- Set output property to be used in next action --> <capture-output/> </hive2> <ok to="loop-decision"/> <error to="Kill"/> </action> <!-- Decision node to determine whether to execute next action --> <decision name="loop-decision"> <switch> <!-- If output is not null, execute next action --> <case to="hive-1c24">${wf:actionData('hive-04f4')['output'] != null}</case> </switch> <!-- If output is null, end the workflow --> <default to="End"/> </decision> <!-- Second Hive action to write data --> <action name="hive-1c24" cred="hive2"> <hive2 xmlns="uri:oozie:hive2-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <jdbc-url>jdbc:hive2://host:10000/default</jdbc-url> <script>${wf:appPath()}/hive-1c24.sql</script> <!-- Use output from previous action as input parameter --> <param>input=${wf:actionData('hive-04f4')['output']}</param> </hive2> <ok to="join"/> <error to="Kill"/> </action> <!-- Join node to synchronize paths after the second action --> <join name="join" to="loop-decision"/> <end name="End"/> </workflow-app>""" The <decision> node (loop-decision) contains a <switch> element with a single <case> element to check if the output of the first Hive action (hive-04f4) is not null. If it's not null, it proceeds to execute the second Hive action (hive-1c24). If it is null, it goes to the <default> path, which ends the workflow. Regards, Chethan YM
... View more
11-23-2023
02:52 AM
Hello @MinhTruong > Did you expand the timeframe at right side of the screen? > From where you ran the query? Impala-shell or Hue? > when its failed what is the error? Regards, Chethan YM
... View more
09-13-2023
07:02 AM
Hello, Do you have any other concerns on the above response? have you tried that for a confirmation? Regards, Chethan YM
... View more
09-13-2023
07:00 AM
Hello @hebamahmoud If the issue is has been from any of the above responses, Could you accept it as a solution? Regards, Chethan YM
... View more
09-05-2023
06:27 AM
Hi, I do not think query id will be present in jdbc driver logs( I have not tested it), It will be present in impalad INFO logs( server side). If you already tested and found query id is not present in driver logs then we cannot make it work other than getting the id from server logs. Get the query id in server logs and search it in driver logs for confirmation. Regards, Chethan YM
... View more
08-30-2023
06:03 AM
Hi, The error message you provided, "Server not found in Kerberos database (7) - LOOKING_UP_SERVER," is indicating an issue with the Kerberos authentication process. This error usually occurs when the Kerberos client is unable to find the server's principal in the Kerberos database. Below is the article to troubleshoot kerberos related issues: https://community.cloudera.com/t5/Customer/Troubleshooting-Kerberos-Related-Issues-Common-Errors-and/ta-p/76192 > Please check if Ensure that DNS is correctly configured for both the client and the server. The client should be able to resolve the hostname of the server to the correct IP address. > Make sure the clocks of the client, server, and KDC are synchronized. Time differences beyond the tolerance set in Kerberos configuration can cause authentication failures. > Ensure that the Key Distribution Center (KDC) is reachable and operational. > Verify that the krb5.conf file on the client machine is correctly configured with the appropriate realms, KDCs, and other Kerberos settings. Regards, Chethan YM
... View more
08-30-2023
05:51 AM
Hello, May i know what is the exact issue? Can you attach the error stack trace OR screenshots? If the connection itself is failing then we do not get the query id. If the connection is succeeded and query is failing then we can see the query profile in the CM -> Impala -> queries section. OR atleast we can see the query id in the impala coordinator logs. Regards, Chethan YM
... View more