Created 11-10-2023 10:26 PM
I am running Hadoop version 3.2.4 in windows and want to perform a WordCount operation on the file located in hadoop/share/hadoop/mapreduce/share/mapreduce-examples-3.2.4.jar. However, it failed, and there is an error message as follows:
Application application_1699601027477_0002 failed 2 times due to AM Container for appattempt_1699601027477_0002_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2023-11-10 14:44:16.479]Exception from container-launch.
Container id: container_1699601027477_0002_02_000001
Exit code: 1
Exception message: '/tmp/hadoop-ryan' is not recognized as an internal or external command,
operable program or batch file.
[2023-11-10 14:44:16.479]Container exited with a non-zero exit code 1.
[2023-11-10 14:44:16.479]Container exited with a non-zero exit code 1.
For more detailed output, check the application tracking page: http://LAPTOP-IPDKGW12:8088/cluster/app/application_1699601027477_0002 Then click on links to logs of each attempt.
. Failing the application.
this is my settings :
core-site.xml:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>C:\hadoop\data\namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>C:\hadoop\data\datanode</value>
</property>
</configuration>
mapred-site.xml:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xml:
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
I appreciate it if anyone can provide input or advice regarding any mistakes I may have made.
Created 11-12-2023 11:11 AM
Hello Ryan,
Troubleshoot with below following steps and I hope you will find the RCA
1. Check Application Logs:
Navigate to the application tracking page: http://LAPTOP-IPDKGW12:8088/cluster/app/application_1699601027477_0002. Examine the complete logs for the application, especially the container logs. Look for any additional error messages, stack traces, or warnings that might provide more insight into the root cause of the problem.
2. **Inspect Container Launch Command:**
Examine the configuration or script that is being used to launch the container. Ensure that the command or script specified is correct and that it points to a valid file or executable. Verify that any environment variables or paths used in the command are properly set.
3. **File/Command Existence and Permissions:**
Check if the file or command at '/tmp/hadoop-ryan' actually exists. Verify the permissions of the file and the directories leading up to it. The Hadoop user should have the necessary permissions to access and execute the required files.
4. **Environment Variables and Configuration:**
Review the environment variables and Hadoop configuration settings. Ensure that the necessary variables, such as HADOOP_HOME or HADOOP_CONF_DIR, are correctly set. Incorrect configurations can lead to the failure of the container.
5. **Debugging Statements:**
If the issue persists, consider adding additional debugging statements within your application code or script. These statements can help you trace the execution flow and identify the point of failure.
6. **Resource Availability:**
Verify that the cluster has sufficient resources (memory, CPU) to execute the application. Insufficient resources can lead to failures during container launch.
7. **Container Isolation:**
Check if the issue is container-specific or if it affects multiple containers. If it's isolated to a particular container, it could be an issue with the data or configuration specific to that container.
Created 11-13-2023 05:41 AM
Welcome to the community @Ryand2. I hope @Ayuuu has helped you identify the issue here. If so, please use the accept as solution button to mark his reply as the answer.