Skip to main content
Amandi
Amandi
Silver ⭐️⭐️⭐️
September 2, 2024
Question

Issues with ImportTsv - Job Fails with Exit Code 1

I am currently facing an issue while using the ImportTsv command to load data into an HBase table. I am running the command and the job is failing with a non-zero exit code 1. 

./hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns="HBASE_ROW_KEY,cf:name,cf:age,cf:gender" -Dimporttsv.separator="\t" my_1 /hbase/test2.txt

ERROR LOG:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/super/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/super/hbase/lib/client-facing-thirdparty/slf4j-reload4j-1.7.33.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
2024-09-02 09:48:21,871 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-09-02 09:48:22,149 INFO [ReadOnlyZKClient-localhost:2181@0x68267da0] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.8.4-9316c2a7a97e1666d8f4593f34dd6fc36ecc436c, built on 2024-02-12 22:16 UTC
2024-09-02 09:48:22,149 INFO [ReadOnlyZKClient-localhost:2181@0x68267da0] zookeeper.ZooKeeper: Client environment:host.name=dc1-apache-hbase.____________
...
2024-09-02 09:48:32,995 INFO [main] mapreduce.Job: 8]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Hadoop version : 3.4.0

Hbase version : 2.4.18

Resource Manager Logs:

[2024-09-02 09:48:32.499]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


For more detailed output, check the application tracking page: http://______________________:8088/cluster/app/application_1725184331906_0005 Then click on links to logs of each attempt.
. Failing the application.
2024-09-02 09:48:32,505 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1725184331906_0005 State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
2024-09-02 09:48:32,506 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractParentQueue: Application removed - appId: application_1725184331906_0005 user: super leaf-queue of parent: root #applications: 0
2024-09-02 09:48:32,506 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1725184331906_0005,name=importtsv_my_1,user=super,queue=root.default,state=FAILED,trackingUrl=http://_________________________:8088/cluster/app/application_1725184331906_0005,appMasterHost=dc1-apache-hbase._____________,submitTime=1725250705839,startTime=1725250705856,launchTime=1725250706283,finishTime=1725250712503,finalStatus=FAILED,memorySeconds=10734,vcoreSeconds=5,preemptedMemorySeconds=0,preemptedVcoreSeconds=0,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0\, vCores:0>,applicationType=MAPREDUCE,resourceSeconds=10734 MB-seconds\, 5 vcore-seconds,preemptedResourceSeconds=0 MB-seconds\, 0 vcore-seconds,applicationTags=,applicationNodeLabel=,diagnostics=Application application_1725184331906_0005 failed 2 times due to AM Container for appattempt_1725184331906_0005_000002 exited with exitCode: 1\nFailing this attempt.Diagnostics: [2024-09-02 09:48:32.496]Exception from container-launch.\nContainer id: container_1725184331906_0005_02_000001\nExit code: 1\n\n[2024-09-02 09:48:32.498]Container exited with a non-zero exit code 1. Error file: prelaunch.err.\nLast 4096 bytes of prelaunch.err :\nLast 4096 bytes of stderr :\nlog4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).\nlog4j:WARN Please initialize the log4j system properly.\nlog4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.\n\n\n[2024-09-02 09:48:32.499]Container exited with a non-zero exit code 1. Error file: prelaunch.err.\nLast 4096 bytes of prelaunch.err :\nLast 4096 bytes of stderr :\nlog4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).\nlog4j:WARN Please initialize the log4j system properly.\nlog4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.\n\n\nFor more detailed output\, check the application tracking page: http://_________________________________________________________--/cluster/app/application_1725184331906_0005 Then click on links to logs of each attempt.\n. Failing the application.,totalAllocatedContainers=2

 

    7 replies

    rki_
    Gold ⭐️
    rki_
    Gold ⭐️
    September 2, 2024

    @Amandi Can you try to run the command from a Region server host. The connection is going to a localhost on port 2181. So if Zookeeper is not running on the host from where you are running the command, it will fail.

    Translate
      Amandi
      AmandiAuthor
      Silver ⭐️⭐️⭐️
      September 2, 2024

      @rki_Thank you for the response. Could you please suggest a solution that doesn’t involve using the Region Server host?

      Here are the details for port 2181 :
      [@dc1-apache-hbase ~]$ ss -tuln | grep 2181
      tcp LISTEN 0 50 *:2181 *:*
      [@dc1-apache-hbase ~]$ lsof -i :2181
      COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
      java 388228 _____ 558u IPv6 10088613 0t0 TCP *:eforward (LISTEN)
      java 388228 _____ 562u IPv6 10092015 0t0 TCP localhost:eforward->localhost:56476 (ESTABLISHED)
      java 388228 _____ 565u IPv6 10092023 0t0 TCP localhost:eforward->localhost:56492 (ESTABLISHED)
      java 388372 _____ 575u IPv6 10092011 0t0 TCP localhost:56476->localhost:eforward (ESTABLISHED)
      java 388566 _____ 575u IPv6 10088671 0t0 TCP localhost:56492->localhost:eforward (ESTABLISHED)

      Translate
        rki_
        Gold ⭐️
        rki_
        Gold ⭐️
        September 2, 2024

        can you confirm if Zookeeper service is running on this host? Apart from the log4J errors, Do you see any others in the Application log?

        Translate
          Amandi
          AmandiAuthor
          Silver ⭐️⭐️⭐️
          September 2, 2024

          The RegionServer host is localhost, and when I run the command jps -l | grep QuorumPeerMain, there is no output. 

          Using the ssh command to connect to the RegionServer host and running the importtsv command still results in the same error as before.

          jps:

          359952 SecondaryNameNode
          360176 ResourceManager
          361026 DataNode
          400017 HQuorumPeer
          400353 HRegionServer
          400159 HMaster
          406172 Jps
          401800 ThriftServer
          359578 NameNode
          360298 NodeManager

          Translate
            rki_
            Gold ⭐️
            rki_
            Gold ⭐️
            September 2, 2024

            If you are still not able to run the command successfully from the Region server host, then its most likely an issue with YARN. Can you check if there are any other WARNs or ERRORs in the app log apart from log4j warnings.

            Translate
              Terms & ConditionsAccessibility statement