Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error while reading using LLAPDump in ChunkedInputStream

Error while reading using LLAPDump in ChunkedInputStream

New Contributor

@Prasanth Jayachandran

To get better understanding of InputFormat API of LLAP, I was trying to run LLAPDump program, but when I fire the query like "Select * from employee", it is coming up with error with following stack trace :

19/02/21 22:36:02 INFO ipc.Server: Auth successful for application_4031782776548657793_0009 (auth:SIMPLE)
19/02/21 22:36:02 INFO authorize.ServiceAuthorizationManager: Authorization successful for application_4031782776548657793_0009 (auth:TOKEN) for protocol=interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol
Exception in thread "main" 19/02/21 22:36:02 INFO llap.AsyncPbRpcProxy: CallScheduler loop exiting
java.io.IOException: LlapTaskUmbilicalExternalClient(attempt_4031782776548657793_0009_0_00_000000_
0): Error while attempting to read 8192 bytes from current chunk
        at org.apache.hadoop.hive.llap.io.ChunkedInputStream.read(ChunkedInputStream.java:90)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at org.apache.hadoop.hive.llap.LlapBaseRecordReader.hasInput(LlapBaseRecordReader.java:266)
        at org.apache.hadoop.hive.llap.LlapBaseRecordReader.next(LlapBaseRecordReader.java:132)
        at org.apache.hadoop.hive.llap.LlapBaseRecordReader.next(LlapBaseRecordReader.java:40)
        at org.apache.hadoop.hive.llap.LlapRowRecordReader.next(LlapRowRecordReader.java:120)
        at org.apache.hadoop.hive.llap.LlapRowRecordReader.next(LlapRowRecordReader.java:67)
        at org.apache.hadoop.hive.llap.LlapDump.main(LlapDump.java:129)
Caused by: java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:197)
        at org.apache.hadoop.hive.llap.io.ChunkedInputStream.read(ChunkedInputStream.java:88)
        ... 9 more
19/02/21 22:36:03 INFO counters.Limits: Counter limits initialized with parameters:  GROUP_NAME_MA
X=256, MAX_GROUPS=3000, COUNTER_NAME_MAX=64, MAX_COUNTERS=10000
19/02/21 22:36:03 WARN ext.LlapTaskUmbilicalExternalClient: No tasks found for heartbeat from task
AttemptId attempt_4031782776548657793_0009_0_00_000000_0
19/02/21 22:36:03 INFO ext.LlapTaskUmbilicalExternalClient: Unexpected heartbeat from attempt_4031
782776548657793_0009_0_00_000000_0

I have tried with various sizes of data, so instead of 8192, it gives that much data and reports the same error with stack trace.

Not getting any clue what to try now ?

Anyone has seen this issue and now what is the issue ?

It seems to be getting the schema correctly as it prints it in the log.

My command to invoke LLAPDump is :

./script.sh --hiveconf hive.zookeeper.quorum=ivlhdp1344.example.com:2181 --hiveconf hive.llap.daemon.service.hosts=@llap0 --hiveconf hive.llap.daemon.service.principal=hive/_HOST@HOSTKRB.EXMADEV.COM --location "jdbc:hive2://ivlhdp1344.example.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-interactive"


But If I use LlapArrowRowInputFormat instead of LLAPRowInputFormat, then LlapDump works fine (obviously it does not use ChunkedInputStream as it is not for arrow).

Could there be a bug when we do not use Arrow format ? Please confirm.


Don't have an account?
Coming from Hortonworks? Activate your account here