Member since
04-12-2016
22
Posts
0
Kudos Received
0
Solutions
07-24-2016
07:50 AM
Hi, I'm applying pool resource management using cloudera, my issue is how to intefrate impala scheduled queries to run with the defined pools. I'm defining pool, i don't know how to submit impala query to run within this pool.
... View more
Labels:
- Labels:
-
Apache Impala
07-09-2016
08:57 AM
Thanks Harsh. Indeed i used -update before but it didn't solve the issue, and since i have 120 cluster nodes, restarting the datanodes wasn't a feasible solution. What solved the issue was using webhdfs instead of hftp.
... View more
07-05-2016
09:03 AM
Hi, Can anyone helpout. This step block my migration from CDH4.
... View more
06-29-2016
08:19 PM
Hi all, I'm planning to migare from CDH4 to CDH5 and i'm using DistCp to copy the historical data between the 2 cluster, my problem that each file in CDH4 HDFS exceeds 150 GB and the nodes with 1G network card, the DistCp failed with such error: Caused by: org.apache.hadoop.tools.mapred.RetriableFileCopyCommand$CopyReadException: java.io.IOException: Got EOF but currentPos = 77828481024 < filelength = 119488762721 at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.readBytes(RetriableFileCopyCommand.java:289) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.copyBytes(RetriableFileCopyCommand.java:257) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.copyToFile(RetriableFileCopyCommand.java:184) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:124) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:100) at org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87) ... 11 more Almost sure the issue is the network Card, but replacing the network card for 120 Nodes isn't easy task. As i know the distcp copy files per mapper, is there away to copy a block per mapper? is there a way to split the files and re merge then after the copy ( want to preserver the file name after the merge also). such issue is limiting my migration to CDH5, hope you can help.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
HDFS