Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

impala sql failed with errno=2 description=Error(2): No such file or directory

avatar
Contributor

hi:

cdh5.2.0 ,impala 2.0.

Every hour I will pull data from hive to impala(parquet file).Sometimes,can not excute sql like below,and restart the impala cluster could solve the problem,why?

 

select id,sub1,(ifnull(sum(click),0)) as clicks > from new_td_impala_test
where (part1>='2014-11-06' and part1<='2014-11-06') and (unix_timestamp(substr(time_stamp,1,19)) >= unix_timestamp('2014-10-31 16:00:00') and unix_timestamp(substr(time_stamp,1,19)) < unix_timestamp('2014-11-07 16:00:00')) group by id,sub1 limit 4 ;
Query: select id,sub1,(ifnull(sum(click),0)) as clicks
from new_td_impala_test
where (part1>='2014-11-06' and part1<='2014-11-06') and (unix_timestamp(substr(time_stamp,1,19)) >= unix_timestamp('2014-10-31 16:00:00') and unix_timestamp(substr(time_stamp,1,19)) < unix_timestamp('2014-11-07 16:00:00')) group by id,sub1 limit 4
WARNINGS: Create file /tmp/impala-scratch/7241111ac15574e0:abef55f7a589b3a5_7241111ac15574e0:abef55f7a589b3a7_dc8fc8e3-f98c-41e8-a86a-d9c8f7840cdb failed with errno=2 description=Error(2): No such file or directory

Backend 1:Create file /tmp/impala-scratch/7241111ac15574e0:abef55f7a589b3a5_7241111ac15574e0:abef55f7a589b3a7_dc8fc8e3-f98c-41e8-a86a-d9c8f7840cdb failed with errno=2 description=Error(2): No such file or directory

 

1 ACCEPTED SOLUTION

avatar
Another alternative is to change your scratch_dirs to be outside of /tmp. That way it might be safe from any job that cleans out /tmp
Regards,
Gautam Gopalakrishnan

View solution in original post

4 REPLIES 4

avatar

When one or more parent directories in the path are missing, file creation will fail with err=2

 

# ls -ld /tmp/impala-scratch
ls: cannot access /tmp/impala-scratch: No such file or directory
# touch /tmp/impala-scratch/test
touch: cannot touch `/tmp/impala-scratch/test': No such file or directory

 

Next time this error occurs, check for the presence of /tmp/impala-scratch before restarting the service. Also, check the HDFS Audit logs to identify which user might be deleting the directory.

Regards,
Gautam Gopalakrishnan

avatar
Another alternative is to change your scratch_dirs to be outside of /tmp. That way it might be safe from any job that cleans out /tmp
Regards,
Gautam Gopalakrishnan

avatar

Hi There,

      Do you know, what are the steps we have to follow to make this change?

      I kept on reading the page but didn't find that helpfull.

 

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_disk_space.html

avatar

I have options to add another directory but I don't know if that will skip default setting or still I will need to change default.