Reply
Highlighted
Contributor
Posts: 78
Registered: ‎06-19-2014
Accepted Solution

impala sql failed with errno=2 description=Error(2): No such file or directory

hi:

cdh5.2.0 ,impala 2.0.

Every hour I will pull data from hive to impala(parquet file).Sometimes,can not excute sql like below,and restart the impala cluster could solve the problem,why?

 

select id,sub1,(ifnull(sum(click),0)) as clicks > from new_td_impala_test
where (part1>='2014-11-06' and part1<='2014-11-06') and (unix_timestamp(substr(time_stamp,1,19)) >= unix_timestamp('2014-10-31 16:00:00') and unix_timestamp(substr(time_stamp,1,19)) < unix_timestamp('2014-11-07 16:00:00')) group by id,sub1 limit 4 ;
Query: select id,sub1,(ifnull(sum(click),0)) as clicks
from new_td_impala_test
where (part1>='2014-11-06' and part1<='2014-11-06') and (unix_timestamp(substr(time_stamp,1,19)) >= unix_timestamp('2014-10-31 16:00:00') and unix_timestamp(substr(time_stamp,1,19)) < unix_timestamp('2014-11-07 16:00:00')) group by id,sub1 limit 4
WARNINGS: Create file /tmp/impala-scratch/7241111ac15574e0:abef55f7a589b3a5_7241111ac15574e0:abef55f7a589b3a7_dc8fc8e3-f98c-41e8-a86a-d9c8f7840cdb failed with errno=2 description=Error(2): No such file or directory

Backend 1:Create file /tmp/impala-scratch/7241111ac15574e0:abef55f7a589b3a5_7241111ac15574e0:abef55f7a589b3a7_dc8fc8e3-f98c-41e8-a86a-d9c8f7840cdb failed with errno=2 description=Error(2): No such file or directory

 

Cloudera Employee
Posts: 578
Registered: ‎01-20-2014

Re: impala sql failed with errno=2 description=Error(2): No such file or directory

When one or more parent directories in the path are missing, file creation will fail with err=2

 

# ls -ld /tmp/impala-scratch
ls: cannot access /tmp/impala-scratch: No such file or directory
# touch /tmp/impala-scratch/test
touch: cannot touch `/tmp/impala-scratch/test': No such file or directory

 

Next time this error occurs, check for the presence of /tmp/impala-scratch before restarting the service. Also, check the HDFS Audit logs to identify which user might be deleting the directory.

Regards,
Gautam Gopalakrishnan
Cloudera Employee
Posts: 578
Registered: ‎01-20-2014

Re: impala sql failed with errno=2 description=Error(2): No such file or directory

Another alternative is to change your scratch_dirs to be outside of /tmp. That way it might be safe from any job that cleans out /tmp
Regards,
Gautam Gopalakrishnan
Explorer
Posts: 7
Registered: ‎06-09-2015

Re: impala sql failed with errno=2 description=Error(2): No such file or directory

Hi There,

      Do you know, what are the steps we have to follow to make this change?

      I kept on reading the page but didn't find that helpfull.

 

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_disk_space.html

Explorer
Posts: 7
Registered: ‎06-09-2015

Re: impala sql failed with errno=2 description=Error(2): No such file or directory

I have options to add another directory but I don't know if that will skip default setting or still I will need to change default.

 

Announcements