Hey abe ,
I am running the sqoop command on node 1 and the cluster comprises of 20 nodes..... 1nn , 19 dn...
I can access hdfs from all the nodes, also able to access directories that was created using hadoop fs -mkdir /user/xyz/abc......., The only issue is unable to access directory created by sqoop import, though the sqoop job was successful
following is the sqoop command:
sqoop import --connect jdbc:mysql://ip-address/db --username abc --password cde --query "select * from hfpoverty WHERE hindus_females > 10 AND \$CONDITIONS" --target-dir /user/jais/test3 -m 1
So, this directory has been created test3........but unable to access it as it says no such file or directory........
Thanks for helping me, the issue has been resolved bby itself, do not know what happened, I felt there some kind of special characters auto generated on top of the directory name, hence it was saying no such file or directory, as soon as I deleted the directory and created the new one, it works fine.
I have somewhat similar issue but my error is little different.
Command line -
sudo -u hdfs sqoop import --connect jdbc:oracle:thin:@//BIDEVDC01.gmpvt.net:1521/BIDEV --table CM_GIS_OUTAGE_EXTRACT --fields-terminated-by '\t' --username DWADM --password dwadm --target-dir /user/etl/sqoop/init/CM_GIS_OUTAGE_EXTRACT --verbose --connection-manager org.apache.sqoop.manager.OracleManager --num-mappers 4 --mapreduce-job-name CM_GIS_OUTAGE_EXTRACT --direct --split-by DEVICE_OID --hive-import &> sqoop-CM_GIS_OUTAGE_EXTRACT.log
15/02/18 17:16:14 INFO mapreduce.Job: map 0% reduce 0%
15/02/18 17:16:14 INFO mapreduce.Job: Job job_1423869234667_0007 failed with state FAILED due to: Application application_1423869234667_0007 failed 2 times due to AM Container for appattempt_1423869234667_0007_000002 exited with exitCode: -1000 due to: Application application_1423869234667_0007 initialization failed (exitCode=255) with output: main : command provided 0
main : user is nobody
main : requested yarn user is hdfs
Can't create directory /data1/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data10/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data11/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data12/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data2/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data3/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data4/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data5/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data6/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data7/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data8/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Can't create directory /data9/yarn/nm/usercache/hdfs/appcache/application_1423869234667_0007 - Permission denied
Did not create any app directories
you could try deleting the /data1/yarn/nm/usercache/hdfs directory from your datanodes - as suggested in here:
That worked for me.