Created 09-24-2020 02:49 AM
I am running the command from hdfs user hdfs@master>
sqoop import-all-tables --connect jdbc:mysql://10.11.11.15:6306/siki_asmet?serverTimezone=Asia/Jakarta --username micronics -P --hive-import --warehouse-dir /warehouse/siki --hive-database siki_ods --exclude-tables "Sheet1$" --m 1;
When i run the above command, i get the following error:
FAILED: SemanticException Line 1:17 Invalid path ''hdfs://master.lpjk.com:8020/warehouse/siki/_asdamkindo_personal_ska_pendidikan'': No files matching path hdfs://master.lpjk.com:8020/warehouse/siki/_asdamkindo_personal_ska_pendidikan (state=42000,code=40000)
But when i try to run the command again, it said file already exists.
Created 09-24-2020 08:34 AM
I solved my problem. In my case one of the talbes' name was starting with a character underscore "_" because of which there was an issue where 2 single quotes were added automatically in the path of the hdfs directory where the copy of the file was stored.
I changed the name of the column by removing the underscore character and now i can import the table into Hive database. I think special characters like that are not easily parsed in Hive or HDFS.
Created 09-24-2020 06:16 AM
Check that the user executing sqoop is able to write to the target directory. You may need to create a service user and directory with proper permissions since the hdfs user cannot write to the intended directory.
If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.
Thanks,
Steven
Created on 09-24-2020 08:02 AM - edited 09-24-2020 08:07 AM
Since i am executing sqoop from the user hdfs (hdfs@master>) it should have access to all the directories inside it right?
if i just use the command without specifying the --warehouse-dir, then also i get the same error.
Also, i disabled permission in hdfs-site.xml from true to false in hdfs.permission.enabled
Created 09-24-2020 08:34 AM
I solved my problem. In my case one of the talbes' name was starting with a character underscore "_" because of which there was an issue where 2 single quotes were added automatically in the path of the hdfs directory where the copy of the file was stored.
I changed the name of the column by removing the underscore character and now i can import the table into Hive database. I think special characters like that are not easily parsed in Hive or HDFS.