Issue with Sqoop

[#1]

Hi, I took the HDPCD practice test on AWS but I am facing few problems with sqoop.

For the Task 10, i used “sqoop export –connect jdbc:mysql://namenode:3306/flightinfo –table weather –export-dir /user/horton/weather –input-fields-terminated-by ‘,’ –username root –password hadoop” and got the following:

Warning: /usr/hdp/2.2.0.0-2041/sqoop/sqoop/bin/../../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. 15/10/03 18:46:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-2041 15/10/03 18:46:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 15/10/03 18:46:56 INFO manager.SqlManager: Using default fetchSize of 1000 15/10/03 18:46:56 INFO tool.CodeGenTool: Beginning code generation 15/10/03 18:46:57 ERROR manager.SqlManager: Error executing statement: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user ‘root’@’%’ to database ‘flightinfo’ com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Access denied for user ‘root’@’%’ to database ‘flightinfo’

And also, I am not able to copy the pig scripts into the solutions folder as mentioned.

cp -f flightdelays_clean.pig /home/horton/solutions/ cp: cannot create regular file ‘/home/horton/solutions/flightdelays_clean.pig’: Permission denied

Am I missing something? Please help.

[#2 · Accepted solution]

Hi Saptak, You reported multiple issues 0

1) /usr/hdp/2.2.0.0-2041/sqoop/sqoop/bin/../../hcatalog does not exist!

Ans: Are you working with HDP 2.2. or 2.3? This could be a hardcoded path in one of the scripts that needs to be updated.

2) Access denied for user ‘root’@’%’ to database ‘flightinfo’

Ans: MySQL access is typically limited to localhost by default. In a multi-node cluster, you need to open up that permission for the nodes that will be connecting to the MySQL Server. You can either do this per server basis (for all data nodes) or for a non-prod environment, you can just open up the permission for root@%. See this article for detailed instructions - https://rtcamp.com/tutorials/mysql/remote-access/

3) cp: cannot create regular file ‘/home/horton/solutions/flightdelays_clean.pig’: Permission denied

Ans: Looks like you just need to make sure the user you are logged in with has the permission to write to the directory you are writing to.

[#3]

@Saptak Sen are you still having issues with this? Can you accept the best answer or provide your own solution?