Community Articles

Find and share helpful community-sourced technical articles.
Labels (1)
avatar
Rising Star

While trying to perform an import/export data from/to MS Parallel Data Warehouse following error is observed

Sqoop Command Output:

sqoop export --connect "jdbc:sqlserver://<DB_URL>:<PORT>;database=<DB_NAME>; --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --username <USERNAME> --password ***--table "<TB_NAME>" --input-fields-terminated-by ',' --export-dir <EXPORT_DIR> -m 1 
/usr/hdp/2.5.3.0-37//sqoop/conf/sqoop-env.sh: line 23: HADOOP_CLASSPATH=${hcat -classpath}: bad substitution
Warning: /usr/hdp/2.5.3.0-37/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/hive/lib/phoenix-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/09/12 13:14:07 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.3.0-37
17/09/12 13:14:07 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/09/12 13:14:07 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
17/09/12 13:14:07 INFO manager.SqlManager: Using default fetchSize of 1000
17/09/12 13:14:07 INFO tool.CodeGenTool: Beginning code generation
17/09/12 13:14:08 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.

ROOT CAUSE:

This is because databases like PDW have don't accept READ COMMITTED isolation level. Relaxed isolation fails in this case.
The isolation for metadata queries doesn't set the transaction isolation level or make it mutable.
This issue has been fixed in following apache jira

https://issues.apache.org/jira/browse/SQOOP-2349

RESOLUTION:

This issue can be fixed by upgrading to HDP 2.5.5 version and above.

2,351 Views
Comments
avatar
Explorer

@akapratwar @VidyaSargur we are using HDP 2.6.3 but still seeing the same issue. Do we have a workaround for this.

 

Sqoop command arguments :
import
             -Dmapreduce.job.user.classpath.first=true
             --connect
          jdbc:sqlserver://<host>:<port>;databaseName=<db_name>
             --username
             <username>
             --password
             <password>
             --driver
             com.microsoft.sqlserver.jdbc.SQLServerDriver
             --table
             <tablename>
             --num-mappers
             10
             --compress
             --compression-codec
             org.apache.hadoop.io.compress.SnappyCodec
             --delete-target-dir
             --target-dir
             <hdfs_path>
             --as-avrodatafile

 

 

avatar
Community Manager

Hi @AdityaShaw , as this is an older article, you would have a better chance of receiving a resolution for your issue by starting a new thread and providing a link to this article in the new post. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question.