@Rajesh Are you able to update with the target with snapshot difference? hadoop distcp -diff s1 s2 -update hdfs://secure_cluster:8020/source hdfs://secure_cluster:8020/target though it is syntactically correct, I don't see its working, I see the error as below. 17/04/18 14:29:53 ERROR tools.DistCp: Invalid arguments: java.lang.IllegalArgumentException: Diff is valid only with update and delete options at org.apache.hadoop.tools.DistCpOptions.validate(DistCpOptions.java:568) at org.apache.hadoop.tools.DistCpOptions.setUseDiff(DistCpOptions.java:284) at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:223) at org.apache.hadoop.tools.DistCp.run(DistCp.java:115) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.tools.DistCp.main(DistCp.java:430) Invalid arguments: Diff is valid only with update and delete options usage: distcp OPTIONS [source_path...] <target_path> ... View more

Hi Kit, I have a teradata table having near to 100 columns. As its not feasible for us create the schema manually. We wanted to create schema using create-hive-table flag. But it doesnt seems to be working I followed your syntax: sqoop create-hive-table \ -libjars ${LIB_JARS} \ --connect $JDBCURL \ --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --username $TDUSER \ --password $TDPASS \ --table $TDTABLE \ --map-column-hive EFF_TSP=STRING \ --hive-table ${HIVEDB}.${HIVETABLE} I see the below error "Error parsing arguments for create-hive-table" 17/03/03 10:35:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-258 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-258/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-258/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Error parsing arguments for create-hive-table: 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: -p 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: --table 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: pdcrdata.DBQLogTbl_Hst_1 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: --map-column-hive 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: EFF_TSP=STRING 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: --hive-table 17/03/03 10:35:48 ERROR tool.BaseSqoopTool: Unrecognized argument: pi_talend_test.teradata_import Try --help for usage instructions. Thanks Suresh ... View more

@SBandaru To my knowledge, it's not advisable to change the pre-defined ambari scripts. These changes might affect the the potential upgrade of ambari server in future. A best practice would be developing an individual maintenance script to change the permissions of the hive warehouse. ... View more