Thanks for the information. In using this command, it did cause some serious performance degradation when writing to HDFS. Every 128MB block would take about 20-30 secs to write to HDFS. The issue had to do with trying to compress the tar file. It's better to remove the "z" flag in tar and not compress.   Just to provide some numbers, writing almost 1TB of data from local disk to HDFS would take 13+ hours with compression (z) and it would actually eventually fail due to kerberos ticket expiration. Removing the "z" flag, the copy to HDFS took less than an hour for the same 1TB of data!  ... View more

Hi Neeraj Sabharwal, @Neeraj Sabharwal @Rushikesh Deshmukh This are the steps i followed for incremental import in sqoop for hbase table. Step 1: Importing a Table To HBase sqoop import --connect "jdbc:sqlserver://x.x.x.x:1433;database=test" --username sa -P --table employee --hbase-table employee --hbase-create-table --column-family cf --hbase-row-key id -m 1 Step 2: SQOOP HBASE INCREMENTAL IMPORT sqoop import --connect "jdbc:sqlserver://x.x.x.x:1433;database=test" --username sa -P --table employee --incremental append --check-column id --last-value 71 -m 1 Step 3: SQOOP JOB CREATION FOR HBASE INCREMENT sqoop job --create incjobsnew -- import --connect "jdbc:sqlserver://x.x.x.x:1433;database=test" --username sa -P --table employee --incremental append --check-column id --last-value 71 -m 1. When i execute sqoop job sqoop job --exec incjobsnew. Sqoop command runs successfully and it show the exact number of records retrieved successfully. When i check in hbase for the records. It doesn't show the retrieved results. Could you tell where is the mistake done. I need to automate this sqoop job in Oozie to run a particular time interval daily. ... View more

@Rushikesh Deshmukh Flatten un-nests tuples as well as bags. consider a relation that has a tuple of the form (a, (b, c)). The expression GENERATE $0, flatten($1), will cause that tuple to become (a, b, c). You can refer to the below link to know more and have better understanding of other operators, just in case if you need them. https://www.qubole.com/resources/cheatsheet/pig-function-cheat-sheet/ ... View more

Does these configuration mentioned in this page work on TEZ engine .I could see SMB working only on MR ... View more

Sure, please accept the answer is satisfied ... View more

@Artem Ervits, thanks for sharing this link. ... View more

@Neeraj Sabharwal, got the required answer, thus closing this thread. ... View more

I was successful in executing a MapReduce Job. Since the method Job.setBy.JarName(WordCount.class) was missing it was unable to find out the Mapper class. Thanks!!! ... View more