Sqoop Performance test case

[#1]

Hi

I have a question

Our environment is Master 1, Walker 4

We are considering using Sqoop when importing data from Oracle DB to Spark Node.

We are going to perform a performance test before using sqoop.

I am interested in test cases and verification methods in this regard.

Regards

MyeongHwan Oh

mhoh@prumdataware.com

[#2]

@MyeongHwan Oh

Do you mean transferring data to Hadoop data node. You can use your own test data in Oracle and import it using sqoop as there are no specific data datasets or benchmarking results for sqoop. Also, There are few common performance improvement techniques for Sqoop e.g. split-by and boundary-query, direct, fetch-size, num-mapper

Please find below links as they are good starting points

https://community.hortonworks.com/articles/70258/sqoop-performance-tuning.html

https://kb.informatica.com/h2l/HowTo%20Library/1/0930-SqoopPerformanceTuningGuidelines-H2L.pdf

http://www.xmsxmx.com/performance-tuning-data-load-into-hadoop-with-sqoop/

https://dzone.com/articles/apache-sqoop-performance-tuning

If this answer your question, please vote/accept best answer.