question Re: What does --m 1 represent in sqoop import statement? Below is the code. in Archives of Support Questions (Read Only)

What does --m 1 represent in sqoop import statement? Below is the code.

sganeshkumar — Mon, 17 Sep 2018 13:29:25 GMT

$ sqoop import \
--connect jdbc:mysql://localhost/userdb \
--username root \
--table emp --m 1

Re: What does --m 1 represent in sqoop import statement? Below is the code.

ssubhas — Mon, 17 Sep 2018 13:33:52 GMT

@Sudharsan Ganeshkumar

-m represents the number of mappers run to extract the data from the source database. Here '-m 1' means running one mapper.

Re: What does --m 1 represent in sqoop import statement? Below is the code.

sganeshkumar — Mon, 17 Sep 2018 13:36:24 GMT

@Sindhu If I increase the mappers, will the time consumed be less or How many mappers can we keep in general ??? Thanks!

Re: What does --m 1 represent in sqoop import statement? Below is the code.

subhash_parise3 — Mon, 17 Sep 2018 13:42:20 GMT

Hi @Sudharsan Ganeshkumar,

-m denotes number of mappers to launch to run your query.

Re: What does --m 1 represent in sqoop import statement? Below is the code.

ssubhas — Mon, 17 Sep 2018 17:37:50 GMT

@Sudharsan Ganeshkumar
Yes, you can increase the number of mappers to improve parallelism depending on your cluster resources.

Re: What does --m 1 represent in sqoop import statement? Below is the code.

ssubhas — Mon, 17 Sep 2018 17:37:51 GMT

@Sudharsan Ganeshkumar
Yes, you can increase the number of mappers to improve parallelism depending on your cluster resources.

Re: What does --m 1 represent in sqoop import statement? Below is the code.

gul_shad — Mon, 17 Sep 2018 17:41:41 GMT

@Sudharsan Ganeshkumar

The optimal number of mappers depends on many variables: you need to take into account your database type, the hardware that is used for your database server, and the impact to other requests that your database needs to serve. There is no optimal number of mappers that works for all scenarios. Instead, you’re encouraged to experiment to find the optimal degree of parallelism for your environment and use case. It’s a good idea to start with a small number of mappers, slowly ramping up, rather than to start with a large number of mappers, working your way down.

When you run sqoop import with -m 1 option, 1 mapper will be launched and in case this parameter is specified, sqoop will run 4 mappers by default.