question Re: I want to import certain tables from multiple SQL sever databases using sqoop to HDFS. Can someone guide me how to do it? an automated script would do well. in Archives of Support Questions (Read Only)

I want to import certain tables from multiple SQL sever databases using sqoop to HDFS. Can someone guide me how to do it? an automated script would do well.

omkarmotoe — Mon, 25 Apr 2016 12:09:11 GMT

Re: I want to import certain tables from multiple SQL sever databases using sqoop to HDFS. Can someone guide me how to do it? an automated script would do well.

KuldeepK — Mon, 25 Apr 2016 12:20:40 GMT

Please refer - https://community.hortonworks.com/articles/14802/sqoop-importexport-tutorial.html

Re: I want to import certain tables from multiple SQL sever databases using sqoop to HDFS. Can someone guide me how to do it? an automated script would do well.

pminovic — Mon, 25 Apr 2016 16:53:18 GMT

Hi @omkar pathallapalli, a single Sqoop command can import only a single table from a given DB server. So, to import multiple tables from multiple servers you need a command, for example a Bash script like this

for tbl in $(cat $4); do
sqoop import –-connect "jdbc:sqlserver://${1}:3464;databaseName=${2}" --username ${3} -P --table ${tbl} --target-dir sqimport
done

And call it once per DB server providing each DB server FQDN, database name, user-name and a file listing the tables you want to import (one table per line). The script will prompt you for the password. You can modify target-dir and/or add more Sqoop properties including the number of mappers used for import, by default 4.

Re: I want to import certain tables from multiple SQL sever databases using sqoop to HDFS. Can someone guide me how to do it? an automated script would do well.

emilysharpe — Fri, 29 Apr 2016 06:51:00 GMT

Nice solution @Predrag Minovic. Simple and neat, thanks! +1