Skip to main content
grlzz
grlzz
Gold ⭐️
January 26, 2022
Solved

sqoop stuck at No connection paramenters specified. Using regular API for making connection.

Hi,

 

 

I am trying to run this import command:

 

sqoop import --connect jdbc:postgresql://[host]:[port]/[db] --username [user] --password-file [path_to_pass] -table [table] --hive-import -hive-drop-import-delims --as-parquetfile --parquet-configurator-implementation hadoop --hive-table district_startdates_test --target-dir [target_with_db] -m 1 --hs2-url="jdbc:hive2://[cdp]/;ssl=true;transportMode=http;httpPath=[path_to_proxy];user=[user];password=[cluster_passw]" --verbose

 

but it gets stuck at  

 DEBUG manager.SqlManager: No connection paramenters specified. Using regular API for making connection.

 

...any idea?

thanks

grlzz
grlzzAnswer
Gold ⭐️
February 3, 2022

Hi @mszurap , 

 

 

thanks for the tool

So, somehow the connection is now working , namely the command we tried to cloudera support, like:

`sqoop eval --connect ...`

or

 ` sqoop list-databases --connect` 

 

before running any sqoop command, we set up a ssh tunnel, with 

ssh -o ServerAliveInterval=60 -M -S mktemp -fN -L local_port:psql_fqdn:remote_p  user@host_IP 

 

(which I was doing also before, so not sure why now it works... 

I will mark it now as solved

 

thanks anyway,

Gr

View in thread

10 replies

mszurap
Platinum ⭐️
mszurap
Platinum ⭐️
January 27, 2022

Hi @grlzz ,

Have you verified the DB connection URL "jdbc:postgresql://[host]:[port]/[db] " - is that working outside of sqoop? (with any external JDBC based tool)

Have you used the same PostgreSQL driver which is supposed to be present under /var/lib/sqoop?

Also if the sqoop command is really "stuck", please check in another terminal window where is it stuck with jstack:

1. Get the process id of the sqoop command

ps -ef | grep sqoop

 

2. Collect the jstack output - with the same user as the sqoop import is running:

/usr/java/latest/bin/jstack $PID

 

This can help to understand what it is trying to do (for example it tries to connect to the database - but maybe the database is ssl enabled?)

Translate
grlzz
grlzzAuthor
Gold ⭐️
January 27, 2022

Hi,

 

thanks for your reply.

I haven't checked other JDBC tools yet, but I attach hereby the jstack output 

(by the way, it actually proceeded to another stage:

 

Heap
 PSYoungGen      total 147456K, used 76137K [0x000000071bb00000, 0x000000072db00000, 0x00000007c0000000)
  eden space 126464K, 60% used [0x000000071bb00000,0x000000072055a788,0x0000000723680000)
  from space 20992K, 0% used [0x0000000724b00000,0x0000000724b00000,0x0000000725f80000)
  to   space 20992K, 0% used [0x0000000723680000,0x0000000723680000,0x0000000724b00000)
 ParOldGen       total 192000K, used 9642K [0x00000005d3000000, 0x00000005deb80000, 0x000000071bb00000)
  object space 192000K, 5% used [0x00000005d3000000,0x00000005d396ab90,0x00000005deb80000)
 Metaspace       used 27157K, capacity 27412K, committed 27776K, reserved 1073152K
  class space    used 2984K, capacity 3100K, committed 3200K, reserved 1048576K

best,

Gr 

Translate
grlzz
grlzzAuthor
Gold ⭐️
January 27, 2022

...where it seems 'stuck' ... (the table is quite small, so I do not expect to take a lot of time -- I stopped after ~ 40 min)

Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
January 31, 2022

@grlzz Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!

Translate
Regards, Diana Torres, Senior Community Moderator
grlzz
grlzzAuthor
Gold ⭐️
February 1, 2022
Hi Diana,

no, this issue is not solved yet

best,
Gr
Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
February 2, 2022

Hi @grlzz 

Sorry that this has not being resolved,

Have you tried the fix above that @mszurap  provided?

Please verify if your connection string is correct with some 3rd party tool - or ask your DBA what is a correct JDBC connection string.

If you are still experiencing the issue, can you provide the information requested or additional details?

 

Thanks

Translate
Regards, Diana Torres, Senior Community Moderator
Terms & ConditionsAccessibility statement