Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

problems using postgresql and sqoop

Highlighted

problems using postgresql and sqoop

Explorer

Hello,

 

I would like to work with my postgres database and sqoop. I would like to get a list of all tables of a database and i would like to import a table from postgresql to sqoop.

 

When i installed everything, i followed the instruction https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_ig_extrnl_pstgrs.html

 

I used following commando to get the informaiton of the tables:

sqoop list-tables --driver=org.postgresql.Driver --connect jdbc:postgresql://0.0.0.0:7433/amon --username xxx --password xxx

 

But I get this Error Msg:

Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/08/29 09:58:50 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.0
18/08/29 09:58:50 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/08/29 09:58:50 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
18/08/29 09:58:50 INFO manager.SqlManager: Using default fetchSize of 1000
18/08/29 09:58:51 ERROR manager.SqlManager: Error reading database metadata: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "10.0.1.233", user "xxx", database "amon", SSL off
org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "10.0.1.233", user "xxx", database "amon", SSL off
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:398)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
        at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
        at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
        at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
        at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
        at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
        at org.postgresql.Driver.makeConnection(Driver.java:393)
        at org.postgresql.Driver.connect(Driver.java:267)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:215)
        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)
        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
        at org.apache.sqoop.manager.SqlManager.listTables(SqlManager.java:539)
        at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Could not retrieve tables list from server
18/08/29 09:58:51 ERROR tool.ListTablesTool: manager.listTables() returned null

 

I also tried to include '?sslmode=require' but it doesnt work.

 

Does someone knows, how i can fix this problem?

 

best regards

1 REPLY 1

Re: problems using postgresql and sqoop

Master Guru
Postgres is sensitive to how you connect to it. You should be using the exact address it listens on, as only that will be allowed by the default configuration.

Your command carries an IP that's 0.0.0.0. While am uncertain if you've masked it or if you are truly using a wildcard-designate IP for a server address, you should ideally be using the exact hostname/IP that the Postgres service is listening on.

Your Postgres service's configuration will carry the listening_address entry in it that designates this. Take a look at this thread over at Stack Exchange: https://dba.stackexchange.com/a/84002
Don't have an account?
Coming from Hortonworks? Activate your account here