Reply
Highlighted
Contributor
Posts: 52
Registered: ‎10-19-2016
Accepted Solution

sqoop confilt with Hive HA metastore

[ Edited ]

Hi, 

I met this issue https://issues.cloudera.org/browse/KITE-762.

Sqoop1 job works if I remove Hive meastore HA and keep only on metastore server.

How could I disable HA only for sqoop1 client?

Thanks.

Posts: 1,657
Kudos: 320
Solutions: 258
Registered: ‎07-31-2013

Re: sqoop confilt with Hive HA metastore

You can try to pass an override specifying a singular URI in such a manner:

sqoop import -Dhive.metastore.uris=thrift://only-one-of-your-HMS-host:9083
…(rest of arguments follow)…

(The override argument via -D must precede all other arguments)
Contributor
Posts: 52
Registered: ‎10-19-2016

Re: sqoop confilt with Hive HA metastore

Thanks. It works.
Will this solved in next version of kite?
HSt
New Contributor
Posts: 2
Registered: ‎12-19-2017

Re: sqoop confilt with Hive HA metastore

Hi,

i´m trying to use sqoop via hue, but i keep getting this error:

2018-01-18 08:31:52,353 [main] WARN  org.kitesdk.data.spi.hive.MetaStoreUtil  - Aborting use of local MetaStore. Allow local MetaStore by setting kite.hive.allow-local-metastore=true in HiveConf
2018-01-18 08:31:52,353 [main] ERROR org.apache.sqoop.tool.ImportTool  - Import failed: Missing Hive MetaStore connection URI

Its not the same but seems to be quite similar.

The cluster is useing HA for Hive metastore.

I tried to set the Hive metastore uri like this:

import -Dhive.metastore.uris=thrift://17.239.167.168:9083 -Dkite.hive.allow-local-metastore=true --connect jdbc:exa:17.239.167.201..205:8563;schema=sks_dp_steuerung --driver com.exasol.jdbc.EXADriver --username sys --password exasol --table dbo_institutspartitionen --hive-import --as-parquetfile --hive-table DBO_INSTITUTSPARTITIONEN --hive-database SKS_DP_STEUERUNG -m 1

but it makes no difference.

The Exception tells to set

kite.hive.allow-local-metastore=true

I tried this via

-Dkite.hive.allow-local-metastore=true

I have no idea whether this would be the right way to do this.

 

Is there something i might have missed? Or is this really a completely different error?

Announcements