Member since
04-20-2016
34
Posts
11
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9698 | 06-02-2016 11:50 AM | |
2145 | 06-02-2016 10:17 AM |
06-20-2017
04:45 PM
I am getting the error regarding specifying connection manager even though I have added the jar file at /usr/hdp/current/sqoop_client/lib Can anyone help? I have used the same command as given above. Error- WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
17/06/20 11:28:48 INFO manager.SqlManager: Using default fetchSize of 1000
17/06/20 11:28:48 INFO tool.CodeGenTool: Beginning code generation
17/06/20 11:28:48 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver
... View more
11-29-2016
10:43 PM
1 Kudo
Hi Omkar, I don't believe it will work with HBase today directly. I have heard that this may come in the future (possibly via Apache Phoenix), but no ETA.
... View more
06-02-2016
10:17 AM
3 Kudos
The error that i am getting is related to the metrics core jar file missing in the configuration? so i have added the metrics-core-2.2.0.jar which is in hbase lib directory to <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> in workflow.xml so this way i resolved the error
... View more
06-02-2016
11:50 AM
1 Kudo
i resolved this by adding hbase/lib metrics-core-2.2.0.jar to /user/root/hbase/ and definer in <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file>
... View more
05-30-2016
06:04 AM
Thanks @Predrag Minovic
... View more
06-02-2016
12:13 PM
1 Kudo
Yes, the order of elements in the xml workflow file is important, see below, "sequence" means that order must be exactly as given. Elements marked with minOccurs="0" are optional and can be omitted. <xs:complexType name="ACTION">
<xs:sequence>
<xs:element name="job-tracker" type="xs:string" minOccurs="1" maxOccurs="1"/>
<xs:element name="name-node" type="xs:string" minOccurs="1" maxOccurs="1"/>
<xs:element name="prepare" type="sqoop:PREPARE" minOccurs="0" maxOccurs="1"/>
<xs:element name="job-xml" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
<xs:element name="configuration" type="sqoop:CONFIGURATION" minOccurs="0" maxOccurs="1"/>
<xs:choice>
<xs:element name="command" type="xs:string" minOccurs="1" maxOccurs="1"/>
<xs:element name="arg" type="xs:string" minOccurs="1" maxOccurs="unbounded"/>
</xs:choice>
<xs:element name="file" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
<xs:element name="archive" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
... View more
05-24-2016
06:32 AM
Thank you Very Much @Josh Elser
... View more
05-23-2016
10:16 PM
1 Kudo
There is no way to tell Sqoop to import all tables into HBase, becuase you have to use "--hbase-table" which is incompatible with "--import-all-tables". Note that HBase is not a general purpose data-base/storage, it's used to store a relatively small number of tables and provide real-time access to them, so it doesn't make sense to import 100s of tables into HBase. For a reasonably small number of tables you can create a script: for t in t1 t2 t3; do
sqoop --connect jdc:mysql://... --table $t --hbase-table $t --hbase-create-table ...
done Note that it's a good idea to pre-create HBase tables, for example to set splitting and compression etc, because Sqoop will not do that. Another approach for your project can be to import all your tables into Hive, create a few Hive tables mapped onto HBase, and populate them using your Hive imported tables.
... View more
05-20-2016
12:35 PM
3 Kudos
Yes, sqoop considers split-by column(--split-by <column-name>) as the rowkey if --hbase-row-key is not specified.
... View more
05-13-2016
06:41 AM
Hive is considered as the "Hadoop Data Warehouse", so indeed you can create fact and dimensional tables. Here is a doc giving an introduction on that: http://www.ibm.com/developerworks/library/bd-hivewarehouse/ If you are new to Hive, I recommend you also to start your journey by downloading the Hortonworks Sandbox and looking at the tutorials we have: http://hortonworks.com/hadoop-tutorial/loading-data-into-the-hortonworks-sandbox/ http://hortonworks.com/apache/hive/#tutorials
... View more