I want to move data from Oracle to hive or HDFS. I have done import using sqoop command. But i want to do it through java code. Please let me know how to do this. I have tried few codes in internet but nothing worked. Please let me know the flow to do this.
If you've got some example code you've tried to run and are getting some sort of error, can we see that so we can help troubleshoot?
I have successfully moved data from oracle to HDFS using the below code (stackoverflow code http://stackoverflow.com/questions/9229611/how-to-use-sqoop-in-java-program). Now the data is stored as part-m-00000 file in HDFS
1.I will download that file
2. create a table in hive manually.
3. Load this downloaded file into the hive table.
But i want to directly move data from java code using sqoop to the hive. I know there is --hive-import in sqoop command line, but i am not gettig how to use this --hive-import in my jode code. Please let me know. Here is the code
// HBase options options.setHBaseTable("HBASE_TABLE_NAME"); options.setHBaseColFamily("colFamily"); options.setCreateHBaseTable(true);// Create HBase table, if it does not exist options.setHBaseRowKeyColumn("log_id");
Sqoop 1.x do not have official Java API and direct use of SqoopOptions and ImportTool classes is not recommended. The problem with such use is that you need to ensure that all configuration, dependencies and environment is set up correctly prior calling the Sqoop classes. I would recommend to stick with the "sqoop" binary that is shipped with CDH as this binary will set up all required.
Hello, does Oozie action "sqoop" is officially supported? And if it is, then how to run sqoop action with desired user? Because even if I run whole oozie workflow as specific user and provide --hive-import, table in hive is generated as hive user although files that were imported from PSQL are owned by correct user. That gives me an error, that files could not be moved from tmp location to hive warehouse, because owners in both locations does not match.
P.S. should I create new thread for this?