Member since
05-16-2016
270
Posts
18
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1727 | 07-23-2016 11:36 AM | |
3077 | 07-23-2016 11:35 AM | |
1579 | 06-05-2016 10:41 AM | |
1167 | 06-05-2016 10:37 AM |
07-01-2016
07:41 AM
@Sindhu: Got it. But, I do not want the data moved out of warehouse dir /target-dir.Is there a solution for that? or I need to do it separately without the hive import option to keep it in hdfs ?Also, the link suggests using hcatalog: http://grokbase.com/t/sqoop/user/143waxddrr/jira-commented-sqoop-1293-hive-import-causes-target-dir-and-warehouse-dir-to-not-be-respected-nor-delete-target-dir .Is it really a solution to the problem?
... View more
07-01-2016
07:19 AM
I believe it is because of the --hive-import argument? I could remove that but I have to use --hive-overwrite argument and I can't use it unless I use --hive-import. @Sindhu . So, how do I use --hive-overwrite while using warehouse-dir /target-dir?
... View more
07-01-2016
07:16 AM
Why would it ignore the argument? I tried it with target-dir as well and that did not work either @Sindhu
... View more
07-01-2016
07:15 AM
@Sindhu You are right. It shows table location as hdfs://FQDN:8020/user/hive/warehouse/magentodb.db/TABLENAME
... View more
07-01-2016
07:00 AM
so if I replace --warehouse-dir with --target-dir , it would permanently store files in target-dir location and then I can have my tables mapped to this location as external table? @Dileep Kumar Chiguruvada
... View more
07-01-2016
06:04 AM
Whenever I use --hive-import argument, I specify a --warehouse-dir as well in my sqoop jobs. Now, I check my hive tables and data is indeed there but my question is why do I not see any files in warehouse dir using hadoop fs -ls command ? Sure I do see them when I replace -hive-import and --warehouse-dir with --target-dir . How does it work? What are the advantages of one over the other?
... View more
Labels:
- Labels:
-
Apache Sqoop
06-28-2016
06:13 AM
@slachterman Hi, Thank you for your response, How exactly do I use hive for data cleaning purpose?Could you please give an example? I have not really touched spark so far so do you think spark is the way to go only for introducing the data cleaning layer in the system? I have heard all the good stuff about spark but would like to know if this would be a stuitable use case for it?
... View more
06-27-2016
10:58 AM
1 Kudo
I am using sqoop to import data to HDFS and hive and have my job scheduled using Oozie. How do I introduce a data cleaning layer in the system before storing this data in hive or hdfs?What are the tools available/suitable for the purpose in hadoop ecosystem?
... View more
Labels:
- Labels:
-
Apache Hadoop