Member since
01-09-2017
55
Posts
14
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3696 | 09-05-2016 10:38 AM | |
1859 | 07-20-2016 08:22 AM | |
3780 | 07-04-2016 08:13 AM | |
1443 | 06-03-2016 08:01 AM | |
2007 | 05-05-2016 12:37 PM |
07-20-2016
01:44 PM
The easiest way is to use the method saveAsObjectFile and read it through the objectFile method... You can easily find them in Spark documentation for further details about them.
... View more
06-03-2016
08:42 PM
please check here: https://community.hortonworks.com/questions/5148/atlas-generating-huge-log-files.html
... View more
04-21-2016
02:29 PM
3 Kudos
You need use sqoop for direct import on hive form sql. First download sql jdbc
https://www.microsoft.com/en-us/download/details.aspx?id=11774 place jar on sqoop master server:
/usr/hdp/current/sqoop-server/lib use this command for import: import --connect jdbc:sqlserver://[SQL_SERVER_NAME]:[SQL_PORT]/[DB-NAME] --username "[SQL_USERNAME]" --password "[PASSWORD]" --query '[INSERT QUERY HERE] WHERE $CONDITIONS' -m 1 --hive-import --hive-database [DB_HIVE] --create-hive-table --hive-table [HIVE_TABLE]
if use --query you must have WHERE $CONDITIONS on query.
... View more
04-20-2016
09:22 AM
1 Kudo
It turned out that the problem was caused by a join with a subquery which made the data to be unevenly distributed among the partitions. Actually I don't know why this is happening but we solved by materializing the subquery. Thank you or the support.
... View more
04-04-2016
12:17 PM
I found the problem. It was a mistake by me: I added the configuration of the tutorial (https://community.hortonworks.com/articles/4868/hive-oom-caused-by-javalangoutofmemoryerror-java-h.html) via the set command in beeline: instead, they have to be configured in the Tez section in Ambari...
... View more
03-29-2016
07:08 AM
1 Kudo
I can't give you the query since it's rather complex (about 1500 lines). Actually we haven't run ANALYZE for the columns... Asap we'll try and let you know. Thank you for your answer.
... View more