Member since
02-04-2016
189
Posts
70
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3793 | 07-12-2018 01:58 PM | |
7936 | 03-08-2018 10:44 AM | |
3813 | 06-24-2017 11:18 AM | |
23377 | 02-10-2017 04:54 PM | |
2316 | 01-19-2017 01:41 PM |
11-18-2016
12:11 PM
Yikes! Have never seen that happen before, but I certainly have no reason to doubt you. Does it happen with hive.execution.engine=tez? If you could grab & sanitize your query/config details & post that as a Hive bug in Jira it would be greatly appreciated... we don't want that problem to bite anyone else.
... View more
11-29-2016
12:30 PM
1 Kudo
you need to set the s3a properties to log in; these are separate from the s3n ones see: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md see also: http://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-trouble/index.html
... View more
11-01-2016
02:36 PM
Perfect. Thanks!
... View more
01-04-2018
11:54 AM
Hey everyone, I have a somewhat similar question, which I posted here: https://community.hortonworks.com/questions/155681/how-to-defragment-hdfs-data.html I would really appreciate any ideas. cc @Lester Martin @Jagatheesh Ramakrishnan @rbiswas
... View more
10-07-2016
08:33 PM
1 Kudo
I think I figured this out. The issue isn't the comma - it was my null character. It would be nice if the error logging here was just a bit clearer.
... View more
09-26-2016
07:21 PM
Thanks. I landed at a similar solution using GetFile.
... View more
09-21-2016
09:07 PM
Yes correct. You could also do both in once with ExtractText processor, but you definitely can use UpdateAttribute and expression language functions to get the result you want. Also if you only need the filename for a processor in a property accepting expression language, you don't need the intermediate step. For example: ${mypath:substringAfterLast('/')} https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html Hope this helps.
... View more
09-19-2016
01:03 PM
Thanks. I'll give this a try!
... View more
09-15-2016
12:53 PM
(I posted another nifi question here if anyone reading this has an answer: https://community.hortonworks.com/questions/56616/options-for-exporting-large-data-sets-from-hive-to.html)
... View more
09-08-2016
04:18 PM
2 Kudos
@Zack Riesland Yes, it is safe to remove these folders and do a clean up. There are already actually cleanup scripts for this. Basically when a client runs a query with HiveServer2, Hive first creates these temporary folders to store intermediate/temporary data. For most queries, this is cleaned up at the end of query but sometimes due to issues with the query, these files are still hanging and you have to do a manual cleanup. Check this link for more details. https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-TemporaryFolders Following link might also give you some ideas on how to cleanup. https://community.hortonworks.com/questions/19204/do-we-have-any-script-which-we-can-use-to-clean-tm.html
... View more