Member since
05-16-2016
785
Posts
114
Kudos Received
39
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1834 | 06-12-2019 09:27 AM | |
3046 | 05-27-2019 08:29 AM | |
5073 | 05-27-2018 08:49 AM | |
4448 | 05-05-2018 10:47 PM | |
2763 | 05-05-2018 07:32 AM |
06-25-2017
03:32 PM
Hi Ravi They have this issue fixed in hive 3.0.0 , They have been able to add hive support for "Not NULL " & "UNIQUE" constraint . Check out this link https://issues.apache.org/jira/browse/HIVE-16575 Regards Vishnu Gupta
... View more
06-24-2017
07:19 AM
[cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 7 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5/
gpgkey =https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/RPM-GPG-KEY-cloudera
gpgcheck = 1 [cloudera-impala]
name=Impala
baseurl=http://archive.cloudera.com/impala/redhat/6/x86_64/impala/1/
gpgkey = http://archive.cloudera.com/impala/redhat/6/x86_64/impala/RPM-GPG-KEY-cloudera
gpgcheck = 1 we used this on our Centos 7 but it was for Centos 6 only Refer this link https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_cdh5_install.html#topic_4_4_1__p_32 Hi above link for Centos 7 $ sudo yum install impala
$ sudo yum install impala-server
$ sudo yum install impala-state-store
$ sudo yum install impala-catalog
... View more
06-23-2017
11:44 AM
1 Kudo
@VincentSF oh ok got it... Go to CM -> Yarn -> Configuration -> search for "yarn.nodemanager.resource.memory-mb" it will show you the memory restriction that you set for each node (it will get configuration from yarn-site.xml) You can tweak this 'little'. Note: 1. The memory is common for all the services. so you cannot use all the memory for Yarn alone. Also don't increase the memroy for the above setting too much because it may create memory overlap issue accross the services. So may be you can set aprox 50% of total memory but again it is depends upon the memory utilization by other services. Since you have 183 nodes, the 50% is not common for all the nodes, it will change case by case 2. Also when you increase your memory on each node, it is not recommended to increase more than yarn.scheduler.maximum-allocation-mb Hope this will give some idea
... View more
06-23-2017
06:30 AM
This has to do with the YARN memory settings. The amount of memory allocated to yarn is only 8 GB. I don't know what the minimum container size is, probably around 1.3 G. That combination of the two determine the amount of containers that can be launched. The result of that for your cluster is 6 containers. Anything beyond that will have to wait for resources to be freed up. https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_ig_yarn_tuning.html
... View more
06-20-2017
09:58 AM
Hi @pdvorak, thanks a lot for you answer, I've already checked that page and it helped. Thanks again. DB
... View more
06-19-2017
09:38 AM
Went ahead and downloaded a fresh .jar and followed the steps in the guide posted above - got it working! Appreciate the help.
... View more
06-16-2017
06:20 PM
@Geek007 in other words in SQL We say “what” is to be accomplished in Pig, we mention “how” a task is to be performed. when to go for Pig - When we want to process larget set of unorganized, unstructured and decentralized data Pig doesnt need to have a schema , it will consume unstructured data with delimiters . please look in to this example . https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=23494745 let me know if this suffice
... View more
06-13-2017
02:12 AM
@mbigelow Sorry for the late response , was on Vacation :)) bleow is my answer to the questionaire is size the amount after the merge? Yes it . What was the average size before? between 50kb to 100 kb . How long did it take to run? 10-15 minutes
... View more
06-01-2017
08:37 PM
Thank you, I gonna verify it.
... View more