Member since 
    
	
		
		
		05-16-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                785
            
            
                Posts
            
        
                114
            
            
                Kudos Received
            
        
                39
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2324 | 06-12-2019 09:27 AM | |
| 3568 | 05-27-2019 08:29 AM | |
| 5719 | 05-27-2018 08:49 AM | |
| 5232 | 05-05-2018 10:47 PM | |
| 3110 | 05-05-2018 07:32 AM | 
			
    
	
		
		
		06-25-2017
	
		
		03:32 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi Ravi      They have this issue fixed in hive 3.0.0 , They have been able to add hive support for "Not NULL " & "UNIQUE" constraint . Check out this link      https://issues.apache.org/jira/browse/HIVE-16575        Regards   Vishnu Gupta  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-24-2017
	
		
		07:19 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 [cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat	or CentOS 7 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5/
gpgkey =https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/RPM-GPG-KEY-cloudera    
gpgcheck = 1        [cloudera-impala]
name=Impala
baseurl=http://archive.cloudera.com/impala/redhat/6/x86_64/impala/1/
gpgkey = http://archive.cloudera.com/impala/redhat/6/x86_64/impala/RPM-GPG-KEY-cloudera    
gpgcheck = 1    we used this on our Centos 7 but it was for Centos 6 only                Refer this link   https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_cdh5_install.html#topic_4_4_1__p_32  Hi above link for Centos 7       $ sudo yum install impala             
$ sudo yum install impala-server      
$ sudo yum install impala-state-store 
$ sudo yum install impala-catalog      
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-23-2017
	
		
		11:44 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @VincentSF     oh ok got it...      Go to CM -> Yarn -> Configuration -> search for "yarn.nodemanager.resource.memory-mb"     it will show you the memory restriction that you set for each node (it will get configuration from yarn-site.xml)     You can tweak this 'little'.        Note:   1. The memory is common for all the services. so you cannot use all the memory for Yarn alone. Also don't increase the memroy for the above setting too much because it may create memory overlap issue accross the services. So may be you can set aprox 50% of total memory but again it is depends upon the memory utilization by other services. Since you have 183 nodes, the 50% is not common for all the nodes, it will change case by case     2. Also when you increase your memory on each node, it is not recommended to increase more than   yarn.scheduler.maximum-allocation-mb     Hope this will give some idea    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-23-2017
	
		
		06:30 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 This has to do with the YARN memory settings.  The amount of memory allocated to yarn is only 8 GB.  I don't know what the minimum container size is, probably around 1.3 G.  That combination of the two determine the amount of containers that can be launched.  The result of that for your cluster is 6 containers.  Anything beyond that will have to wait for resources to be freed up.     https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/  https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_ig_yarn_tuning.html 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-20-2017
	
		
		09:58 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @pdvorak,     thanks a lot for you answer, I've already checked that page and it helped.     Thanks again.  DB 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-19-2017
	
		
		09:38 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Went ahead and downloaded a fresh .jar and followed the steps in the guide posted above - got it working! Appreciate the help. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-16-2017
	
		
		06:20 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Geek007     in other words   in SQL  We say “what” is to be accomplished   in Pig,  we mention “how” a task is to be performed.     when to go for Pig - When we want to process larget set of unorganized, unstructured and decentralized data  Pig doesnt need to have a schema , it will consume unstructured data with delimiters .   please look in to this example .      https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=23494745     let me know if this suffice  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-13-2017
	
		
		02:12 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @mbigelow Sorry for the late response , was on Vacation :))   bleow is my answer to the questionaire      is size the amount after the merge?  Yes it .  What was the average size before?    between 50kb to 100 kb .  How long did it take to run?   10-15 minutes                
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-01-2017
	
		
		08:37 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thank you, I gonna verify it. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













