Member since 
    
	
		
		
		02-24-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                175
            
            
                Posts
            
        
                56
            
            
                Kudos Received
            
        
                3
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1922 | 06-16-2017 10:40 AM | |
| 16466 | 05-27-2016 04:06 PM | |
| 1631 | 03-17-2016 01:29 PM | 
			
    
	
		
		
		08-18-2016
	
		
		02:31 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Smart Solutions  HDP 2.5 should be available in the next weeks. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-22-2016
	
		
		01:15 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 Hi @Smart Solutions,  I think this would be sufficient to certify that the libraries are installed and your applications will be able to find them. You can find several examples that are ready to run under /usr/hdp/current/spark-client/examples/src/main/python/mllib. You can substitute python with your preferred language to find examples that correspond to the appropriate API.  In terms of optimized configurations, it is hard to tune that upfront as it will be highly dependent upon on your application, dataset, and cluster. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-15-2016
	
		
		05:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 There could be a problem with the certificate itself. I recommend regenerating it and trying again. You can follow instructions in the Apache Knox Users Guide to generate a self-signed certificate:  http://knox.apache.org/books/knox-0-6-0/user-guide.html#Generating+a+self-signed+cert+for+use+in+testing+or+development+environments  If you want to use a more legitimate certificate you can generate and sign it yourself with OpenSSL or from a CA, and follow the steps in the next section of the guide, Using a CA Signed Key Pair. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-01-2016
	
		
		01:31 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Steven Hirsch   I think you can try it for one application, one possibility is to simply switch off ATS for a bit that helped me once but not a second time ( Tez still tries to log to it ) So if you really want to switch it off completely you can add the following settings:  tez.history.logging.service.class
= org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
and to see the logs:
tez.simple.history.logging.dir=/mylogdir
  Also removing the following ATSHooks  hive.exec.pre.hooks=
hive.exec.post.hooks=  Also potentially reduce log levels  hive.tez.log.level=ERROR  And see if it makes things faster. Again if you don't see a difference you may have other issues. But its worth to rule out. ATS 1.5 has been enabled in HDP2.4. Also ATS1.0 has some tuning options. If that is really your bottleneck Hortonworks support may be able to help.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-20-2016
	
		
		11:37 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have the same pb but i don't know how to delete my application from spark-history. Could you explain, please ?  thanks 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-09-2016
	
		
		04:33 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 HI @Jitendra Yadav , I restarted couple of times and it worked smoothly so did not do anything else.   Thank you 🙂 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-07-2016
	
		
		03:49 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Alex Miller Well in our scenario both the A/D are more or less replicas. Anyway I got this fixed.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-25-2016
	
		
		01:15 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Hi @Smart Solutions.  Right now... there isn't really a good, quick, clean, easy way to achieve this.  You've already identified the thread that I would otherwise point you towards for some ideas. You're just too good!  There are two main approaches that I would recommend thinking about, the first is ceasing to make any changes via the web-ui, and only making changes via the API, that way you can just call both your clusters one after the other to make the configuration changes.  The second is to use some of the ideas from the thread you linked to, where you would continue maintaining the configs on your "master" cluster, but then extract the configs from your "master" cluster on a regular basis (or on a trigger of config changed?), diff them between the previous "master" cluster config version, and then push the resulting deltas to your "slave" cluster, again via API calls.  Either way, there's quite a bit of automation that would be required there.  I'd strongly suggest if you want to go down this path, doing your work out in the open, this is something I see come up now and again, so I think you may well get others who would be interested in working with you on this.  Longer term, Ambari will no doubt support multi cluster, and this functionality would be a natural extension of that, but progress on those public JIRA's has been slow, with other more important items taking priority.  Happy to hear if you have other ideas too, sorry I couldn't be more direct help, but let me know if you plan on cutting some code moving forward, I'm sure it'd be an interesting project.  Many thanks. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-25-2016
	
		
		09:39 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 @Smart Solutions  You can put several Knox instances behind a load balancer to have HA. So multiple Knox instances are required for HA  Knox documentation gives an example on how to do it with Apache HTTP Server https://knox.apache.org/books/knox-0-9-0/user-guide.html#High+Availability  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
         
					
				













