Member since 
    
	
		
		
		01-15-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                82
            
            
                Posts
            
        
                29
            
            
                Kudos Received
            
        
                10
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 8615 | 04-03-2017 09:35 PM | |
| 5431 | 12-29-2016 02:22 PM | |
| 1729 | 06-27-2016 11:18 AM | |
| 1318 | 06-21-2016 10:08 AM | |
| 1476 | 05-26-2016 01:43 PM | 
			
    
	
		
		
		03-17-2016
	
		
		05:27 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Use jar files from your azure cluster, not sandbox. You need exactly same versions of libs used on azure cluster.  Also copy core-site.xml to flume classpath (FLUME_HOME/conf should be fine)  Regards 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-16-2016
	
		
		11:52 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Seems a common issue for single node cluster and sandboxes:)  
 https://www.google.pl/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=hadoop+file+could+only+be+replicated+to+0+nodes+instead+of+minReplication+(%3D1)  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-16-2016
	
		
		11:45 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 This is actually steps for windows. And i tested it locally - it works 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-15-2016
	
		
		08:38 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 I can propose much easier steps:  1. Download flume binaries - http://flume.apache.org/download.html and extract it somewhere (this is going to be a FLUME_HOME)  2. Download winutils and put it somwhere (f.e. C:/winutils/bin, in this case C:/winutils is going to be a HADOOP_HOME)  3. Copy all missed hdfs libs to your FLUME_HOME/lib (you can find them in your hadoop cluster, is always preferable to have exact the same versions as in /usr/hdp/current/hadoop or /usr/hdp/current/hadoop-hdfs)  4. Run flume agent with the following command:  bin\flume-ng agent -name MyAgent -f conf/MyAgent.properties -property "flume.root.logger=INFO,LOGFILE,console;flume.log.file=MyLog.log;hadoop.home.dir=C:/winutils" 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-15-2016
	
		
		08:30 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 That should be commons-configuration, commons-io and htrace-core from /usr/hdp/current/hadoop/lib 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-14-2016
	
		
		03:36 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 I've never tried that scenario, but it should be possible.  All you need is to install flume on windows machine (just extract zip file) and add jars needed to connect to azure (if any).  You can use hdfs.kerberosPrincipal, hdfs.kerberosKeytab properties if you have secure hdfs  Regards 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-08-2016
	
		
		06:59 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Just put everything to single config.  Like  Agent1.sources..
Agent1.sinks..
Agent1.channels..
Agent2.sources..
Agent2.sinks..
Agent2.channels..
  Note, is possible to manage those agent separately (ambari will split them) and each one will run in a separate process 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-03-2016
	
		
		06:44 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		3 Kudos
		
	
				
		
	
		
					
							 Why dont use ambari rest api to manage Flume configs outside of main ambari screen.
Personally i didnt tested it, but it should be a valid approach.   Also one issue is still there (it prevents us to use ambari for Flume managing): https://issues.apache.org/jira/browse/AMBARI-9421
And the workaround mentioned doesn't works with more than 1 agent 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-03-2016
	
		
		06:35 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Replace  <ok to="get_run_date" />  With  <ok to="join-fork-actions" />  In general each of the "subflows" should end up with the join node. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-16-2016
	
		
		11:40 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I'd recommend to test connectivity first. You can do it with beeline tool. Once you connected to jdbc url from beeline u can continue experiments with eclipse.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		- « Previous
 - Next »