Member since 
    
	
		
		
		07-19-2020
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                2
            
            
                Posts
            
        
                0
            
            
                Kudos Received
            
        
                0
            
            
                Solutions
            
        
			
    
	
		
		
		07-20-2020
	
		
		11:27 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 You can use Nifi to save your Kafka messages into HDFS (for instance).     Something like this :         - ConsumeKafka : flowfile content is the Kafka message itself, and you have access to some attributes : topic name, partition, offset, key...(but not timestamp !). When i need it I store the timestamp in the key.  - ReplaceText : build your backup line using flowfile content and attributes  - MergeContent : to build a big file containing multiple Kafka message  - Extracttext : to set attribute to be used as filename  - PutHDFS : to save the created file into HDFS     And you can do the reverse if you need to push it bash to your kafka cluster. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        
