Member since 
    
	
		
		
		09-27-2018
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                138
            
            
                Posts
            
        
                23
            
            
                Kudos Received
            
        
                10
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 11962 | 02-28-2021 10:23 PM | |
| 2518 | 02-08-2021 11:53 PM | |
| 36711 | 12-16-2020 11:31 PM | |
| 9278 | 12-14-2020 11:02 PM | |
| 6420 | 12-14-2020 12:18 AM | 
			
    
	
		
		
		02-17-2021
	
		
		05:35 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @justenji      Thanks a lot! The suggestion you proposed works great!     However, the source of the problem was not a lack of schema definition. In fact, I managed also to have a working flow with the option Schema Access Strategy = Use String Fields From Header in the CSVReader processor.     The warning "None of the fields in the record map to the columns defined by the tablename table." is also obtained when the processor is not able to find the table and this can happen also when the table name is correctly configured in PutDatabaseRecord but there is some issue with user access rights (which ended up to be the actual cause of my error ...). 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-08-2021
	
		
		11:53 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @Jarinek     I think you have to set -pv in front of the new value:    nifi set-param -u http://localhost:8088 -pcid 85b8b5bf-0177-1000-3881-8ddd8d31fed8 -pn sample.URL -pv newvalue        This one worked on my local NiFi.    Remark: Haven't seen that you found it yourself. Great! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-08-2021
	
		
		07:30 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @ashwinrk       maybe this information will help you further:      https://community.cloudera.com/t5/Support-Questions/NiFi-InvokeHTTP-POST-JSON/m-p/307990#M223438     It depends on the concrete target whether the "Authorization" has to be base64encoded or you can use basic authentication. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-01-2021
	
		
		01:20 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @believe7   One possibility is shown here.  Maybe there is a more comfortable way, but this would be my solution.    Flow  Details processors        Result response  If you have further questions please let me know. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-24-2020
	
		
		07:35 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @adhishankarit     There is nothing you can pull form the NiFi Rest-API that is going to tell you about successful outcomes form processor execution on a FlowFile.  Depending on data volumes, this also sounds like a resource expensive endeavor.    That being said, NiFi does have a Site-To-SIte (S2S) Bulletin reporting task. When a processor throws an error it will produce a bulletin and this reporting task can capture those bulletins and send them via NiFi's S2S protocol to another NiFi instance directly into a dataflow were you can handle via dataflow design however you like.    Only way you can get INFO level logs in to bulletins is by setting the bulletin level to INFO on all you processors. This only works if you have also configured your NiFI logback.xml so that all NiFi components log at the INFO level as well.  Downsides to this:  1. Every processor would display the red bulletin square in upper right corner of processor, This makes using this to find components that are having issues difficult  2. This results in a lot of INFO level logging to the nifi-app.log.    You mention edge nodes.  You could setup a tailFile processor that tails the nifi-app.log and then send via a dataflow that log data via FlowFiles to some monitor NiFi cluster where another dataflows parses those records via a partitionRecord processor by log_level and then routes based on that log_level for additional handling/notification processing. Downside here:  1. Since you what to track success, you still need INFO level logging enabled for all components.  This means even this log collection flow is producing log output.  So large logs and logs being written to even when actual data being processed in other flows is not happening.    NiFi does have a master bulletin board which you could hit via the rest-api, but this does not get you past the massive logging you may be producing to monitor success.  https://nifi.apache.org/docs/nifi-docs/rest-api/index.html    Hope this gives you some ideas,  Matt 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-18-2020
	
		
		05:42 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Excellent news.  Can you accept the first 2 responses to close this solution?       
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-17-2020
	
		
		01:24 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @justenji   Same for me, I've tried to use avro schema generator, including the inferschema from Nifi, but no luck. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-15-2020
	
		
		12:44 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @justenji      Yes, exactly. All "main" elements are part of an array in the input. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-14-2020
	
		
		01:12 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @jainN      If you are looking to route flowfiles that end in json versus those that are not, check out RouteOnAttribute with something similar to json => ${filename:endsWith('.json')}.  You would use this after using your method of choice to list/fetch the files which would then provide a $filename for every flowfile.   With this json property added to RouteOnAttribute you can drag the json route to a triggering flow, and send everything else (not json: unmatched) to a holding flow.   NiFi Wait/Notify should be able to provide the trigger logic, but there are many other ways to do it with out wait/notify by using another datastore, map cache, etc.  For example, your non json flow could simply write to a new location and finish.  Then your json flow can process that new location some known amount of time later.  The logic there is your use case ofc ourse, the point is to use RouteOnAttribute to split your flow.     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks,  Steven 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-14-2020
	
		
		07:23 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello @justenji      Thanks for the detailed explanation, this is also what I have tested on my side and in my opinion this does the job being "record oriented" 🙂     Have a nice day 
						
					
					... View more