Member since 
    
	
		
		
		04-05-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                139
            
            
                Posts
            
        
                144
            
            
                Kudos Received
            
        
                16
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 43786 | 02-14-2019 02:53 PM | |
| 3177 | 01-04-2019 08:39 PM | |
| 12476 | 11-05-2018 03:38 PM | |
| 6281 | 09-27-2018 04:21 PM | |
| 3415 | 07-05-2018 02:56 PM | 
			
    
	
		
		
		05-25-2022
	
		
		01:41 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							  The ListS3 and FetchS3 processors in Apache NiFi are commonly used to retrieve objects from Amazon S3 buckets, but they can be easily configured to retrieve objects from IBM Cloud buckets.     Assume, I have an IBM Cloud bucket that contains three CSV files:            First, get the following from your IBM Cloud bucket configuration :   Bucket Name  Private Endpoint          Then, from the Service Credentials of your Cloud Object Storage, get:   Access Key ID  Secret Access Key        Note: If you don't have Service Credentials for the storage instance, create a new one with HMAC set to "true" (https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main)     Create or confirm that your IBM Cloud user has the necessary Bucket Access Policy to view and download objects (https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam-bucket-permissions) :            With this setup information confirmed, add to and connect ListS3 and FetchS3 processors on your NiFi canvas, similar to the following:            In the List S3 configuration, enter the Bucket, Access Key ID, Secret Access Key and Endpoint URL:             Note: The Region property is ignored when the Endpoint Override URL property is used.     Run the ListS3 processor and you will see a FlowFile generated for each of the bucket objects:            Looking at the queue details:            Now configure the FetchS3 similarly with the Bucket Name, Access Key ID, Secret Access Key and Endpoint Override URL:            Run the FetchS3 processor and the three CSV files are retrieved from the IBM Cloud Bucket:         
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
	
					
			
		
	
	
	
	
				
		
	
	
			
    
	
		
		
		05-04-2020
	
		
		11:46 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @yuvapraveen_k   NiFi Registry is not supported on Windows. You can see the system requirements here:  https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#system-requirements 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-06-2020
	
		
		01:06 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @MarkH Thanks for the comment and especially for the update. Glad to hear the flow is working for you and appreciate the workaround you documented to help others out in the community if they run into the same issue. It had been a while since I had run this flow so I tried it out today and ran a successful test in the latest released NiFi (Version 1.11.3). 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-16-2019
	
		
		01:57 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 The issue seems to be due to the error documented in the Registry Admin Guide (https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#system-requirements) if you don't have a JDK installed:     When running Registry with only a JRE you may encounter the following error as Flyway (database migration tool) attempts to utilize a resource from the JDK:     java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1.3 (DropBucketItemNameUniqueness)     Try installing the  JDK , deleting the `database` directory that was created prior and then start the Registry. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-14-2019
	
		
		02:51 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Are you sure the driver path is correct? Can you provide a screenshot of your Finder window where the jar file is located?  The only way I can reproduce the "Not a valid URL or file" warning is if the path is wrong. When I point to this path on my machine (Mac OS 10.14.2), I have no errors:  /Users/andrewmlim/Downloads/mysql-connector-java-8.0.17/mysql-connector-java-8.0.17.jar    Also, the HCC UI is not helpful since the link is very small, but it is better to select "Add comment" as a response to an answer instead of creating a new Answer. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-13-2019
	
		
		01:19 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I believe your path to your driver is incorrect. It should be:      /Users/kiran/Downloads/mysql/...      Here is an example processor setup from my CDC article (https://community.hortonworks.com/content/kbentry/113941/change-data-capture-cdc-with-apache-nifi-version-1-1.html😞         
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-23-2019
	
		
		02:22 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Thuy Le  Click on the value you want to change and you should see an edit dialog as follows:      
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-22-2019
	
		
		12:18 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @Thuy Le  Sorry if I am misunderstanding your question, but have you used the Variables Window to manage your variables?  https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables_Window 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-25-2019
	
		
		05:37 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I wrote a blog about this setup (https://community.hortonworks.com/content/kbentry/210286/storing-apache-nifi-versioned-flows-in-a-git-repos.html) and it seems like you did all the configuration steps I included.  However, I was not using Git VSTS. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		04-09-2019
	
		
		04:16 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 convert-xml-to-csv.xml 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
         
					
				













