Move Flows to another bucket in NiFi Registry

[#1]

I was curious if it was possible to move flows from one bucket to another and keep the version history in Registry. The way we have our environment set up today is we just have one bucket that we use in our DEV environment, as well as in production. It gets messy and we would like to clean it up. Here is my idea:

We have a bucket for each, DEV, Prod, & Archive. We keep versions controlled for the development process, if any, and when its ready to be deployed to another environment we can the move the flow versioning info over to the production bucket. And then if a flow is old but we don't want to delete it, we can then move it over to the archive bucket. 

The DEV to Prod bucket is not as important because we frequently are updating things that are in prod through DEV so it wouldn't be necessary to always move it back into the dev bucket since prod still tracks the version, however, the archive portion is important to us.

If moving flows from bucket to bucket is not possible in CFM 1.1 today, how do I add it as a feature request?

[#2]

We can export and import flow from one NiFi Registry to another, using nifi-toolkit cli.sh. 

Let's say we have two clusters, CFM-A and CFB-B, each running its own NiFi Registry. 

• CFM-A NiFi Registry URL : http://nifi-regA:18080
• CFM-B  NiFi Registry URL: http://nifi-regB:18080

Export flow from CFM-A NiFi Registry to the local directory:

/opt/cloudera/parcels/CFM-1.1.0.0/TOOLKIT/bin/cli.sh

Type 'help' to see a list of available commands, use tab to auto-complete.

1. List buckets from CFM-A NiFi Registry:

#> registry list-buckets -u http://nifi-regA:18080

#   Name         Id                                     Description   

-   ----------   ------------------------------------   -----------   

1   TestBucket   187b3d50-03ee-4e45-a717-eb113c6edbf2   (empty)    

2. List flows from CFM-A NiFi Registry using bucketIdentifier:

#> registry list-flows -u http://nifi-regA:18080 --bucketIdentifier 187b3d50-03ee-4e45-a717-eb113c6edbf2                            

#   Name           Id                                     Description   

-   ------------   ------------------------------------   -----------   

1   TailFileFlow   98ea1331-cd61-41bb-be06-0a0e85c9a275      

3. Export flow locally on the file-system using flowIdentifier. This command will store flow locally under file /tmp/test.json,  in Jason format, on the Node where cli.sh is running:

#> registry export-flow-version -u http://nifi-regA:18080 --flowIdentifier 98ea1331-cd61-41bb-be06-0a0e85c9a275 --outputFile /tmp/test.json --outputType json

Now we can Import /tmp/test.json flow to CFM-B NiFi Registry

• On CFM-B NiFi Registry, create an empty flow using create-flow. 
• Create a bucket on CFM-B NiFi Registry and use the same bucket OR create a new bucket. 
• List buckets from CFM-A NiFi Registry

#> registry list-buckets -u http://nifi-regB:18080 

# Name Id Description 

- ---- ------------------------------------ ----------- 

1 TestFlow cb152ab7-d569-4dcd-b332-8ca9025c8161 (empty)

1. Create flow-name test2 under bucket TestFlow, using bucketIdentifier:

#>registry create-flow -u http://nifi-regB:18080 --bucketIdentifier cb152ab7-d569-4dcd-b332-8ca9025c8161 --flowName test2

2. List flows from CFM-B NiFi Registry using bucketIdentifier:

#>registry list-flows -u http://nifi-regB:18080 --bucketIdentifier cb152ab7-d569-4dcd-b332-8ca9025c8161 

# Name Id Description 

- ---- ------------------------------------ ----------- 

1 test2 ef6550d5-6306-416a-9372-81286a635e7b (empty)

3. Import flow from local file /tmp/test.json using flowIdentifier, which can be viewed above the list-flows output:

#>registry import-flow-version -u http://nifi-regB:18080 --flowIdentifier ef6550d5-6306-416a-9372-81286a635e7b --input /tmp/test.json 

4. Check and verify flow name and version details on CFM-B NiFi Registry URL from the Web Browser.

[#3]

But what if we have 1 Registry that we want to move with in itself. We want to move from the bucket that is called Production, into the bucket that says Archive. Currently it appears they cant move inside and need to be "remove version control" and then "Add version control" to the other bucket.

[#4 · Accepted solution]

https://www.datainmotion.dev/2020/09/devops-working-with-parameter-contexts.html

download the flow/backup up/store in git

copy a flow to archive

remove from production

https://www.datainmotion.dev/2019/11/nifi-toolkit-cli-for-nifi-110.html