Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Move Flows to another bucket in NiFi Registry

Solved Go to solution

Move Flows to another bucket in NiFi Registry

New Contributor

I was curious if it was possible to move flows from one bucket to another and keep the version history in Registry. The way we have our environment set up today is we just have one bucket that we use in our DEV environment, as well as in production. It gets messy and we would like to clean it up. Here is my idea:

 

We have a bucket for each, DEV, Prod, & Archive. We keep versions controlled for the development process, if any, and when its ready to be deployed to another environment we can the move the flow versioning info over to the production bucket. And then if a flow is old but we don't want to delete it, we can then move it over to the archive bucket. 

 

The DEV to Prod bucket is not as important because we frequently are updating things that are in prod through DEV so it wouldn't be necessary to always move it back into the dev bucket since prod still tracks the version, however, the archive portion is important to us.

 

If moving flows from bucket to bucket is not possible in CFM 1.1 today, how do I add it as a feature request?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Move Flows to another bucket in NiFi Registry

Super Guru
3 REPLIES 3
Highlighted

Re: Move Flows to another bucket in NiFi Registry

Cloudera Employee

We can export and import flow from one NiFi Registry to another, using nifi-toolkit cli.sh. 

Let's say we have two clusters, CFM-A and CFB-B, each running its own NiFi Registry. 

Export flow from CFM-A NiFi Registry to the local directory:

 

/opt/cloudera/parcels/CFM-1.1.0.0/TOOLKIT/bin/cli.sh

 

Type 'help' to see a list of available commands, use tab to auto-complete.

 

1. List buckets from CFM-A NiFi Registry:

 

#> registry list-buckets -u http://nifi-regA:18080

 

#   Name         Id                                     Description   

-   ----------   ------------------------------------   -----------   

1   TestBucket   187b3d50-03ee-4e45-a717-eb113c6edbf2   (empty)    

 

2. List flows from CFM-A NiFi Registry using bucketIdentifier:

 

#> registry list-flows -u http://nifi-regA:18080 --bucketIdentifier 187b3d50-03ee-4e45-a717-eb113c6edbf2                            

 

#   Name           Id                                     Description   

-   ------------   ------------------------------------   -----------   

1   TailFileFlow   98ea1331-cd61-41bb-be06-0a0e85c9a275      

 

3. Export flow locally on the file-system using flowIdentifier. This command will store flow locally under file /tmp/test.json,  in Jason format, on the Node where cli.sh is running:

 

#> registry export-flow-version -u http://nifi-regA:18080 --flowIdentifier 98ea1331-cd61-41bb-be06-0a0e85c9a275 --outputFile /tmp/test.json --outputType json

 

Now we can Import /tmp/test.json flow to CFM-B NiFi Registry

 

  • On CFM-B NiFi Registry, create an empty flow using create-flow
  • Create a bucket on CFM-B NiFi Registry and use the same bucket OR create a new bucket. 
  • List buckets from CFM-A NiFi Registry

#> registry list-buckets -u http://nifi-regB:18080 

 

 

# Name Id Description 

 

- ---- ------------------------------------ ----------- 

 

1 TestFlow cb152ab7-d569-4dcd-b332-8ca9025c8161 (empty)

 

1. Create flow-name test2 under bucket TestFlow, using bucketIdentifier:

 

#>registry create-flow -u http://nifi-regB:18080 --bucketIdentifier cb152ab7-d569-4dcd-b332-8ca9025c8161 --flowName test2

 

2. List flows from CFM-B NiFi Registry using bucketIdentifier:

 

#>registry list-flows -u http://nifi-regB:18080 --bucketIdentifier cb152ab7-d569-4dcd-b332-8ca9025c8161 

 

 

# Name Id Description 

 

- ---- ------------------------------------ ----------- 

 

1 test2 ef6550d5-6306-416a-9372-81286a635e7b (empty)

 

3. Import flow from local file /tmp/test.json using flowIdentifier, which can be viewed above the list-flows output:

 

#>registry import-flow-version -u http://nifi-regB:18080 --flowIdentifier ef6550d5-6306-416a-9372-81286a635e7b --input /tmp/test.json 

 

4. Check and verify flow name and version details on CFM-B NiFi Registry URL from the Web Browser.

Highlighted

Re: Move Flows to another bucket in NiFi Registry

New Contributor

But what if we have 1 Registry that we want to move with in itself. We want to move from the bucket that is called Production, into the bucket that says Archive. Currently it appears they cant move inside and need to be "remove version control" and then "Add version control" to the other bucket.

Highlighted

Re: Move Flows to another bucket in NiFi Registry

Super Guru
Don't have an account?
Coming from Hortonworks? Activate your account here