Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Guru

The objective of this article is to clearly demonstrate how to use NiFi's REST API to create a template on a secured cluster. I plan to publish multiple articles on using NiFi rest api based on community feedback around the use cases in the field. No pie in the sky type of knowledge share. Adding complexity to the workflows as I publish new articles and how they look via rest api publishing.

Use Case Setup:

Create template of my process group UVindex via rest api. UVindex resides within a process group hierarchy

Root->SunileManjeeProcessGroup->RestAPI Test->UNIndex

UNindex is a simply process groups which polls Chiago uv index data every 5 minutes and stores on hdfs. CurrentWeatherData is in the same process group but for this use case I only want to create template for UVIndex process group.

13279-ysg4r.png

Steps

  1. Fetch Access Token
  2. Fetch Client ID
  3. Fetch Process Group attributes
  4. Create Snippet
  5. Create Template

Step 1 - Fetch Access Token

Access token is required for rest api interactions on secured nifi cluster. Token will be used in the header for all rest api interactions

-H "Authorization: Bearer myLongToken

API

/nifi-api/access/token

Fetch Access Token

curl -X POST -H "Accept: */*" -H "Accept-Encoding: gzip" -H "Accept-Language: en-US,en;q=0.5" -H "Connection: keep-alive" -H "Content-Length: 38" -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "Host: https://nifi.com:9091" -H "Referer: https://nifi.com:9091/nifi/login" -H "X-Requested-With: XMLHttpRequest" -d 'username=myUserName&password=YourPasswordHere' https://nifi.com:9091/nifi-api/access/token | gunzip

Step 2

Fetch your client ID . Client ID will be used for all rest api interactions

curl -X GET -H "Accept: */*" -H "Accept-Encoding: gzip" -H "Accept-Language: en-US,en;q=0.5" -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJjbj1" -H "Connection: keep-alive" -H "Host: hdf20-0.field.hortonworks.com:9091" -H "Referer: https://nifi.com:9091/nifi/login" -H "X-Requested-With: XMLHttpRequest" https://nifi.com:9091/nifi-api/flow/client-id | gunzip

Step 3- Fetch Process Group Attributes

Fetch the UUID of the process group. To create a template a UUID of the object you want to create a template for is required.

API
/flow/process-groups/parentProcessGroupUUID

Or Rest API..Here my parent process group UUID is

8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a

Fetch Process Group Info

curl -X GET -H "Accept: */*" -H "Accept-Encoding: gzip" -H "Accept-Language: en-US,en;q=0.5" -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJjbj1OaUZpIFVzZXI" -H "Connection: keep-alive" -H "Host: https://nifi.com:9091" -H "Referer: https://nifi.com:9091/nifi/login" -H "X-Requested-With: XMLHttpRequest" https://nifi.com:9091/nifi-api/flow/process-groups/8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a | gunzip

Which returns large json object. Here you can parse the json object for the name of the process group (UVindex) and grab the UUID and version.

13295-3ubke.png

Step 4 - Create Snippet

To create a template, first a snippet must be created . The snippet ID will be used to name and create template. All details of what a template should be composed of are within a snippet

API

/nifi-api/snippets

For the payload populate PartentGroupID using the parent process group UUID. Under "processGroups" populate with UUID of the process group you want to create a template of, your client ID, and version captured in step 3. You may have connections or other objects which may require values within the payload. For this template it does not. In follow up articles I will create higher complexity work flows which will require such objection to be populated during snippet creation.

{
  "snippet": {
    "parentGroupId": "8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a",
    "processors": {},
    "funnels": {},
    "inputPorts": {},
    "outputPorts": {},
    "remoteProcessGroups": {},
    "processGroups": {
      "d8b73f15-e918-1afc-9db2-e7cdfbf19076": {
        "clientId": "980e3bca-b76a-183f-2e3b-5e75dccd3aec",
        "version": 1
      }
    },
    "connections": {},
    "labels": {}
  }
}

Example

curl -X POST -H "Accept: application/json, text/javascript, */*; q=0.01" -H "Accept-Encoding: gzip" -H "Accept-Language: en-US,en;q=0.5" -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzd" -H "Connection: keep-alive" -H "Content-Length: 305" -H "Content-Type: application/json" -H "Host: nifi.com:9091" -H "Referer: https://hdf20-0.field.hortonworks.com:9091/nifi/login" -H "X-Requested-With: XMLHttpRequest" -d '{"snippet":{"parentGroupId":"8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a","processors":{},"funnels":{},"inputPorts":{},"outputPorts":{},"remoteProcessGroups":{},"processGroups":{"d8b73f15-e918-1afc-9db2-e7cdfbf19076":{"clientId":"980e3bca-b76a-183f-2e3b-5e75dccd3aec","version":1}},"connections":{},"labels":{}}}' https://nifi.com:9091/nifi-api/snippets | gunzip

Output

{
  "snippet": {
    "id": "980e46c5-b76a-183f-ffff-ffffb4770059",
    "uri": "https://nifi.com:9091/nifi-api/process-groups/8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a/snippets/980e46c5-b76a-183f-ffff-ffffb4770059",
    "parentGroupId": "8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a",
    "processGroups": {
      "d8b73f15-e918-1afc-9db2-e7cdfbf19076": {
        "clientId": "980e3bca-b76a-183f-2e3b-5e75dccd3aec",
        "version": 1
      }
    },
    "remoteProcessGroups": {},
    "processors": {},
    "inputPorts": {},
    "outputPorts": {},
    "connections": {},
    "labels": {},
    "funnels": {}
  }
}

Fetch the snippet ID from the output

Step 5 - Create Template

Using the snippet ID from step 4 create a template named "My Template"

Payload

{
  "name": "My Template",
  "description": "",
  "snippetId": "980e46c5-b76a-183f-ffff-ffffb4770059"
}

API

/nifi-api/process-groups/ParentProcessGroupUUID/templates

Example

curl -X POST -H "Accept: application/json, text/javascript, */*; q=0.01" -H "Accept-Encoding: gzip" -H "Accept-Language: en-US,en;q=0.5" -H "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJz" -H "Connection: keep-alive" -H "Content-Length: 90" -H "Content-Type: application/json" -H "Host: nifi.com:9091" -H "Referer: https://nifi.com:9091/nifi/login" -H "X-Requested-With: XMLHttpRequest" -d '{"name":"My Template","description":"","snippetId":"980e46c5-b76a-183f-ffff-ffffb4770059"}' https://nifi.com:9091/nifi-api/process-groups/8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a/templates | gunzip

Which returns

{
  "template": {
    "uri": "https://nifi.com:9091/nifi-api/templates/e3cad030-733e-3568-b7f5-90b2457e1637",
    "id": "e3cad030-733e-3568-b7f5-90b2457e1637",
    "groupId": "8c3d29aa-d1ea-1c85-ffff-ffffe6b3853a",
    "name": "My Template",
    "description": "",
    "timestamp": "03/05/2017 05:29:21 UTC",
    "encoding-version": "1.0"
  }
}

In the NiFi UI you will see the newly created template

13330-3.jpg


ylkmt.pngp92pz.pngtuojs.png
10,129 Views
Comments
Not applicable

hi, i have a question

you write in the sektion Fetch Access Token

  1. -d 'username=myUserName&password=YourPasswordHere' \

where i find the username and the password of nifi, while i used it without a username and password!

when i need to create one how could i do it?

Contributor

Hi, i am trying to follow your steps. unfortunately getting error, please look once

Step1: i am using to get token

curl -X POST --insecure -H \ -H "Accept: */*" \ -H "Accept-Encoding: gzip" \ -H "Accept-Language: en-US,en;q=0.5" \ -H "Connection: keep-alive" \ -H "Content-Length: 38" \ -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" \ -H "Host: https://nifitest.com/nifi" \ -H "Referer: https://nifitest.com/nifi/login" \ -H "X-Requested-With: XMLHttpRequest" \ -d 'username=XXXX&password=XXXX' \ https://nifitest.com/nifi-api/access/token | gunzip

error:

Curl: (6) Couldn't resolve host 'Accept: *' %

Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 34 0 298 --:--:-- --:--:-- --:--:-- 0 gzip: stdin: unexpected end of file

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 01:57 PM
Updated by:
 
Contributors
Top Kudoed Authors