Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

Solved Go to solution
Highlighted

How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

New Contributor

I am able to Export and Import controller services from one instance to another.

Processors which are referencing to a controller service are not being mapped/selected.

Looking for better possible way.

 

Thanks in Advance

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

Master Guru

@anil35759 

 

If you create a NiFi template that includes a NiFi processor that references a controller service, that controller service will be included in the generated template.  So if you import and instantiate that template on to the canvas of another NiFi, the controller service will be added as well and be associated with the processor instantiate from that same template.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates

 

So I am not clear on why you need to import the controller service separately via the toolkit.

 

Controller services are not mapped to processors.  The association happens from the processor side.   The processor is mapped to a specific controller service via the controller services assigned uuid.

Keep in mind that anything you can do within the NiFi Ui can also be accomplished using NiFi rest-api endpoints.  Using your browsers developer tools to capture the requests as you make them through the NiFi UI is a great way to learn how to interact with NiFi rest-api endpoints.  The rest-api endpoints specific to your NiFi release version can be found under help within your install.
Here are the Apache NiFi rest-api docs for the latest Apache release version:
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

Essentially what you need to do here is to update the processor configuration to reference the UUID of whichever controller service you want it to use.

curl 'http://<nifi-hostname>:<nifi-port>/nifi-api/processors/b10fd083-0170-1000-0000-00007f7c905f' -X PUT --data-binary '{"component":{"id":"b10fd083-0170-1000-0000-00007f7c905f","name":"ExecuteSQL","config":{"concurrentlySchedulableTaskCount":"1","schedulingPeriod":"0 sec","executionNode":"ALL","penaltyDuration":"30 sec","yieldDuration":"1 sec","bulletinLevel":"WARN","schedulingStrategy":"TIMER_DRIVEN","comments":"","autoTerminatedRelationships":[],"properties":{"Database Connection Pooling Service":"e60cb24c-95c5-3a97-bcb9-9e537006317d"}},"state":"STOPPED"},"revision":{"clientId":"affea95c-0170-1000-988b-73bf756785b3","version":2},"disconnectedNodeAcknowledged":false}

 

So you'll notice from above example, I am updating a processor's (ExecuteSQL) configuration so that the Database Connection Pooling Service is mapped to the UUID of my DBCPConnectionPool controller service's uuid (e60cb24c-95c5-3a97-bcb9-9e537006317d).

 

Note:  The "clientId" string can be anything. 

 

Hope this helps,

Matt

 

View solution in original post

4 REPLIES 4
Highlighted

Re: How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

Master Guru

@anil35759 

 

It may be helpful if you can share the exact commands you are performing now to export and import your controller services.

Thanks,
Matt

Highlighted

Re: How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

New Contributor

I have created a sample controller service and mapped to a processor in source.

Source:

controller service.JPG

 

created a process group and execute sql processor and assigned created controller service to "Execute SQL" processor

Executesql before map.JPGMapped controller serviceMapped controller service

 

Able to export Controller service as Json object with Nifi toolkit Class "PGGetControllerServices".

 

Destination:

Created a new instance with different port 8086 and manually exported and imported Process group by creating a template.

Now my Destination instance looks similar to Source instance without any controller services.

 

Destination without controller serviceDestination without controller service

 

Now I am able to Import/create controller service from external location by using 

"PGCreateControllerService".

 

After importing to destinationAfter importing to destination

 

Now i am looking how to map the controller service with Processor using Toolkit or any is there any other way to achieve.

 

Thanks in Advance

Highlighted

Re: How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

Master Guru

@anil35759 

 

If you create a NiFi template that includes a NiFi processor that references a controller service, that controller service will be included in the generated template.  So if you import and instantiate that template on to the canvas of another NiFi, the controller service will be added as well and be associated with the processor instantiate from that same template.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates

 

So I am not clear on why you need to import the controller service separately via the toolkit.

 

Controller services are not mapped to processors.  The association happens from the processor side.   The processor is mapped to a specific controller service via the controller services assigned uuid.

Keep in mind that anything you can do within the NiFi Ui can also be accomplished using NiFi rest-api endpoints.  Using your browsers developer tools to capture the requests as you make them through the NiFi UI is a great way to learn how to interact with NiFi rest-api endpoints.  The rest-api endpoints specific to your NiFi release version can be found under help within your install.
Here are the Apache NiFi rest-api docs for the latest Apache release version:
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

Essentially what you need to do here is to update the processor configuration to reference the UUID of whichever controller service you want it to use.

curl 'http://<nifi-hostname>:<nifi-port>/nifi-api/processors/b10fd083-0170-1000-0000-00007f7c905f' -X PUT --data-binary '{"component":{"id":"b10fd083-0170-1000-0000-00007f7c905f","name":"ExecuteSQL","config":{"concurrentlySchedulableTaskCount":"1","schedulingPeriod":"0 sec","executionNode":"ALL","penaltyDuration":"30 sec","yieldDuration":"1 sec","bulletinLevel":"WARN","schedulingStrategy":"TIMER_DRIVEN","comments":"","autoTerminatedRelationships":[],"properties":{"Database Connection Pooling Service":"e60cb24c-95c5-3a97-bcb9-9e537006317d"}},"state":"STOPPED"},"revision":{"clientId":"affea95c-0170-1000-988b-73bf756785b3","version":2},"disconnectedNodeAcknowledged":false}

 

So you'll notice from above example, I am updating a processor's (ExecuteSQL) configuration so that the Database Connection Pooling Service is mapped to the UUID of my DBCPConnectionPool controller service's uuid (e60cb24c-95c5-3a97-bcb9-9e537006317d).

 

Note:  The "clientId" string can be anything. 

 

Hope this helps,

Matt

 

View solution in original post

Highlighted

Re: How to map/select controller service in processor after importing/creating controller service using Nifi toolkit

New Contributor

Thank you it helped and worked.

Don't have an account?
Coming from Hortonworks? Activate your account here