Member since
04-11-2016
466
Posts
317
Kudos Received
118
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
969 | 03-09-2018 05:31 PM | |
1189 | 03-07-2018 09:45 AM | |
1187 | 03-07-2018 09:31 AM | |
2074 | 03-03-2018 01:37 PM | |
1092 | 10-17-2017 02:15 PM |
09-16-2021
02:40 AM
1 Kudo
With the release of CDP 7.2.11, it is now super easy to deploy your custom components on your Flow Management DataHub clusters by dropping your components in a bucket of your cloud provider. Until now, when building custom components in NiFi, you had to SSH to all of your NiFi nodes to deploy your components and make them available to use in your flow definitions. This was adding an operational overhead and was also causing issues when scaling up clusters. From now on, it's easy to configure your NiFi clusters to automatically fetch custom components from an external location in the object store of the cloud provider where NiFi is running. All of your nodes will fetch the components after you dropped them in the configured location. You can find more information in the documentation.
... View more
09-16-2021
02:31 AM
1 Kudo
With the release of CDP 7.2.11, you now have the possibility to scale up and down both your light duty and heavy duty Flow Management clusters on all cloud providers. You can find more information in the documentation.
... View more
11-25-2018
11:17 AM
Not sure to understand the process you're following. Are you using the NiFi Registry? If yes, then when deploying a registered flow, it'll use the variables as defined when you committed the version in the Registry. After the import, in the target NiFi, you'd have to update the variables with the values of the target environment. Makes sense?
... View more
11-23-2018
03:42 PM
Hey. Are you sure you have deployed the correct version of your Groovy script on the NiFi node(s)? It looks like the Groovy is not correct. (I checked the one you provided and looks good to me though)
... View more
09-01-2018
04:46 PM
Hi @A
C
, Depending on the query you're executing, it might not result in an actual job to be running on YARN. It depends of your parameters around "fetch tasks" in Hive. Regarding supervisors, it's related to the Storm service, it's safe to ignore it. HTH
... View more
05-24-2018
08:28 AM
1 Kudo
Hi @Jérôme ROUSSEAU, A new version of the connector (1.5.4) should be available for download on the website very soon. Pierre
... View more
04-18-2018
09:42 AM
Hi @Chad Woodhead, Thanks for reporting the issue, I reproduced it and created NIFI-5092 to track it: https://issues.apache.org/jira/browse/NIFI-5092 Current workaround: after a NiFi restart, stop the reporting task, clear the state of the reporting task and start the reporting task. A fix will be provided very quickly and if needed you would be able to build the NAR containing the reporting task and drop it in NiFi to use the fixed version. Hope this helps, Pierre
... View more
03-16-2018
03:35 PM
Hi @Amira khalifa, You might want to have a look at ValidateRecord processor or ValidateCSV processor. Hope this helps.
... View more
03-12-2018
08:23 PM
1 Kudo
Hi @Haitam Dadsi, That something you could easily achieve using Apache NiFi. You could have a simple workflow looking like: GenerateTableFetch -> ExecuteSQL -> PublishKafkaRecord You will find plenty of examples for such processors. Hope this helps.
... View more
03-11-2018
09:58 AM
What you could do is to play with the combination of backpressure parameters and scheduling frequency for the first processor after the reception of the data. Or just use the ControlRate processor. When backpressure is enabled between the RPG and ControlRate, the RPG will stop pulling the data from the remote system.
... View more
03-10-2018
10:06 AM
Hi @Gireesh Kumar Gopinathan, I think you might be interested by this article: https://community.hortonworks.com/articles/109629/how-to-achieve-better-load-balancing-using-nifis-s.html Hope this helps.
... View more
03-09-2018
05:31 PM
1 Kudo
Hi @Jessica David, I confirm this is a bug. I created a JIRA for that: https://issues.apache.org/jira/browse/NIFI-4955 I will submit a fix in a minute. Thanks for reporting the issue. I assume you're using the header as the schema access strategy in the CSV Reader. If you're able to use a different strategy (schema name, or schema text), it should solve the problem even though you need to explicitly define the schema.
... View more
03-08-2018
08:55 AM
Hi @Saeed Barghi, To enable authentication/authorization on NiFi, the first requirement is to enable SSL. You will need to generate certificates for your nodes, create appropriate keystores/truststores and configure NiFi accordingly. One NiFi is secured, you have multiple options for authentication: individual SSL certificates, Kerberos, LDAP. In your case you would want to issue a certificate for each of your user and they will use it into their browser to authenticate against NiFi. Then you can define the appropriate authorizations to ensure multi-tenancy. This post might give you a better understanding of what needs to be done: https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/ If you're installing HDF with Ambari, then a lot of this can be automatically done for you using the internal CA provided with NiFi toolkit. Hope this helps.
... View more
03-08-2018
08:41 AM
1 Kudo
Hi @Sami Ahmad, As stated in the processor description/documentation: This processor uses Hive Streaming to send flow file data to an Apache
Hive table. The incoming flow file is expected to be in Avro format and
the table must exist in Hive. Please see the Hive documentation for
requirements on the Hive table (format, partitions, etc.). The partition
values are extracted from the Avro record based on the names of the
partition columns as specified in the processor. NOTE: If multiple
concurrent tasks are configured for this processor, only one table can
be written to at any time by a single thread. Additional tasks intending
to write to the same table will wait for the current task to finish
writing to the table. You'll need to convert your data into avro first. The best approach is to use the record processors for that. Hope this helps.
... View more
03-08-2018
08:38 AM
Could you share the full content of nifi.properties file? (remove any sensitive value) Did you configure something regarding authentication/authorization?
... View more
03-08-2018
08:25 AM
1 Kudo
That's completely valid. Let's say that HTTP has the advantage of not requiring any new configuration when installing NiFi. In some environments, adding the use of an additional port can create additional work with network/security teams. Another advantage of HTTP is the possibility to use a proxy if that's required to allow communication between multiple sites. If you expect to have high load with S2S and can manage the extra conf, RAW is certainly a better option. Hope this helps.
... View more
03-07-2018
09:51 AM
1 Kudo
Hi @User 805, You could use the InferAvroSchema processor to infer the schema from your JSON data and put this schema as an attribute of the flow files, then you can define your JSON Reader controller service to use this attribute as the schema source. Note that not defining in a schema registry the schema of each source you have might source of errors: if some of your JSONs are missing some fields, you will miss columns in the generated CSV. Also infering the schema for every single flow file can be costly in terms of performance. What I'd recommend is to use the InferAvroSchema processor to help you generating the schema for each one of your sources and then put the schemas in the schema registry of your choice and use it. I'll be more performant and you can leverage the features of the schema registry such as schema versioning, backward/forward compatibility, reuse of the schemas over multiple components, etc. Hope this helps.
... View more
03-07-2018
09:45 AM
1 Kudo
Hi @Chad Woodhead, This property needs to be set if and only if you are doing S2S using RAW protocol. Any reason not to use HTTP protocol for S2S? This way, you don't have anything to set and it'll work OOTB with default configuration. If, however, you want to use RAW and you are configuring NiFi using Ambari, you don't need to use config groups, you can set the property to: {{nifi_node_host}} It's a placeholder that contains the FQDN of each node. Note that this property already has a default value generated if you let it blank (see the description of this property in Ambari). Unless you have a specific network configuration, the only property you would need to specify is nifi.remote.input.socket.port to define the port that will be used for RAW S2S. Hope this helps.
... View more
03-07-2018
09:37 AM
Hi @Amira khalifa, Why don't you want to define the schema or your target tables in the schema registry? You can use InferAvroSchema to help you initializing the schema, then you can adapt it and copy/paste it into the schema registry of your choice. Using InferAvroSchema for every single flow file you'll ingest can be costly in terms of performance and it's only inferring the schema on a subset of data: as you can see, the schema might not be completely correct to take into account all the possibilities. What you could do, even though it's not what I'd recommend as a long term solution, is to use expression language to update the avro schema generated as an attribute of your flow files. Hope this helps.
... View more
03-07-2018
09:31 AM
Hi @Manikandan Jeyabal, This is not possible at the moment. The number of threads used at one point in time can be set at NiFi level. There is no way to fix the size of the threads pool per Process Group. It could be an interesting feature, feel free to raise a JIRA on https://issues.apache.org/jira/projects/NIFI Hope this helps.
... View more
03-03-2018
01:46 PM
OK, that's weird because the correct version is displayed as current on your hosts. You could try the following: ambari-server set-current --cluster=hdplid4 --version-display-name=HDP-2.6.4.0 If that does not help, I'd try looking at the 'host component state' table in Ambari database.
... View more
03-03-2018
01:37 PM
Hi, What are the RUNNING jobs that stop making progress? In particular are these jobs oozie-launcher jobs? You can check that by looking at the name of the jobs in the Scheduler view of the RM UI. Oozie has a non-intuitive way of launching jobs due to legacy behaviors from Hadoop 1 (note that this will be fixed with Oozie 5.x). In short, an Oozie action will launch an oozie-launcher job (1 AM container + 1 mapper) that will be responsible for actually launching the job that you defined in your Oozie action. In the end, your Oozie action will actually require two jobs from YARN point of view. When running multiple workflows at the same time, you could end with a lot of oozie-launcher jobs filling up the queue capacity and preventing the actual jobs to be launched (they will remain in ACCEPTED state). A common practice is to have a dedicated queue for oozie-launcher jobs created by Oozies workflows. This way you prevent this kind of deadlock situation. IIRC, you can set the queue for oozie-launcher jobs using oozie.launcher.mapred.job.queue.name. Hope this helps.
... View more
03-03-2018
10:18 AM
Hi Carlos, It sounds like a display bug. Did you try a force refresh in your browser? To be honest, I believe that the correct version will be installed if you add Druid because it'll rely on the repository files deployed when you installed the new version. Pierre
... View more
03-03-2018
10:16 AM
1 Kudo
Hi Jay, You need to provide a file such as https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/TestExtractGrok/patterns You could also use the ConvertRecord processor with a GrokReader. In this case there is already a default pattern file pre-loaded with the reader. Hope this helps, Pierre
... View more
12-22-2017
03:54 PM
Or... since you're not using a LDAP, you could directly use the File Authorizer instead of the managed one: <authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity"></property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
</authorizer> And then just reference this idenfitier in nifi.properties file: nifi.security.user.authorizer=file-provider
... View more
12-22-2017
03:46 PM
Hi @Zeeshan Cornelius, To have something configurable from NiFi UI (allowing you to manage users/groups from the Users view), I believe you'd need to go through the definition of a Composite Configurable User Group provider. Your authorizers.xml file should look like: <authorizers>
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">admin</property>
</userGroupProvider>
<userGroupProvider>
<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">composite-configurable-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">admin</property>
<property name="Legacy Authorized Users File"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers> Let me know if this helps, Pierre.
... View more
10-30-2017
11:01 AM
In case someone faces the same issue: in my case, I solved it by ensuring that 'atlas' user is known in Ranger.
... View more
10-17-2017
02:15 PM
1 Kudo
Hi @msumbul, The ConvertJsonToSQL processor has been developed to work with the SQL processors. The Hive processor is expecting a different naming convention on the attributes, instead of: sql.args.N.type/value it is expecting: hiveql.args.N.type/value The syntax between SQL and HQL are close enough and the generated query is also valid for Hive but the attributes need to be renamed. What you can do is to use an UpdateAttribute processor to have the expected attributes before using the PutHiveQL processor. Hope this helps.
... View more
09-19-2017
12:08 PM
1 Kudo
@Onkar Pathak You can use a GET for this. Example: GET http://localhost:8080/nifi-api/flow/process-groups/aa246331-015a-1000-fac3-b72f7046408b/controller-services Response: {"currentTime":"14:06:21 CEST","controllerServices":[{"revision":{"version":0},"id":"fbec5165-015a-1000-ef40-26bc04ec2b63","uri":"http://localhost:8080/nifi-api/controller-services/fbec5165-015a-1000-ef40-26bc04ec2b63","permissions":{"canRead":true,"canWrite":true},"bulletins":[],"component":{"id":"fbec5165-015a-1000-ef40-26bc04ec2b63","parentGroupId":"aa246331-015a-1000-fac3-b72f7046408b","name":"StandardHttpContextMap","type":"org.apache.nifi.http.StandardHttpContextMap","bundle":{"group":"org.apache.nifi","artifact":"nifi-http-context-map-nar","version":"1.4.0-SNAPSHOT"},"controllerServiceApis":[{"type":"org.apache.nifi.http.HttpContextMap","bundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}],"comments":"","state":"DISABLED","persistsState":false,"restricted":false,"deprecated":false,"multipleVersionsAvailable":false,"properties":{"Maximum Outstanding Requests":"10000","Request Expiration":"1 min"},"descriptors":{"Maximum Outstanding Requests":{"name":"Maximum Outstanding Requests","displayName":"Maximum Outstanding Requests","description":"The maximum number of HTTP requests that can be outstanding at any one time. Any attempt to register an additional HTTP Request will cause an error","defaultValue":"5000","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Request Expiration":{"name":"Request Expiration","displayName":"Request Expiration","description":"Specifies how long an HTTP Request should be left unanswered before being evicted from the cache and being responded to with a Service Unavailable status code","defaultValue":"1 min","required":true,"sensitive":false,"dynamic":false,"supportsEl":false}},"referencingComponents":[{"revision":{"version":0,"lastModifier":"anonymous"},"id":"fbec5167-015a-1000-998a-858ea9611244","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fc7d4a17-015a-1000-7731-9659aaa9b098","id":"fbec5167-015a-1000-998a-858ea9611244","name":"HandleHttpRequest","type":"HandleHttpRequest","state":"STOPPED","properties":{"Listening Port":"9999","Hostname":"localhost","SSL Context Service":null,"HTTP Context Map":"fbec5165-015a-1000-ef40-26bc04ec2b63","Allowed Paths":null,"Default URL Character Set":"UTF-8","Allow GET":"true","Allow POST":"true","Allow PUT":"true","Allow DELETE":"true","Allow HEAD":"false","Allow OPTIONS":"false","Additional HTTP Methods":null,"Client Authentication":"No Authentication","container-queue-size":"50"},"descriptors":{"Listening Port":{"name":"Listening Port","displayName":"Listening Port","description":"The Port to listen on for incoming HTTP requests","defaultValue":"80","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Hostname":{"name":"Hostname","displayName":"Hostname","description":"The Hostname to bind to. If not specified, will bind to all hosts","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"SSL Context Service":{"name":"SSL Context Service","displayName":"SSL Context Service","description":"The SSL Context Service to use in order to secure the server. If specified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requests","allowableValues":[],"required":false,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.ssl.RestrictedSSLContextService","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fc7d8d0b-015a-1000-e0f4-d79e69229742"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}},"Allowed Paths":{"name":"Allowed Paths","displayName":"Allowed Paths","description":"A Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFound","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"Default URL Character Set":{"name":"Default URL Character Set","displayName":"Default URL Character Set","description":"The character set to use for decoding URL parameters if the HTTP Request does not supply one","defaultValue":"UTF-8","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow GET":{"name":"Allow GET","displayName":"Allow GET","description":"Allow HTTP GET Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow POST":{"name":"Allow POST","displayName":"Allow POST","description":"Allow HTTP POST Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow PUT":{"name":"Allow PUT","displayName":"Allow PUT","description":"Allow HTTP PUT Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow DELETE":{"name":"Allow DELETE","displayName":"Allow DELETE","description":"Allow HTTP DELETE Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow HEAD":{"name":"Allow HEAD","displayName":"Allow HEAD","description":"Allow HTTP HEAD Method","defaultValue":"false","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow OPTIONS":{"name":"Allow OPTIONS","displayName":"Allow OPTIONS","description":"Allow HTTP OPTIONS Method","defaultValue":"false","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Additional HTTP Methods":{"name":"Additional HTTP Methods","displayName":"Additional HTTP Methods","description":"A comma-separated list of non-standard HTTP Methods that should be allowed","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"Client Authentication":{"name":"Client Authentication","displayName":"Client Authentication","description":"Specifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is not specified or the SSL Context provided uses only a KeyStore and not a TrustStore.","defaultValue":"No Authentication","allowableValues":[{"allowableValue":{"displayName":"No Authentication","value":"No Authentication","description":"Processor will not authenticate clients. Anyone can communicate with this Processor anonymously"},"canRead":true},{"allowableValue":{"displayName":"Want Authentication","value":"Want Authentication","description":"Processor will try to verify the client but if unable to verify will allow the client to communicate anonymously"},"canRead":true},{"allowableValue":{"displayName":"Need Authentication","value":"Need Authentication","description":"Processor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context Service"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"container-queue-size":{"name":"container-queue-size","displayName":"Container Queue Size","description":"The size of the queue for Http Request Containers","defaultValue":"50","required":true,"sensitive":false,"dynamic":false,"supportsEl":false}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=fbec5165-015a-1000-ef40-26bc04ec2b63] is disabled"],"referenceType":"Processor","activeThreadCount":0}},{"revision":{"version":0,"lastModifier":"anonymous"},"id":"fbec516d-015a-1000-3273-3d36a22c0501","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fc7d4a17-015a-1000-7731-9659aaa9b098","id":"fbec516d-015a-1000-3273-3d36a22c0501","name":"HandleHttpResponse","type":"HandleHttpResponse","state":"STOPPED","properties":{"HTTP Status Code":"${invokehttp.status.code:replaceNull('200')}","HTTP Context Map":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"descriptors":{"HTTP Status Code":{"name":"HTTP Status Code","displayName":"HTTP Status Code","description":"The HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.","required":true,"sensitive":false,"dynamic":false,"supportsEl":true},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fc7d8d0b-015a-1000-e0f4-d79e69229742"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=fbec5165-015a-1000-ef40-26bc04ec2b63] is disabled"],"referenceType":"Processor","activeThreadCount":0}},{"revision":{"version":0,"lastModifier":"anonymous"},"id":"fc7c07df-015a-1000-31cb-383c9f986aa6","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fc7d4a17-015a-1000-7731-9659aaa9b098","id":"fc7c07df-015a-1000-31cb-383c9f986aa6","name":"HandleHttpResponse","type":"HandleHttpResponse","state":"STOPPED","properties":{"HTTP Status Code":"200","HTTP Context Map":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"descriptors":{"HTTP Status Code":{"name":"HTTP Status Code","displayName":"HTTP Status Code","description":"The HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.","required":true,"sensitive":false,"dynamic":false,"supportsEl":true},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fc7d8d0b-015a-1000-e0f4-d79e69229742"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=fbec5165-015a-1000-ef40-26bc04ec2b63] is disabled"],"referenceType":"Processor","activeThreadCount":0}}],"extensionMissing":false}},{"revision":{"version":0},"id":"f6aa99d0-0159-1000-d428-5edef0de643e","uri":"http://localhost:8080/nifi-api/controller-services/f6aa99d0-0159-1000-d428-5edef0de643e","permissions":{"canRead":true,"canWrite":true},"bulletins":[],"component":{"id":"f6aa99d0-0159-1000-d428-5edef0de643e","parentGroupId":"aa246331-015a-1000-fac3-b72f7046408b","name":"StandardHttpContextMap","type":"org.apache.nifi.http.StandardHttpContextMap","bundle":{"group":"org.apache.nifi","artifact":"nifi-http-context-map-nar","version":"1.4.0-SNAPSHOT"},"controllerServiceApis":[{"type":"org.apache.nifi.http.HttpContextMap","bundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}],"comments":"","state":"DISABLED","persistsState":false,"restricted":false,"deprecated":false,"multipleVersionsAvailable":false,"properties":{"Maximum Outstanding Requests":"5000","Request Expiration":"1 min"},"descriptors":{"Maximum Outstanding Requests":{"name":"Maximum Outstanding Requests","displayName":"Maximum Outstanding Requests","description":"The maximum number of HTTP requests that can be outstanding at any one time. Any attempt to register an additional HTTP Request will cause an error","defaultValue":"5000","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Request Expiration":{"name":"Request Expiration","displayName":"Request Expiration","description":"Specifies how long an HTTP Request should be left unanswered before being evicted from the cache and being responded to with a Service Unavailable status code","defaultValue":"1 min","required":true,"sensitive":false,"dynamic":false,"supportsEl":false}},"referencingComponents":[{"revision":{"version":0,"lastModifier":"anonymous"},"id":"f6a768c2-0159-1000-99a8-009ed9ec4783","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fbec3833-015a-1000-ae7f-9ff2391f7f7b","id":"f6a768c2-0159-1000-99a8-009ed9ec4783","name":"HandleHttpResponse","type":"HandleHttpResponse","state":"STOPPED","properties":{"HTTP Status Code":"${invokehttp.status.code}","HTTP Context Map":"f6aa99d0-0159-1000-d428-5edef0de643e"},"descriptors":{"HTTP Status Code":{"name":"HTTP Status Code","displayName":"HTTP Status Code","description":"The HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.","required":true,"sensitive":false,"dynamic":false,"supportsEl":true},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=f6aa99d0-0159-1000-d428-5edef0de643e] is disabled"],"referenceType":"Processor","activeThreadCount":0}},{"revision":{"version":0,"lastModifier":"anonymous"},"id":"f6badf53-0159-1000-33cb-6ded4fad66f9","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fbec3833-015a-1000-ae7f-9ff2391f7f7b","id":"f6badf53-0159-1000-33cb-6ded4fad66f9","name":"HandleHttpResponse","type":"HandleHttpResponse","state":"STOPPED","properties":{"HTTP Status Code":"200","HTTP Context Map":"f6aa99d0-0159-1000-d428-5edef0de643e"},"descriptors":{"HTTP Status Code":{"name":"HTTP Status Code","displayName":"HTTP Status Code","description":"The HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.","required":true,"sensitive":false,"dynamic":false,"supportsEl":true},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=f6aa99d0-0159-1000-d428-5edef0de643e] is disabled"],"referenceType":"Processor","activeThreadCount":0}},{"revision":{"version":0,"lastModifier":"anonymous"},"id":"f6a4d400-0159-1000-b6cd-91781ea84d73","permissions":{"canRead":true,"canWrite":true},"component":{"groupId":"fbec3833-015a-1000-ae7f-9ff2391f7f7b","id":"f6a4d400-0159-1000-b6cd-91781ea84d73","name":"HandleHttpRequest","type":"HandleHttpRequest","state":"STOPPED","properties":{"Listening Port":"9999","Hostname":"localhost","SSL Context Service":null,"HTTP Context Map":"f6aa99d0-0159-1000-d428-5edef0de643e","Allowed Paths":null,"Default URL Character Set":"UTF-8","Allow GET":"true","Allow POST":"true","Allow PUT":"true","Allow DELETE":"true","Allow HEAD":"false","Allow OPTIONS":"false","Additional HTTP Methods":null,"Client Authentication":"No Authentication","container-queue-size":"50"},"descriptors":{"Listening Port":{"name":"Listening Port","displayName":"Listening Port","description":"The Port to listen on for incoming HTTP requests","defaultValue":"80","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Hostname":{"name":"Hostname","displayName":"Hostname","description":"The Hostname to bind to. If not specified, will bind to all hosts","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"SSL Context Service":{"name":"SSL Context Service","displayName":"SSL Context Service","description":"The SSL Context Service to use in order to secure the server. If specified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requests","allowableValues":[],"required":false,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.ssl.RestrictedSSLContextService","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}},"HTTP Context Map":{"name":"HTTP Context Map","displayName":"HTTP Context Map","description":"The HTTP Context Map Controller Service to use for caching the HTTP Request Information","allowableValues":[{"allowableValue":{"displayName":"StandardHttpContextMap","value":"f6aa99d0-0159-1000-d428-5edef0de643e"},"canRead":true},{"allowableValue":{"displayName":"StandardHttpContextMap","value":"fbec5165-015a-1000-ef40-26bc04ec2b63"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false,"identifiesControllerService":"org.apache.nifi.http.HttpContextMap","identifiesControllerServiceBundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.4.0-SNAPSHOT"}},"Allowed Paths":{"name":"Allowed Paths","displayName":"Allowed Paths","description":"A Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFound","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"Default URL Character Set":{"name":"Default URL Character Set","displayName":"Default URL Character Set","description":"The character set to use for decoding URL parameters if the HTTP Request does not supply one","defaultValue":"UTF-8","required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow GET":{"name":"Allow GET","displayName":"Allow GET","description":"Allow HTTP GET Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow POST":{"name":"Allow POST","displayName":"Allow POST","description":"Allow HTTP POST Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow PUT":{"name":"Allow PUT","displayName":"Allow PUT","description":"Allow HTTP PUT Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow DELETE":{"name":"Allow DELETE","displayName":"Allow DELETE","description":"Allow HTTP DELETE Method","defaultValue":"true","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow HEAD":{"name":"Allow HEAD","displayName":"Allow HEAD","description":"Allow HTTP HEAD Method","defaultValue":"false","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Allow OPTIONS":{"name":"Allow OPTIONS","displayName":"Allow OPTIONS","description":"Allow HTTP OPTIONS Method","defaultValue":"false","allowableValues":[{"allowableValue":{"displayName":"true","value":"true"},"canRead":true},{"allowableValue":{"displayName":"false","value":"false"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"Additional HTTP Methods":{"name":"Additional HTTP Methods","displayName":"Additional HTTP Methods","description":"A comma-separated list of non-standard HTTP Methods that should be allowed","required":false,"sensitive":false,"dynamic":false,"supportsEl":false},"Client Authentication":{"name":"Client Authentication","displayName":"Client Authentication","description":"Specifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is not specified or the SSL Context provided uses only a KeyStore and not a TrustStore.","defaultValue":"No Authentication","allowableValues":[{"allowableValue":{"displayName":"No Authentication","value":"No Authentication","description":"Processor will not authenticate clients. Anyone can communicate with this Processor anonymously"},"canRead":true},{"allowableValue":{"displayName":"Want Authentication","value":"Want Authentication","description":"Processor will try to verify the client but if unable to verify will allow the client to communicate anonymously"},"canRead":true},{"allowableValue":{"displayName":"Need Authentication","value":"Need Authentication","description":"Processor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context Service"},"canRead":true}],"required":true,"sensitive":false,"dynamic":false,"supportsEl":false},"container-queue-size":{"name":"container-queue-size","displayName":"Container Queue Size","description":"The size of the queue for Http Request Containers","defaultValue":"50","required":true,"sensitive":false,"dynamic":false,"supportsEl":false}},"validationErrors":["'HTTP Context Map' validated against 'StandardHttpContextMap' is invalid because Controller Service StandardHttpContextMap[id=f6aa99d0-0159-1000-d428-5edef0de643e] is disabled"],"referenceType":"Processor","activeThreadCount":0}}],"extensionMissing":false}}]}
... View more
09-19-2017
09:59 AM
1 Kudo
Hi @Balakrishnan Ramasamy, You might be interested by https://github.com/Chaffelson/nipyapi (if you want a python wrapper for the full NiFi API) Also note that there is on-going work around the sub-probect NiFi Registry (https://nifi.apache.org/registry.html) to ease the deployment of workflows between environments.
... View more