NiFi Execute Stream Command Curl Call

[#1]

I am trying to use NiFi to make a CURL call to SMM. Because SMM is using kerberos authentication I want to make a POST request to the SMM API using Kerberos. I am now at the last step where I need to sent a POST request to create a Kafka topic. 

I am having a problem where when I send the request using the ExecuteStreamCommand I think some formatting happens to the payload before the call gets sent to the SMM API. I tried escaping the quotes in the payload but still no luck.

Here are my configurations for the ExecuteStreamCommand:

Command Arguments Strategy: Command Arguments Property

Command Arguments: -X POST -H referer:${Referer} -H 'Content-Type: application/json' -d  '{\"newTopics\": [{\"name\":\"testing123\",\"numPartitions\":3,\"replicationFactor\":3}], \"allTopicNames\":[\"testing123\"]}' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/topics

Command Path: curl

Ignore STDIN: false

Argument Delimiter: " " (Space)

Right now it's giving me a 415 - Unsupported Media Type Error.

Can someone please help.

[#2]

Any updates here?

[#3 · Accepted solution]

@drewski7 
The removal of quotes from the "command arguments" is expected behavior in the ExecuteStreamCommand processor.  This processor was introduced to NiFi more than 10 years ago and was originally designed for a more minimal scope of work including the expectation that FlowFile content would be passed to the script/command being executed.  

As time passed on the use cases that were trying to be solutioned via the ExecuteStreamCommand expanded; however, handling those use case would potential break users already implemented and working dataflow.  So rather then change that default behavior, a new property "Command Arguments Strategy" was added with the original "Command Arguments Property" as the default (legacy method) and a new "Dynamic property arguments" option.

This change is part of this JIra and implemented as of Apache NiFi 1.10:
https://issues.apache.org/jira/browse/NIFI-3221

In your use case, you'll want to switch to using the "Dynamic property arguments".  This will then require you to click on the "+" to add a new dynamic property.  The property names MUST use this format:

command.argument.<num>

So in your case you might try something like:

command.argument.1 = -X POST -H referer:${Referer} -H 'Content-Type: application/json' -d  '{"newTopics": [{"name":"testing123","numPartitions":3,"replicationFactor":3}], "allTopicNames":["testing123"]}' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/topics

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt