Member since
05-20-2022
66
Posts
6
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1174 | 07-17-2023 11:25 PM | |
1158 | 04-17-2023 02:29 PM | |
5564 | 02-15-2023 08:47 PM | |
907 | 02-08-2023 06:02 PM | |
4422 | 10-17-2022 11:48 AM |
12-01-2023
01:13 PM
You are awesome @SAMSAL Thanks so much for the great information. This helps me a lot. Thank goodness for this community, because I think the documentation is nearly adequate.
... View more
12-01-2023
11:01 AM
@SAMSAL Thank you for the response. You are correct, this does work. I created a new flow from scratch and it worked. BUT, what I've noticed is that when you set the state on this processor it makes the state VERY STICKY. You can't change anything in this processor once it has been initialized and then ran. Even if you turn the processor off, it still remembers the state. I wasn't aware that the state was that persistent and sticky. So to make changes to the attribute variable you need to use and configure a new processor. The next question then is how do you initialize more than one attribute variable?
... View more
11-30-2023
07:46 PM
This question has been asked before (here) but never answered so I'm hopeful that today is my lucky day and someone knows the answer. Because unfortunately, the documentation does not cover the topic. In the UpdateAttribute there is an option to Store State of an attribute by setting the value to "Store state locally". But to do that I first need to set the "Stateful Variables Initial Value", but the documentation doesn't say how to do that, and everything I've tried fails. The documentation provides an example for referencing an existing attribute variable, theCount, but there is no mention of how to initialize this attribute variable. I've attempted setting the initial value using: theCount=1 {"theCount":1} ${theCount=1} 1 ${setStateValue('theCount')} <- grasping at straws with this one None of these work. When I look at the value of the attribute theCount it always just shows: "Empty string set". I'm including a screen shot of my processor so you can see the configuration. Also, including a screenshot of the Attributes tab showing the theCount attribute.
... View more
Labels:
- Labels:
-
Apache NiFi
08-30-2023
11:57 AM
Just to add to this, I created a Java version of this code which I verified works from the command line; I get the SSE feed printing to the console. However, when I attempt to use this same code in an ExecuteStreamCommand processor then I get the exact same behavior, which is that the processor is running but there isn't any data coming out of it. I'm missing a detail that I hope someone can shed some light on.
... View more
08-29-2023
10:36 PM
Hi @mmaher22 I spun my wheels on this for quite a while with no success; I can get the authorization token, but that's it. Do you have an example of using a script in the ExecuteScript (or ExecuteGroovyScript) that can make an HTTP request for a token and then use that token to start an SSE stream? I'd really appreciate whatever you are willing to share. Many thanks! Here is what I've come up with so far, but I can't get the SSE responses to output to flowfiles. @Grab(group='org.apache.httpcomponents', module='httpclient', version='4.5.13')
import org.apache.http.impl.client.CloseableHttpClient
import org.apache.http.impl.client.HttpClients
import org.apache.http.client.methods.HttpGet
import org.apache.http.HttpEntity
import org.apache.http.util.EntityUtils
import java.util.Base64
// Function to retrieve the access token
def retrieveAccessToken() {
def tokenUrl = new URL("http://kc.example.com/realms/aqua-services/protocol/openid-connect/token")
def clientId = "aqua-forma"
def clientSecret = "ls4kdjfOWIE5TRU6s2lkjfL3ASK9"
def grantType = "client_credentials"
def credentials = "${clientId}:${clientSecret}"
def credentialsBase64 = Base64.getEncoder().encodeToString(credentials.getBytes("utf-8"))
def authHeader = "Basic ${credentialsBase64}"
def data = "grant_type=${grantType}"
def connection = tokenUrl.openConnection() as HttpURLConnection
connection.setRequestMethod("POST")
connection.setRequestProperty("Authorization", authHeader)
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded")
connection.doOutput = true
def writer = new OutputStreamWriter(connection.getOutputStream())
writer.write(data)
writer.flush()
def responseCode = connection.getResponseCode()
if (responseCode == 200) {
def inputStream = connection.getInputStream()
def reader = new BufferedReader(new InputStreamReader(inputStream))
def response = new StringBuilder()
String line
while ((line = reader.readLine()) != null) {
response.append(line)
}
reader.close()
def tokenData = new groovy.json.JsonSlurper().parseText(response.toString())
return tokenData.access_token
} else {
return null
}
}
// SSE Code
def accessToken = retrieveAccessToken()
def sseUrl = "http://example.com/api/v1/read/search/sse?query=SELECT%20%2A%20FROM%20Game_Species"
// Create an HTTP client
CloseableHttpClient httpClient = HttpClients.createDefault()
try {
// Create an HTTP GET request
HttpGet httpGet = new HttpGet(sseUrl)
httpGet.setHeader("Authorization", "Bearer " + accessToken)
def response = httpClient.execute(httpGet)
def entity = response.getEntity()
if (entity != null) {
entity.content.eachLine { line ->
if (line.startsWith("data:")) {
def payload = line.substring(6).trim()
def flowFile = session.create()
flowFile = session.write(flowFile, { outputStream ->
outputStream.write(payload.getBytes("UTF-8"))
} as OutputStreamCallback)
session.transfer(flowFile, REL_SUCCESS)
}
}
}
} finally {
httpClient.close()
}
... View more
08-07-2023
11:37 AM
The only way to use OAuth2 that I'm aware of is with the InvokeHTTP processor, which in theory should work for connecting to SSE end points. However, I haven't been able to get it to work and I've tried adjusting many of the Request and Response parameters to no avail. If you figure it out then please share.
... View more
08-07-2023
08:30 AM
Are you referring to validation failures? If this is what you are referring to then the answer is, you can't. If you get failures then you'll need to queue the failures up and validate them manually to see why they failed. I use xmllint. For example: xmllint --noout --schema my_data_schema.xsd my_data.xml
... View more
07-17-2023
11:25 PM
1 Kudo
I managed to get it working by putting the parameter context reference in quotes and then using an evaluateString() function as shown below. ${#{'Entity Service'}:evaluateELString()}
... View more
07-17-2023
01:34 PM
Hello Community, I can assign an attribute's value using a reference to a parameter context value ( e.g. #{variablle_abc} ). But when I attempt to do this same thing within the "Advanced" window in an UpdateAttribute processor, it fails. I've verified the Process Group Parameter Context is set. Anyone know how to make this work in the Advanced window of an UpdateAttribute processor?
... View more
Labels:
- Labels:
-
Apache NiFi
06-01-2023
10:33 AM
https://issues.apache.org/jira/browse/NIFI-11627 Good Idea. Here is the reference.
... View more