Member since
11-21-2022
7
Posts
0
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
403 | 02-21-2023 07:30 AM | |
452 | 02-16-2023 06:38 AM | |
646 | 02-15-2023 02:00 PM |
02-21-2023
07:30 AM
I used Correlation Attribute Name , setting it to `${schema.name}`, and it's working as expected. Quote from documentation: > If specified, two FlowFiles will be binned together only if they have the same value for this Attribute. If not specified, FlowFiles are bundled by the order in which they are pulled from the queue.
... View more
02-16-2023
06:38 AM
I was able to achieve this with `ExecuteScript` with pyhon. import json
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
flow_file = session.get()
if flow_file is not None:
# get the flowfile content as json
stream_content = session.read(flow_file)
text_content = IOUtils.toString(stream_content, StandardCharsets.UTF_8)
json_content = json.loads(text_content)
# close the stream; without this, `putAttribute` will fail
stream_content.close()
# get table name and fields
table_name = json_content["name"]
fields = json_content["fields"]
columns = ",".join(col["name"] for col in fields)
# format the select statement
select_statement = "select " + columns + " from " + table_name
# set the select statement as an attr
flow_file = session.putAttribute(flow_file, "select_statement", select_statement)
# finalize and pass it on
session.transfer(flow_file, REL_SUCCESS)
session.commit()
... View more
02-15-2023
02:00 PM
Figured out the issue. The java SDK that nifi uses does not honour "noProxy" flags.
... View more
11-21-2022
07:02 AM
I posted a similar thread here. Nifi doesn't seem to be honouring HTTP_PROXY settings. https://community.cloudera.com/t5/Support-Questions/Nifi-tasks-not-honouring-HTTP-PROXY-settings/m-p/358074#M237742
... View more