Member since
07-28-2017
47
Posts
6
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
15109 | 03-13-2018 12:04 PM | |
8537 | 10-12-2017 08:52 AM |
08-29-2023
08:39 AM
Can you help me with a similar issue? My code: import json import collections import java.io import ast from org.apache.commons.io import IOUtils from java.nio.charset import StandardCharsets from org.apache.nifi.processor.io import StreamCallback class ModJSON(StreamCallback) : def __init__(self): pass def process(self, inputStream, outputStream): text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) obj = ast.literal_eval(text) mail = obj['mail'] outputStream.write(bytearray(json.dumps(obj['mail'], indent=4).encode('utf-8'))) outputStream.write(bytearray(json.dumps(obj['id'], indent=4).encode('utf-8'))) flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile, ModJSON()) flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').split('.')[0]+'_translated.json') flowFile = session.putAttribute(flowFile, "mail", mail) session.transfer(flowFile, REL_SUCCESS) session.commit() I want to get "mail" value and create a new attribute for it. However, mail variable is not accessible when I insert it into the putattribute command. Any feedback?
... View more
07-21-2021
11:27 PM
@Kasun, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.
... View more
04-22-2020
09:17 AM
check this https://community.cloudera.com/t5/Support-Questions/Apache-Nifi-username-password-for-Rest-Api/td-p/206404
... View more
02-22-2018
03:05 PM
@balalaika ok, with no code changes here is the Structured version. Structured Streaming is basically just GA in Spark 2.2 which is HDP 2.6.4 and above. It works fine, a little different from the old style. They will probably keep the old one for until 2.5 or maybe 3.0. Both styles are nice. Another option is to use Apache Beam or Streaming Analytics Manager. https://community.hortonworks.com/content/kbentry/174105/hdp-264-hdf-31-apache-spark-structured-streaming-i.html
... View more
01-29-2018
04:58 PM
@balalaika By Default HDF install the NiFi service to run as the "nifi" user. You issue is likely that the nifi user is unable to navigate down that directory tree to the target script. Try becoming the nifi user (su - nifi) and navigating to the script directory. Thank you, Matt If this answer has addressed your question, please take a moment to click the "accept" link below the answer.
... View more
12-14-2017
03:58 PM
2 Kudos
@balalaika For that case you need to specify Demarcator property as Shift+enter Configs:- For merge content reference https://community.hortonworks.com/questions/149047/nifi-how-to-handle-with-mergecontent-processor.html
... View more
11-29-2017
04:45 PM
3 Kudos
@balalaika Use SplitJson Processor with the following configs:- Input Json:- {
"objectIdFieldName": "ID",
"objectIds": [
64916,
67266,
67237,
64511]
} Output:- will be different flowfiles per objectId 64916 67266 67237 64511 Use Extract Text processor:- extract the content of the flowfile by adding new property id as (.*) Right now you will have id attribute added to the flowfile then you can use the id attribute when you are doing InvokeHTTP processor Flow:- GetHTTP (get response JSON) --> SplitJson -->ExtractText--> InvokeHTTP (new query per ID)
... View more
11-01-2017
03:14 PM
Hi @Abdelkrim Hadjidj for now i will implement it with GetFTP, Nifi is in service provider network and i cannot upgrade at will 😞 Do you maybe know a way to tell GetFTP not to download files that have already been downloaded in the past to avoid unneccesary buffers?
... View more
10-12-2017
04:07 PM
@balalaika ^^
... View more
10-12-2017
02:25 PM
Hi @Alexandru Anghel, ive uploaded a new question with my whole use case and logic here. Any help really appreciated!
... View more