Member since
11-16-2015
905
Posts
665
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 425 | 09-30-2025 05:23 AM | |
| 756 | 06-26-2025 01:21 PM | |
| 650 | 06-19-2025 02:48 PM | |
| 844 | 05-30-2025 01:53 PM | |
| 11362 | 02-22-2024 12:38 PM |
10-20-2016
02:53 PM
If your Python script is just calling out to the operating system, consider using ExecuteStreamCommand or ExecuteProcess instead. Otherwise there are (hopefully!) some pure Python module(s) for doing HTTP requests, rather than using curl via os.system().
... View more
10-15-2016
06:10 PM
Try building with the -Phortonworks profile, or adding http://repo.hortonworks.com/content/repositories/releases/ to the list of repositories in the POM
... View more
10-14-2016
07:06 PM
1 Kudo
You are running into NIFI-2873, this will be fixed in an upcoming version.
... View more
10-14-2016
06:31 PM
You'd need multiple calls to dis.readXYZ(), call toHexString() on each, then concatenate before storing in the "attr" or whatever value will be the result (going into the first16hex attribute). For 36 Hex characters, it's probably two dis.readLongs() followed by a dis.readUnsignedShort().
... View more
10-13-2016
01:58 PM
As of NiFi 1.0.0 (HDF 2.0.0), there is no way to directly evaluate Avro fields, instead you might use ConvertAvroToJSON and apply the same approach as described in your other question. If/When NIFI-962 is implemented, then such operations might be available directly on the Avro file.
... View more
10-10-2016
05:02 PM
Where are you getting your JSON from? If it is a static value, you could use GenerateFlowFile -> ReplaceText to set the value for the body, if it comes from a file you can use GetFile or ListFile -> FetchFile. If the mime.type attribute is not already set for the flow file (containing the JSON response), you can use UpdateAttribute to set mime.type to "application/json", and your auth key (called maybe "auth.key") to your key. Then in InvokeHttp you can list mime.type,auth.key in the "Attributes to send" property, and make sure "Send Message Body" is true. These steps will ensure that the headers are sent and the JSON content from the flow file is sent as the body.
... View more
10-08-2016
05:40 PM
2 Kudos
You could use the ExecuteScript processor if you are comfortable with Groovy, Javascript, Jython, JRuby, or Lua. Here's an example of a Groovy script that I think will do what you're asking: import java.io.DataInputStream
def flowFile = session.get()
if(!flowFile) return
def attr = ''
session.read(flowFile, {inputStream ->
dis = new DataInputStream(inputStream)
attr = Integer.toHexString(dis.readUnsignedShort())
} as InputStreamCallback)
flowFile = session.putAttribute(flowFile, 'first16hex', attr)
session.transfer(flowFile, REL_SUCCESS) This maintains the content in the flow file but adds an attribute called 'first16hex' that contains a string representation of the first 16 bits of the incoming flow file content. Please let me know if I've misunderstood anything here, and I will try to help. I should mention that a full hexdump processor could be helpful, feel free to raise a Jira for this feature.
... View more
10-05-2016
01:00 PM
1 Kudo
At the time of this writing (after HDP 2.5 was released), the sandbox download is still incorrectly named OVF. Changing to OVA still works.
... View more
10-04-2016
01:08 PM
1 Kudo
Great answer! Just to add a caveat, if you are using HDF 2.0 and HDP 2.5, please see the following: https://community.hortonworks.com/questions/59681/puthivestreaming-nifi-processor-various-errors.html
... View more
10-04-2016
12:36 PM
7 Kudos
The issue for Hive Streaming between HDF 2.0 and HDP 2.5 is captured as NIFI-2828 (albeit under a different title, it is the same cause and fix). In the meantime as a possible workaround I have built a Hive NAR that you can try if you wish, just save off your other one (from the lib/ folder with a version like 1.0.0.2.0.0-159 or something) and replace it with this one.
... View more