Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Need to convert a hex file to another format using HDF

avatar

I have a couple of rather large hex files I need to convert to another format with the intention of then stripping out certain attributes and storing results in SQL Server and/or Hive. The files are 12.7 MB and 3.3 MB. When I use the code from this HCC answer https://community.hortonworks.com/questions/60597/hexdump-nifi-processor-nifi-hexdump-processor.html

import java.io.DataInputStream
def flowFile = session.get()
if(!flowFile) return
def attr = ''
session.read(flowFile, {inputStream ->
   dis = new DataInputStream(inputStream)
   attr = Long.toHexString(dis.readLong())
   attr2 = Long.toHexString(dis.readLong())
   
} as InputStreamCallback)
flowFile = session.putAttribute(flowFile, 'first16hex', attr+attr2)
session.transfer(flowFile, REL_SUCCESS)

in an ExecuteProcess I get a "File too large" error.

I'm also aware of this JIRA but am looking for a good workaround. https://issues.apache.org/jira/browse/NIFI-2997

1 ACCEPTED SOLUTION

avatar

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

View solution in original post

5 REPLIES 5

avatar
Master Guru

Can you share a stack trace / error log from logs/nifi-app.log? I'm curious to see what part of the code gives a "File too large" error.

avatar

I apologize. The actual error is "File name is too long". The name of the file is COWFILE1.DAT and COWFILE1.ARC.

9251-2016-11-08-15-01-04.jpg

avatar

Here is the repeating error in nifi-app.log.nifi-app.txt

avatar

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

avatar

Thanks @Andrew Grande! That worked! I feel like a noob 🙂 but appreciate all the help!