Support Questions

Find answers, ask questions, and share your expertise

Need to convert a hex file to another format using HDF

I have a couple of rather large hex files I need to convert to another format with the intention of then stripping out certain attributes and storing results in SQL Server and/or Hive. The files are 12.7 MB and 3.3 MB. When I use the code from this HCC answer https://community.hortonworks.com/questions/60597/hexdump-nifi-processor-nifi-hexdump-processor.html

import java.io.DataInputStream
def flowFile = session.get()
if(!flowFile) return
def attr = ''
session.read(flowFile, {inputStream ->
   dis = new DataInputStream(inputStream)
   attr = Long.toHexString(dis.readLong())
   attr2 = Long.toHexString(dis.readLong())
   
} as InputStreamCallback)
flowFile = session.putAttribute(flowFile, 'first16hex', attr+attr2)
session.transfer(flowFile, REL_SUCCESS)

in an ExecuteProcess I get a "File too large" error.

I'm also aware of this JIRA but am looking for a good workaround. https://issues.apache.org/jira/browse/NIFI-2997

1 ACCEPTED SOLUTION

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

View solution in original post

5 REPLIES 5

Super Guru

Can you share a stack trace / error log from logs/nifi-app.log? I'm curious to see what part of the code gives a "File too large" error.

I apologize. The actual error is "File name is too long". The name of the file is COWFILE1.DAT and COWFILE1.ARC.

9251-2016-11-08-15-01-04.jpg

Here is the repeating error in nifi-app.log.nifi-app.txt

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

Thanks @Andrew Grande! That worked! I feel like a noob 🙂 but appreciate all the help!

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.