Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Need to convert a hex file to another format using HDF

Solved Go to solution

Need to convert a hex file to another format using HDF

I have a couple of rather large hex files I need to convert to another format with the intention of then stripping out certain attributes and storing results in SQL Server and/or Hive. The files are 12.7 MB and 3.3 MB. When I use the code from this HCC answer https://community.hortonworks.com/questions/60597/hexdump-nifi-processor-nifi-hexdump-processor.html

import java.io.DataInputStream
def flowFile = session.get()
if(!flowFile) return
def attr = ''
session.read(flowFile, {inputStream ->
   dis = new DataInputStream(inputStream)
   attr = Long.toHexString(dis.readLong())
   attr2 = Long.toHexString(dis.readLong())
   
} as InputStreamCallback)
flowFile = session.putAttribute(flowFile, 'first16hex', attr+attr2)
session.transfer(flowFile, REL_SUCCESS)

in an ExecuteProcess I get a "File too large" error.

I'm also aware of this JIRA but am looking for a good workaround. https://issues.apache.org/jira/browse/NIFI-2997

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Need to convert a hex file to another format using HDF

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

5 REPLIES 5

Re: Need to convert a hex file to another format using HDF

Super Guru

Can you share a stack trace / error log from logs/nifi-app.log? I'm curious to see what part of the code gives a "File too large" error.

Re: Need to convert a hex file to another format using HDF

I apologize. The actual error is "File name is too long". The name of the file is COWFILE1.DAT and COWFILE1.ARC.

9251-2016-11-08-15-01-04.jpg

Re: Need to convert a hex file to another format using HDF

Here is the repeating error in nifi-app.log.nifi-app.txt

Re: Need to convert a hex file to another format using HDF

Scott,

Use ExecuteScript processor, as mentioned in the article. Looks like you are putting an embedded script into ExecuteProcess, which is meant to invoke OS shell commands, hence the error.

Highlighted

Re: Need to convert a hex file to another format using HDF

Thanks @Andrew Grande! That worked! I feel like a noob :-) but appreciate all the help!

Don't have an account?
Coming from Hortonworks? Activate your account here