Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to unpack in nifi to preserve folder structure related information

Highlighted

How to unpack in nifi to preserve folder structure related information

I was using UnpackContent processor to unpack tar files decompressed from *.gz files. These files contain compressed tex files. It becomes critical for me to preserve the folder structure related information to later use them to parse the tex files properly as the tex files call each other with the folder prefixes.

The unpack Content processor doesn't give me any details about it in the attributes or content.

Lets take an example of data.tex that is packed in a subfolder "child" in a parent file 8808.gz. This file is kept in s3 on a prefix S3://PRE/DIST/. When I unpack I get:

absolute.path=/usr/local/Cellar/nifi/1.1.1/libexec/
path=null/

as nifi attributes

Am I not using the Unpack processor properly?

2 REPLIES 2
Highlighted

Re: How to unpack in nifi to preserve folder structure related information

What are you seeing in the "path" and "absolute.path" attributes on each flow file?

It looks like the code is attempting to set path to the parent of the entry from the tar, although there very well could be something that is not working as expected.

Highlighted

Re: How to unpack in nifi to preserve folder structure related information

Lets take an example of data.tex that is packed in a subfolder "child" in a parent file 8808.gz. This file is kept in s3 on a prefix `S3://PRE/DIST/`. When I unpack I get:

`absolute.path=/usr/local/Cellar/nifi/1.1.1/libexec/`

`path=null/`

as nifi attributes

Don't have an account?
Coming from Hortonworks? Activate your account here