Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Unzip file containing log.gz files with nifi

avatar
Contributor

Hello,

In Nifi I would like to unzip a file containing log.gz files.

I use a UnpackContent processor with zip value for Packaging Format field.

Unfortunatly it does not work and I encounter the following error :

2018-09-17 15:16:18,481 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.standard.UnpackContent UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4] Unable to unpack Stan
dardFlowFileRecord[uuid=9350821c-f4ac-4ad7-8cca-d32087a621c5,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1537197378339-415, container=default, section=415]
, offset=871972, length=55962],offset=0,name=backup-proxy-logs-2018-07-29-0001.zip,size=55962] due to org.apache.nifi.processor.exception.ProcessException: IOException thrown fro
m UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4]: org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: unsupported feature data descriptor used in en
try cloud_17338_20180729000000.log.gz; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: IOException thrown from UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4]: org.apache.commons.compress.archivers.zip.Un
supportedZipFeatureException: unsupported feature data descriptor used in entry cloud_17338_20180729000000.log.gz
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2590)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker$1.process(UnpackContent.java:383)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2175)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2145)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker.unpack(UnpackContent.java:356)
        at org.apache.nifi.processors.standard.UnpackContent.onTrigger(UnpackContent.java:255)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: unsupported feature data descriptor used in entry cloud_17338_20180729000000.log.gz
        at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(ZipArchiveInputStream.java:417)
        at java.io.InputStream.read(InputStream.java:101)
        at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker$1$1.process(UnpackContent.java:386)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571)

If someone could help me I would appreciate a lot

Thanks in advacne

LR

3 REPLIES 3

avatar
Contributor

Anyone solved this? having same problem when trying to unpack a zip:

unsupported feature data descriptor used in entry /folder/file#1.gz

avatar
New Contributor

Hello!

You have solved it?

avatar
Contributor
I ended up executing unzip from execute process and then a ListFiles to get
the new files created by unzip command.