Support Questions
Find answers, ask questions, and share your expertise

Unzip file containing log.gz files with nifi

Explorer

Hello,

In Nifi I would like to unzip a file containing log.gz files.

I use a UnpackContent processor with zip value for Packaging Format field.

Unfortunatly it does not work and I encounter the following error :

2018-09-17 15:16:18,481 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.standard.UnpackContent UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4] Unable to unpack Stan
dardFlowFileRecord[uuid=9350821c-f4ac-4ad7-8cca-d32087a621c5,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1537197378339-415, container=default, section=415]
, offset=871972, length=55962],offset=0,name=backup-proxy-logs-2018-07-29-0001.zip,size=55962] due to org.apache.nifi.processor.exception.ProcessException: IOException thrown fro
m UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4]: org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: unsupported feature data descriptor used in en
try cloud_17338_20180729000000.log.gz; routing to failure: {}
org.apache.nifi.processor.exception.ProcessException: IOException thrown from UnpackContent[id=09876d75-09bc-1704-0000-0000220050c4]: org.apache.commons.compress.archivers.zip.Un
supportedZipFeatureException: unsupported feature data descriptor used in entry cloud_17338_20180729000000.log.gz
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2590)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker$1.process(UnpackContent.java:383)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2175)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2145)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker.unpack(UnpackContent.java:356)
        at org.apache.nifi.processors.standard.UnpackContent.onTrigger(UnpackContent.java:255)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.commons.compress.archivers.zip.UnsupportedZipFeatureException: unsupported feature data descriptor used in entry cloud_17338_20180729000000.log.gz
        at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(ZipArchiveInputStream.java:417)
        at java.io.InputStream.read(InputStream.java:101)
        at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
        at org.apache.nifi.processors.standard.UnpackContent$ZipUnpacker$1$1.process(UnpackContent.java:386)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2571)

If someone could help me I would appreciate a lot

Thanks in advacne

LR

3 REPLIES 3

Contributor

Anyone solved this? having same problem when trying to unpack a zip:

unsupported feature data descriptor used in entry /folder/file#1.gz

New Contributor

Hello!

You have solved it?

Contributor
I ended up executing unzip from execute process and then a ListFiles to get
the new files created by unzip command.