Skip to main content
knarayanan
Gold ⭐️
Gold ⭐️
Sep 26, 2017 11:55 AM UTC

Using NiFi and Pdfbox to extract images from PDF

  • Oct 5, 2026 9:15 AM UTC
  • 3 replies
  • 15685 views
  • Translate All

In this article we will go over how we can use nifi to ingest PDFs and while we ingest we will use a custom groovy script with executescript processor to extract images from this PDF. The images will be tagged with PDF filename, page num and imagenum , so that it can be indexed in hbase/solr for quick searching and doing some machine learning or other analytics.

The Groovy code below depends on the following jars. Download and copy them to a folder on your computer. I my case they were under /var/pdimagelib/.

  • pdfbox-2.0.7.jar
  • fontbox-2.0.7.jar
  • jai_imageio.jar
  • commons-logging-1.1.1.jar

Copy the code below to a file on your computer, in my case the file was under /var/scripts/pdimage.groovy

import java.nio.charset.*;

import org.apache.commons.io.IOUtils;

import java.awt.image.BufferedImage;

import java.awt.image.RenderedImage;

import java.io.File;

import java.io.FileOutputStream;

import java.util.Iterator;

import java.util.List;

import java.util.ArrayList;

import java.util.Map;

import javax.imageio.ImageIO;

import org.apache.pdfbox.cos.COSName;

import org.apache.pdfbox.pdmodel.PDDocument;

import org.apache.pdfbox.pdmodel.PDPage;

import org.apache.pdfbox.pdmodel.PDPageTree;

import org.apache.pdfbox.pdmodel.PDResources;

import org.apache.pdfbox.pdmodel.graphics.PDXObject;

import org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject;



def flowFile = session.get();

if (flowFile == null) {

return;

}

def ffList = new ArrayList<FlowFile>()

try {

session.read(flowFile,

{inputStream ->

PDDocument document = PDDocument.load(inputStream);

int imageNum = 1;

int pageNum = 1;

document.getDocumentCatalog().getPages().each(){page ->

PDResources pdResources = page.getResources();

pdResources.getXObjectNames().each(){ cosName->

if (cosName != null) {

PDXObject image =pdResources.getXObject(cosName);

if(image instanceof PDImageXObject){

PDImageXObject pdImage = (PDImageXObject)image;

BufferedImage imageStream = pdImage.getImage();

imgFF = session.create(flowFile);

outputStream = session.write(imgFF,

{ outputStream->

ImageIO.write((RenderedImage)imageStream, "png", outputStream);

} as OutputStreamCallback)

imgFF = session.putAttribute(imgFF, "imageNum",String.valueOf(imageNum));

imgFF = session.putAttribute(imgFF, "pageNum",String.valueOf(pageNum));

ffList.add(imgFF);

imageNum++;

}

}

}

pageNum++;



}

} as InputStreamCallback)

session.transfer(ffList, ExecuteScript.REL_SUCCESS)

session.remove(flowFile);

} catch (Exception e) {

log.warn(e.getMessage());

e.printStackTrace();

session.remove(ffList);

session.transfer(flowFile, ExecuteScript.REL_FAILURE);

}

Below is a screenshot of executescript after it has been setup correctly.

39539-screen-shot-2017-09-26-at-124404-am.png

To ingest the PDF, i used a simple GetFile, though this approach should work for pdfs ingested with any other nifi processor. Below is a snapshot of the nifi flow. When a PDF is ingested, executescript will leverage groovy and pdfbox to extract images. The images will be tagged with the pdf filename,pagenum and imagenum. This can now be sent to hbase or any indexing solution for search or analytics.

39540-screen-shot-2017-09-26-at-124334-am.png

Hope you find the article useful. Please comment withy your thoughts/questions.

3 replies

TimothySpann
Platinum ⭐️⭐️⭐️
Platinum ⭐️⭐️⭐️
Sep 26, 2017 10:54 PM UTC

sounds like a good custom processor

Translate
knarayanan
Gold ⭐️
Gold ⭐️
Dec 20, 2017 2:44 PM UTC