Community Articles

TimothySpann · ‎12-19-2016

Drones / unmanned aerial vehicle (UAV) Ingest

We have several options on ingesting data from Drones depending on the model, location, networking and needs.

MQTT is a popular option if you are on the same WIFI or network. See my article on MQTT https://community.hortonworks.com/articles/72420/ingesting-remote-sensor-feeds-into-apache-phoenix.h.... Many UAVs will let you push up to a cloud MQTT server and asynchronously consume this data from your enterprise HDF system.

They can also be access via TCP, UDP, REST and offline download.

For my example I am parsing a directory full of downloaded BEBOP Parrot 2 Drone images.

NIFI with Four Parallel Flows from the Ingested File

Step 1: GetFile from directory of images.

Step 2: PutHDFS Store FlowFile Raw images to HDFS.

Step 3: Update Attribute add an attribute (API KEY) to pass to Microsoft Cognitive Services.

Step 4: ReplaceText to convert body to a JSON message with image

Step 5: InvokeHTTP to call https://api.projectoxford.ai/vision/v1.0/describe?maxCandidates=1

Step 6: UpdateAttribute to change the filename for saving the Microsoft results ${filename:replace('jpg','json')}

Step 7: PutHDFS to store the JSON file in Hadoop

Step 8: ExtractImageMetadata to pull out attribute values

Step 9: PutHDFS to store JSON metadata file in Hadoop

Step 10: AttributesToJSON to convert all the attributes into one JSON file (so we are replacing the image file with JSON)

Step 11: UpdateAttribute to change the filename for saving the Microsoft results ${filename:replace('jpg','json')}

Step 12: PutHDFS to store JSON metadata file in Hadoop

Step 14: ExtraMediaMetadata to pull out attributes from image using Apache Tika.

Step 15: PutHDFS to store JSON metadata file in Hadoop

Step 16: ResizeImage to resize the original image to a 250x250 pixel image.

Step 17: PutHDFS to store the resized image in Hadoop

To gain insight on these images, we can extract a large amount of metadata using HDF's ExtractMediaMetadata and ExtractImageMetadata.

Example Data Extracted From Drone Image

Aperture Value
F2.3
Color Space
Undefined
Component 1
Y component: Quantization table 0, Sampling factors 2 horiz/2 vert
Component 2
Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert
Component 3
Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert
Components Configuration
YCbCr
Compression
JPEG
Compression Type
Baseline
Content-Type
image/jpeg
Creation-Date
2016-09-20T08:42:30
Data Precision
8 bits
date
2016-09-20T08:42:30
Date/Time
2016:09:20 08:42:30
Date/Time Digitized
2016:09:20 08:42:30
Date/Time Original
2016:09:20 08:42:30
dc:description
{ "product_id": "090C", "uuid": "CE112B4276E13339AFBAE10E9ED794E3", "run_date": "2016-09-20T083537-0400", "filename": "Bebop_2_2016-09-20T084230-0400_CE112B.jpg", "media_date": "2016-09-20T084230-0400" }
dcterms:created
2016-09-20T08:42:30
dcterms:modified
2016-09-20T08:42:30
description
{ "product_id": "090C", "uuid": "CE112B4276E13339AFBAE10E9ED794E3", "run_date": "2016-09-20T083537-0400", "filename": "Bebop_2_2016-09-20T084230-0400_CE112B.jpg", "media_date": "2016-09-20T084230-0400" }
exif:DateTimeOriginal
2016-09-20T08:42:30
exif:ExposureTime
0.004004237243907637
exif:FNumber
2.2999999166745724
exif:FocalLength
1.8300001180412324
exif:IsoSpeedRatings
613
Exif Image Height
1088 pixels
Exif Image Width
1920 pixels
Exif Version
2.10
Exposure Mode
Auto exposure
Exposure Program
Program action (high-speed program)
Exposure Time
8599034/2147483647 sec
file.creationTime
2016-09-23T17:56:23+0000
file.group
root
file.lastAccessTime
2016-09-23T17:56:24+0000
file.lastModifiedTime
2016-09-23T17:56:23+0000
file.owner
root
file.permissions
rw-r--r--
File Modified Date
Fri Sep 23 17:56:24 UTC 2016
filename
Bebop2_20160920084230-0400.json
Bebop2_20160920084230-0400.jpg (previous)
File Name
apache-tika-4338169443739021156.tmp
File Size
482262 bytes
FlashPix Version
1.00
F-Number
F2.3
Focal Length
1.83 mm
Focal Length 35
6mm
geo:lat
40.754615
geo:long
-73.080665
GPS Altitude
23 metres
GPS Altitude Ref
Sea level
GPS Latitude
40° 45' 16.62"
GPS Latitude Ref
N
GPS Longitude
-73° 4' 50.4"
GPS Longitude Ref
W
GPS Version ID
2.200
Image Description
{ "product_id": "090C", "uuid": "CE112B4276E13339AFBAE10E9ED794E3", "run_date": "2016-09-20T083537-0400", "filename": "Bebop_2_2016-09-20T084230-0400_CE112B.jpg", "media_date": "2016-09-20T084230-0400" }
Image Height
1088 pixels
Image Width
1920 pixels
ISO Speed Ratings
613
Last-Modified
2016-09-20T08:42:30
Last-Save-Date
2016-09-20T08:42:30
Make
PARROT
Max Aperture Value
F2.3
meta:creation-date
2016-09-20T08:42:30
meta:save-date
2016-09-20T08:42:30
Metering Mode
(Other)
mime.type
application/json
Model
Bebop 2
modified
2016-09-20T08:42:30
Number of Components
3
Orientation
Top, left side (Horizontal / normal)
path
/
Photometric Interpretation
YCbCr
Resolution Unit
Inch
Shutter Speed Value
1/249 sec
Software
Dragon 3.9.0
tiff:BitsPerSample
8
tiff:ImageLength
1088
tiff:ImageWidth
1920
tiff:Make
PARROT
tiff:Model
Bebop 2
tiff:Orientation
1
tiff:ResolutionUnit
Inch
tiff:Software
Dragon 3.9.0
tiff:XResolution
72.0
tiff:YResolution
72.0
Unique Image ID
[32 bytes]
Unknown tag (0xc62f)
[19 bytes]
uuid
63c748ad-7b4d-412b-ab35-9ae3152d5aa0
White Balance
Flash
White Balance Mode
Auto white balance
XMP Value Count
9
X-Parsed-By
org.apache.tika.parser.DefaultParser, org.apache.tika.parser.jpeg.JpegParser
X Resolution
72 dots per inch
YCbCr Positioning
Center of pixel array
Y Resolution
72 dots per inch

Hive DDL for NOAA Weather Data

CREATE EXTERNAL TABLE current_observation(
version STRING,
xsinoNamespaceSchemaLocation STRING,
credit STRING,
credit_URL STRING,
suggested_pickup STRING,
suggested_pickup_period STRING,
location STRING,
station_id STRING,
latitude STRING,
longitude STRING,
observation_time STRING,
observation_time_rfc822 STRING,
weather STRING,
temperature_string STRING,
temp_f STRING,
temp_c STRING,
relative_humidity STRING,
wind_string STRING,
wind_dir STRING,
wind_degrees STRING,
wind_mph STRING,
wind_kt STRING,
pressure_string STRING,
pressure_mb STRING,
pressure_in STRING,
dewpoint_string STRING,
dewpoint_f STRING,
dewpoint_c STRING,
icon_url_base STRING,
two_day_history_url STRING,
icon_url_name STRING,
ob_url STRING,
disclaimer_url STRING,
copyright_url STRING,
privacy_policy_url STRING)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
LOCATION 'hdfs://xxxx.field.hortonworks.com:8020/drone/weathercurrent';

Create Drone Data Table in Apache Phoenix

CREATE TABLE dronedata1 (datekey varchar not null primary key,
compression varchar,
exifVersion varchar,
componentsConfiguration varchar,
fileGroup varchar,
compressionType varchar,
imageDescription varchar,
numberOfComponents varchar,
component2 varchar,
focalLength varchar,
component1 varchar,
yCbCrPositioning varchar,
tiffResolutionUnit varchar,
uuid varchar,
dateTimeOriginal varchar,
shutterSpeedValue varchar,
xResolution varchar,
tiffMake varchar,
path varchar,
photometricInterpretation varchar,
component3 varchar,
uniqueImageID varchar,
fNumber varchar,
modified varchar,
focalLength35 varchar,
tiffBitsPerSample varchar,
exposureProgram varchar,
gPSVersionID varchar,
gPSLatitudeRef varchar,
metacreationdate varchar,
exifFNumber varchar,
gPSAltitudeRef varchar,
exposureTime varchar,
gPSLongitude varchar,
creationDate varchar,
iSOSpeedRatings varchar,
make varchar,
orientation varchar,
meteringMode varchar,
tiffOrientation varchar,
gPSLongitudeRef varchar,
tiffSoftware varchar,
exifFocalLength varchar,
xMPValueCount varchar,
geolong varchar,
fileOwner varchar,
software varchar,
exifImageHeight varchar,
tiffYResolution varchar,
yResolution varchar,
gPSLatitude varchar,
dcDescription varchar,
geolat varchar,
flashPixVersion varchar,
dataPrecision varchar,
whiteBalance varchar,
tiffImageLength varchar,
description varchar,
dcTermsCreated varchar,
dcTermsModified varchar,
lastModified varchar,
filePermissions varchar,
exifExposureTime varchar,
lastSaveDate varchar,
gPSAltitude varchar,
absolutePath varchar,
colorSpace varchar,
fileSize varchar,
metaSaveDate varchar,
fileCreationTime varchar,
dateTimeDigitized varchar,
fileName varchar,
contentType varchar,
apertureValue varchar,
zParsedBy varchar,
dileModifiedDate varchar,
tiffXResolution varchar,
filelastModifiedTime varchar,
exifDateTimeOriginal varchar,
sateTime varchar,
wxifImageWidth varchar,
imageHeight varchar,
imageWidth varchar,
unknownTag varchar,
resolutionUnit varchar,
tiffModel varchar,
exifIsoSpeedRatings varchar,
maxApertureValue varchar,
exposureMode varchar,
model varchar,
fileLastAccessTime varchar,
tiffImageWidth varchar,
whiteBalanceMode varchar);

PutSQL for Phoenix

upsert into dronedata1 (datekey, inception, fileName, gPSAltitude, gPSLatitude, gPSLongitude, orientation,geolat,geolong) 
values ('${'date'}','${'inception'}','${'filename'}','${'GPS Altitude'}','${'GPS Latitude':replaceAll("'", "")}','${'GPS Longitude':replaceAll("'",  "")}','${'Orientation'}', '${'geo:lat'}','${'geo:long'}')

Viewing Drone Data via Microservice

Java Class To Display Bebop Parrot Drone Data

package com.dataflowdeveloper;

import java.io.Serializable;

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;

/**
 * 
 * @author tspann
 *
 */
@JsonInclude(JsonInclude.Include.NON_NULL)
public class DroneDataFull implements Serializable {

 private static final long serialVersionUID = 8198045227677015331L;
 
@JsonProperty("date")
private String date;
@JsonProperty("Compression")
private String compression;
@JsonProperty("Exif Version")
private String exifVersion;
@JsonProperty("Components Configuration")
private String componentsConfiguration;
@JsonProperty("file.group")
private String fileGroup;
@JsonProperty("Compression Type")
private String compressionType;
@JsonProperty("Image Description")
private String imageDescription;
@JsonProperty("Number of Components")
private String numberOfComponents;
@JsonProperty("Component 2")
private String component2;
@JsonProperty("Focal Length")
private String focalLength;
@JsonProperty("Component 1")
private String component1;
@JsonProperty("YCbCr Positioning")
private String yCbCrPositioning;
@JsonProperty("tiff:ResolutionUnit")
private String tiffResolutionUnit;
@JsonProperty("uuid")
private String uuid;
@JsonProperty("Date/Time Original")
private String dateTimeOriginal;
@JsonProperty("Shutter Speed Value")
private String shutterSpeedValue;
@JsonProperty("X Resolution")
private String xResolution;
@JsonProperty("tiff:Make")
private String tiffMake;
@JsonProperty("path")
private String path;
@JsonProperty("Photometric Interpretation")
private String photometricInterpretation;
@JsonProperty("Component 3")
private String component3;
@JsonProperty("Unique Image ID")
private String uniqueImageID;
@JsonProperty("F-Number")
private String fNumber;
@JsonProperty("modified")
private String modified;
@JsonProperty("Focal Length 35")
private String focalLength35;
@JsonProperty("tiff:BitsPerSample")
private String tiffBitsPerSample;
@JsonProperty("Exposure Program")
private String exposureProgram;
@JsonProperty("GPS Version ID")
private String gPSVersionID;
@JsonProperty("GPS Latitude Ref")
private String gPSLatitudeRef;
@JsonProperty("meta:creation-date")
private String metaCreationDate;
@JsonProperty("exif:FNumber")
private String exifFNumber;
@JsonProperty("GPS Altitude Ref")
private String gPSAltitudeRef;
@JsonProperty("Exposure Time")
private String exposureTime;
@JsonProperty("GPS Longitude")
private String gPSLongitude;
@JsonProperty("Creation-Date")
private String creationDate;
@JsonProperty("ISO Speed Ratings")
private String iSOSpeedRatings;
@JsonProperty("Make")
private String make;
@JsonProperty("Orientation")
private String orientation;
@JsonProperty("Metering Mode")
private String meteringMode;
@JsonProperty("tiff:Orientation")
private String tiffOrientation;
@JsonProperty("GPS Longitude Ref")
private String gPSLongitudeRef;
@JsonProperty("tiff:Software")
private String tiffSoftware;
@JsonProperty("exif:FocalLength")
private String exifFocalLength;
@JsonProperty("filename")
private String filename;
@JsonProperty("XMP Value Count")
private String xMPValueCount;
@JsonProperty("geo:long")
private String geoLong;
@JsonProperty("file.owner")
private String fileOwner;
@JsonProperty("Software")
private String software;
@JsonProperty("Exif Image Height")
private String exifImageHeight;
@JsonProperty("tiff:YResolution")
private String tiffYResolution;
@JsonProperty("Y Resolution")
private String yResolution;
@JsonProperty("GPS Latitude")
private String gPSLatitude;
@JsonProperty("dc:description")
private String dcDescription;
@JsonProperty("geo:lat")
private String geoLat;
@JsonProperty("FlashPix Version")
private String flashPixVersion;
@JsonProperty("Data Precision")
private String dataPrecision;
@JsonProperty("White Balance")
private String whiteBalance;
@JsonProperty("tiff:ImageLength")
private String tiffImageLength;
@JsonProperty("description")
private String description;
@JsonProperty("dcterms:created")
private String dctermsCreated;
@JsonProperty("dcterms:modified")
private String dctermsModified;
@JsonProperty("Last-Modified")
private String lastModified;
@JsonProperty("file.permissions")
private String filePermissions;
@JsonProperty("exif:ExposureTime")
private String exifExposureTime;
@JsonProperty("Last-Save-Date")
private String lastSaveDate;
@JsonProperty("GPS Altitude")
private String gPSAltitude;
@JsonProperty("absolute.path")
private String absolutePath;
@JsonProperty("Color Space")
private String colorSpace;
@JsonProperty("File Size")
private String fileSize;
@JsonProperty("meta:save-date")
private String metaSaveDate;
@JsonProperty("file.creationTime")
private String fileCreationTime;
@JsonProperty("Date/Time Digitized")
private String dateTimeDigitized;
@JsonProperty("File Name")
private String fileName;
@JsonProperty("Content-Type")
private String contentType;
@JsonProperty("Aperture Value")
private String apertureValue;
@JsonProperty("X-Parsed-By")
private String xParsedBy;
@JsonProperty("File Modified Date")
private String fileModifiedDate;
@JsonProperty("tiff:XResolution")
private String tiffXResolution;
@JsonProperty("file.lastModifiedTime")
private String fileLastModifiedTime;
@JsonProperty("exif:DateTimeOriginal")
private String exifDateTimeOriginal;
@JsonProperty("Date/Time")
private String dateTime;
@JsonProperty("Exif Image Width")
private String exifImageWidth;
@JsonProperty("Image Height")
private String imageHeight;
@JsonProperty("Image Width")
private String imageWidth;
@JsonProperty("Unknown tag (0xc62f)")
private String unknownTag0xc62f;
@JsonProperty("Resolution Unit")
private String resolutionUnit;
@JsonProperty("tiff:Model")
private String tiffModel;
@JsonProperty("exif:IsoSpeedRatings")
private String exifIsoSpeedRatings;
@JsonProperty("Max Aperture Value")
private String maxApertureValue;
@JsonProperty("Exposure Mode")
private String exposureMode;
@JsonProperty("Model")
private String model;
@JsonProperty("file.lastAccessTime")
private String fileLastAccessTime;
@JsonProperty("tiff:ImageWidth")
private String tiffImageWidth;
@JsonProperty("White Balance Mode")
private String whiteBalanceMode;
@JsonProperty("inception")
private String inception;


public String getInception() {
 return inception;
}

public void setInception(String inception) {
 this.inception = inception;
}

/**
* No args constructor for use in serialization
* 
*/
public DroneDataFull() {
 super();
}

/**
* 
* @param tiffImageLength
* @param maxApertureValue
* @param exposureTime
* @param apertureValue
* @param xMPValueCount
* @param tiffSoftware
* @param exifFocalLength
* @param contentType
* @param shutterSpeedValue
* @param dataPrecision
* @param software
* @param yCbCrPositioning
* @param whiteBalanceMode
* @param tiffImageWidth
* @param description
* @param dctermsCreated
* @param fileLastModifiedTime
* @param compression
* @param exifFNumber
* @param exposureProgram
* @param compressionType
* @param iSOSpeedRatings
* @param filePermissions
* @param gPSAltitudeRef
* @param gPSVersionID
* @param orientation
* @param fileGroup
* @param exifImageHeight
* @param fileOwner
* @param flashPixVersion
* @param tiffBitsPerSample
* @param fileCreationTime
* @param gPSLatitude
* @param modified
* @param focalLength
* @param resolutionUnit
* @param exposureMode
* @param exifImageWidth
* @param xParsedBy
* @param dateTimeDigitized
* @param dcDescription
* @param fileName
* @param fileModifiedDate
* @param geoLong
* @param uuid
* @param make
* @param imageWidth
* @param exifVersion
* @param tiffXResolution
* @param model
* @param fileLastAccessTime
* @param dateTime
* @param exifDateTimeOriginal
* @param fNumber
* @param yResolution
* @param uniqueImageID
* @param tiffModel
* @param exifExposureTime
* @param whiteBalance
* @param unknownTag0xc62f
* @param date
* @param gPSLongitude
* @param gPSAltitude
* @param dctermsModified
* @param fileSize
* @param gPSLatitudeRef
* @param tiffYResolution
* @param numberOfComponents
* @param path
* @param xResolution
* @param tiffMake
* @param meteringMode
* @param tiffResolutionUnit
* @param photometricInterpretation
* @param focalLength35
* @param imageDescription
* @param imageHeight
* @param lastModified
* @param component3
* @param component1
* @param component2
* @param gPSLongitudeRef
* @param lastSaveDate
* @param componentsConfiguration
* @param absolutePath
* @param exifIsoSpeedRatings
* @param creationDate
* @param tiffOrientation
* @param geoLat
* @param metaSaveDate
* @param colorSpace
* @param filename
* @param dateTimeOriginal
* @param metaCreationDate
*/
public DroneDataFull(String date, String compression, String exifVersion, String componentsConfiguration, String fileGroup, String compressionType, String imageDescription, String numberOfComponents, String component2, String focalLength, String component1, String yCbCrPositioning, String tiffResolutionUnit, String uuid, String dateTimeOriginal, String shutterSpeedValue, String xResolution, String tiffMake, String path, String photometricInterpretation, String component3, String uniqueImageID, String fNumber, String modified, String focalLength35, String tiffBitsPerSample, String exposureProgram, String gPSVersionID, String gPSLatitudeRef, String metaCreationDate, String exifFNumber, String gPSAltitudeRef, String exposureTime, String gPSLongitude, String creationDate, String iSOSpeedRatings, String make, String orientation, String meteringMode, String tiffOrientation, String gPSLongitudeRef, String tiffSoftware, String exifFocalLength, String filename, String xMPValueCount, String geoLong, String fileOwner, String software, String exifImageHeight, String tiffYResolution, String yResolution, String gPSLatitude, String dcDescription, String geoLat, String flashPixVersion, String dataPrecision, String whiteBalance, String tiffImageLength, String description, String dctermsCreated, String dctermsModified, String lastModified, String filePermissions, String exifExposureTime, String lastSaveDate, String gPSAltitude, String absolutePath, String colorSpace, String fileSize, String metaSaveDate, String fileCreationTime, String dateTimeDigitized, String fileName, String contentType, String apertureValue, String xParsedBy, String fileModifiedDate, String tiffXResolution, String fileLastModifiedTime, String exifDateTimeOriginal, String dateTime, String exifImageWidth, String imageHeight, String imageWidth, String unknownTag0xc62f, String resolutionUnit, String tiffModel, String exifIsoSpeedRatings, String maxApertureValue, String exposureMode, String model, String fileLastAccessTime, String tiffImageWidth, String whiteBalanceMode) {
super();
this.date = date;
this.compression = compression;
this.exifVersion = exifVersion;
this.componentsConfiguration = componentsConfiguration;
this.fileGroup = fileGroup;
this.compressionType = compressionType;
this.imageDescription = imageDescription;
this.numberOfComponents = numberOfComponents;
this.component2 = component2;
this.focalLength = focalLength;
this.component1 = component1;
this.yCbCrPositioning = yCbCrPositioning;
this.tiffResolutionUnit = tiffResolutionUnit;
this.uuid = uuid;
this.dateTimeOriginal = dateTimeOriginal;
this.shutterSpeedValue = shutterSpeedValue;
this.xResolution = xResolution;
this.tiffMake = tiffMake;
this.path = path;
this.photometricInterpretation = photometricInterpretation;
this.component3 = component3;
this.uniqueImageID = uniqueImageID;
this.fNumber = fNumber;
this.modified = modified;
this.focalLength35 = focalLength35;
this.tiffBitsPerSample = tiffBitsPerSample;
this.exposureProgram = exposureProgram;
this.gPSVersionID = gPSVersionID;
this.gPSLatitudeRef = gPSLatitudeRef;
this.metaCreationDate = metaCreationDate;
this.exifFNumber = exifFNumber;
this.gPSAltitudeRef = gPSAltitudeRef;
this.exposureTime = exposureTime;
this.gPSLongitude = gPSLongitude;
this.creationDate = creationDate;
this.iSOSpeedRatings = iSOSpeedRatings;
this.make = make;
this.orientation = orientation;
this.meteringMode = meteringMode;
this.tiffOrientation = tiffOrientation;
this.gPSLongitudeRef = gPSLongitudeRef;
this.tiffSoftware = tiffSoftware;
this.exifFocalLength = exifFocalLength;
this.filename = filename;
this.xMPValueCount = xMPValueCount;
this.geoLong = geoLong;
this.fileOwner = fileOwner;
this.software = software;
this.exifImageHeight = exifImageHeight;
this.tiffYResolution = tiffYResolution;
this.yResolution = yResolution;
this.gPSLatitude = gPSLatitude;
this.dcDescription = dcDescription;
this.geoLat = geoLat;
this.flashPixVersion = flashPixVersion;
this.dataPrecision = dataPrecision;
this.whiteBalance = whiteBalance;
this.tiffImageLength = tiffImageLength;
this.description = description;
this.dctermsCreated = dctermsCreated;
this.dctermsModified = dctermsModified;
this.lastModified = lastModified;
this.filePermissions = filePermissions;
this.exifExposureTime = exifExposureTime;
this.lastSaveDate = lastSaveDate;
this.gPSAltitude = gPSAltitude;
this.absolutePath = absolutePath;
this.colorSpace = colorSpace;
this.fileSize = fileSize;
this.metaSaveDate = metaSaveDate;
this.fileCreationTime = fileCreationTime;
this.dateTimeDigitized = dateTimeDigitized;
this.fileName = fileName;
this.contentType = contentType;
this.apertureValue = apertureValue;
this.xParsedBy = xParsedBy;
this.fileModifiedDate = fileModifiedDate;
this.tiffXResolution = tiffXResolution;
this.fileLastModifiedTime = fileLastModifiedTime;
this.exifDateTimeOriginal = exifDateTimeOriginal;
this.dateTime = dateTime;
this.exifImageWidth = exifImageWidth;
this.imageHeight = imageHeight;
this.imageWidth = imageWidth;
this.unknownTag0xc62f = unknownTag0xc62f;
this.resolutionUnit = resolutionUnit;
this.tiffModel = tiffModel;
this.exifIsoSpeedRatings = exifIsoSpeedRatings;
this.maxApertureValue = maxApertureValue;
this.exposureMode = exposureMode;
this.model = model;
this.fileLastAccessTime = fileLastAccessTime;
this.tiffImageWidth = tiffImageWidth;
this.whiteBalanceMode = whiteBalanceMode;
}

@Override
public String toString() {
 StringBuilder builder = new StringBuilder();
 builder.append("DroneData [date=");
 builder.append(date);
 builder.append(", compression=");
 builder.append(compression);
 builder.append(", exifVersion=");
 builder.append(exifVersion);
 builder.append(", componentsConfiguration=");
 builder.append(componentsConfiguration);
 builder.append(", fileGroup=");
 builder.append(fileGroup);
 builder.append(", compressionType=");
 builder.append(compressionType);
 builder.append(", imageDescription=");
 builder.append(imageDescription);
 builder.append(", numberOfComponents=");
 builder.append(numberOfComponents);
 builder.append(", component2=");
 builder.append(component2);
 builder.append(", focalLength=");
 builder.append(focalLength);
 builder.append(", component1=");
 builder.append(component1);
 builder.append(", yCbCrPositioning=");
 builder.append(yCbCrPositioning);
 builder.append(", tiffResolutionUnit=");
 builder.append(tiffResolutionUnit);
 builder.append(", uuid=");
 builder.append(uuid);
 builder.append(", dateTimeOriginal=");
 builder.append(dateTimeOriginal);
 builder.append(", shutterSpeedValue=");
 builder.append(shutterSpeedValue);
 builder.append(", xResolution=");
 builder.append(xResolution);
 builder.append(", tiffMake=");
 builder.append(tiffMake);
 builder.append(", path=");
 builder.append(path);
 builder.append(", photometricInterpretation=");
 builder.append(photometricInterpretation);
 builder.append(", component3=");
 builder.append(component3);
 builder.append(", uniqueImageID=");
 builder.append(uniqueImageID);
 builder.append(", fNumber=");
 builder.append(fNumber);
 builder.append(", modified=");
 builder.append(modified);
 builder.append(", focalLength35=");
 builder.append(focalLength35);
 builder.append(", tiffBitsPerSample=");
 builder.append(tiffBitsPerSample);
 builder.append(", exposureProgram=");
 builder.append(exposureProgram);
 builder.append(", gPSVersionID=");
 builder.append(gPSVersionID);
 builder.append(", gPSLatitudeRef=");
 builder.append(gPSLatitudeRef);
 builder.append(", metaCreationDate=");
 builder.append(metaCreationDate);
 builder.append(", exifFNumber=");
 builder.append(exifFNumber);
 builder.append(", gPSAltitudeRef=");
 builder.append(gPSAltitudeRef);
 builder.append(", exposureTime=");
 builder.append(exposureTime);
 builder.append(", gPSLongitude=");
 builder.append(gPSLongitude);
 builder.append(", creationDate=");
 builder.append(creationDate);
 builder.append(", iSOSpeedRatings=");
 builder.append(iSOSpeedRatings);
 builder.append(", make=");
 builder.append(make);
 builder.append(", orientation=");
 builder.append(orientation);
 builder.append(", meteringMode=");
 builder.append(meteringMode);
 builder.append(", tiffOrientation=");
 builder.append(tiffOrientation);
 builder.append(", gPSLongitudeRef=");
 builder.append(gPSLongitudeRef);
 builder.append(", tiffSoftware=");
 builder.append(tiffSoftware);
 builder.append(", exifFocalLength=");
 builder.append(exifFocalLength);
 builder.append(", filename=");
 builder.append(filename);
 builder.append(", xMPValueCount=");
 builder.append(xMPValueCount);
 builder.append(", geoLong=");
 builder.append(geoLong);
 builder.append(", fileOwner=");
 builder.append(fileOwner);
 builder.append(", software=");
 builder.append(software);
 builder.append(", exifImageHeight=");
 builder.append(exifImageHeight);
 builder.append(", tiffYResolution=");
 builder.append(tiffYResolution);
 builder.append(", yResolution=");
 builder.append(yResolution);
 builder.append(", gPSLatitude=");
 builder.append(gPSLatitude);
 builder.append(", dcDescription=");
 builder.append(dcDescription);
 builder.append(", geoLat=");
 builder.append(geoLat);
 builder.append(", flashPixVersion=");
 builder.append(flashPixVersion);
 builder.append(", dataPrecision=");
 builder.append(dataPrecision);
 builder.append(", whiteBalance=");
 builder.append(whiteBalance);
 builder.append(", tiffImageLength=");
 builder.append(tiffImageLength);
 builder.append(", description=");
 builder.append(description);
 builder.append(", dctermsCreated=");
 builder.append(dctermsCreated);
 builder.append(", dctermsModified=");
 builder.append(dctermsModified);
 builder.append(", lastModified=");
 builder.append(lastModified);
 builder.append(", filePermissions=");
 builder.append(filePermissions);
 builder.append(", exifExposureTime=");
 builder.append(exifExposureTime);
 builder.append(", lastSaveDate=");
 builder.append(lastSaveDate);
 builder.append(", gPSAltitude=");
 builder.append(gPSAltitude);
 builder.append(", absolutePath=");
 builder.append(absolutePath);
 builder.append(", colorSpace=");
 builder.append(colorSpace);
 builder.append(", fileSize=");
 builder.append(fileSize);
 builder.append(", metaSaveDate=");
 builder.append(metaSaveDate);
 builder.append(", fileCreationTime=");
 builder.append(fileCreationTime);
 builder.append(", dateTimeDigitized=");
 builder.append(dateTimeDigitized);
 builder.append(", fileName=");
 builder.append(fileName);
 builder.append(", contentType=");
 builder.append(contentType);
 builder.append(", apertureValue=");
 builder.append(apertureValue);
 builder.append(", xParsedBy=");
 builder.append(xParsedBy);
 builder.append(", fileModifiedDate=");
 builder.append(fileModifiedDate);
 builder.append(", tiffXResolution=");
 builder.append(tiffXResolution);
 builder.append(", fileLastModifiedTime=");
 builder.append(fileLastModifiedTime);
 builder.append(", exifDateTimeOriginal=");
 builder.append(exifDateTimeOriginal);
 builder.append(", dateTime=");
 builder.append(dateTime);
 builder.append(", exifImageWidth=");
 builder.append(exifImageWidth);
 builder.append(", imageHeight=");
 builder.append(imageHeight);
 builder.append(", imageWidth=");
 builder.append(imageWidth);
 builder.append(", unknownTag0xc62f=");
 builder.append(unknownTag0xc62f);
 builder.append(", resolutionUnit=");
 builder.append(resolutionUnit);
 builder.append(", tiffModel=");
 builder.append(tiffModel);
 builder.append(", exifIsoSpeedRatings=");
 builder.append(exifIsoSpeedRatings);
 builder.append(", maxApertureValue=");
 builder.append(maxApertureValue);
 builder.append(", exposureMode=");
 builder.append(exposureMode);
 builder.append(", model=");
 builder.append(model);
 builder.append(", fileLastAccessTime=");
 builder.append(fileLastAccessTime);
 builder.append(", tiffImageWidth=");
 builder.append(tiffImageWidth);
 builder.append(", whiteBalanceMode=");
 builder.append(whiteBalanceMode);
 builder.append("]");
 return builder.toString();
}

public String getDate() {
 return date;
}

public void setDate(String date) {
 this.date = date;
}

public String getCompression() {
 return compression;
}

public void setCompression(String compression) {
 this.compression = compression;
}

public String getExifVersion() {
 return exifVersion;
}

public void setExifVersion(String exifVersion) {
 this.exifVersion = exifVersion;
}

public String getComponentsConfiguration() {
 return componentsConfiguration;
}

public void setComponentsConfiguration(String componentsConfiguration) {
 this.componentsConfiguration = componentsConfiguration;
}

public String getFileGroup() {
 return fileGroup;
}

public void setFileGroup(String fileGroup) {
 this.fileGroup = fileGroup;
}

public String getCompressionType() {
 return compressionType;
}

public void setCompressionType(String compressionType) {
 this.compressionType = compressionType;
}

public String getImageDescription() {
 return imageDescription;
}

public void setImageDescription(String imageDescription) {
 this.imageDescription = imageDescription;
}

public String getNumberOfComponents() {
 return numberOfComponents;
}

public void setNumberOfComponents(String numberOfComponents) {
 this.numberOfComponents = numberOfComponents;
}

public String getComponent2() {
 return component2;
}

public void setComponent2(String component2) {
 this.component2 = component2;
}

public String getFocalLength() {
 return focalLength;
}

public void setFocalLength(String focalLength) {
 this.focalLength = focalLength;
}

public String getComponent1() {
 return component1;
}

public void setComponent1(String component1) {
 this.component1 = component1;
}

public String getyCbCrPositioning() {
 return yCbCrPositioning;
}

public void setyCbCrPositioning(String yCbCrPositioning) {
 this.yCbCrPositioning = yCbCrPositioning;
}

public String getTiffResolutionUnit() {
 return tiffResolutionUnit;
}

public void setTiffResolutionUnit(String tiffResolutionUnit) {
 this.tiffResolutionUnit = tiffResolutionUnit;
}

public String getUuid() {
 return uuid;
}

public void setUuid(String uuid) {
 this.uuid = uuid;
}

public String getDateTimeOriginal() {
 return dateTimeOriginal;
}

public void setDateTimeOriginal(String dateTimeOriginal) {
 this.dateTimeOriginal = dateTimeOriginal;
}

public String getShutterSpeedValue() {
 return shutterSpeedValue;
}

public void setShutterSpeedValue(String shutterSpeedValue) {
 this.shutterSpeedValue = shutterSpeedValue;
}

public String getxResolution() {
 return xResolution;
}

public void setxResolution(String xResolution) {
 this.xResolution = xResolution;
}

public String getTiffMake() {
 return tiffMake;
}

public void setTiffMake(String tiffMake) {
 this.tiffMake = tiffMake;
}

public String getPath() {
 return path;
}

public void setPath(String path) {
 this.path = path;
}

public String getPhotometricInterpretation() {
 return photometricInterpretation;
}

public void setPhotometricInterpretation(String photometricInterpretation) {
 this.photometricInterpretation = photometricInterpretation;
}

public String getComponent3() {
 return component3;
}

public void setComponent3(String component3) {
 this.component3 = component3;
}

public String getUniqueImageID() {
 return uniqueImageID;
}

public void setUniqueImageID(String uniqueImageID) {
 this.uniqueImageID = uniqueImageID;
}

public String getfNumber() {
 return fNumber;
}

public void setfNumber(String fNumber) {
 this.fNumber = fNumber;
}

public String getModified() {
 return modified;
}

public void setModified(String modified) {
 this.modified = modified;
}

public String getFocalLength35() {
 return focalLength35;
}

public void setFocalLength35(String focalLength35) {
 this.focalLength35 = focalLength35;
}

public String getTiffBitsPerSample() {
 return tiffBitsPerSample;
}

public void setTiffBitsPerSample(String tiffBitsPerSample) {
 this.tiffBitsPerSample = tiffBitsPerSample;
}

public String getExposureProgram() {
 return exposureProgram;
}

public void setExposureProgram(String exposureProgram) {
 this.exposureProgram = exposureProgram;
}

public String getgPSVersionID() {
 return gPSVersionID;
}

public void setgPSVersionID(String gPSVersionID) {
 this.gPSVersionID = gPSVersionID;
}

public String getgPSLatitudeRef() {
 return gPSLatitudeRef;
}

public void setgPSLatitudeRef(String gPSLatitudeRef) {
 this.gPSLatitudeRef = gPSLatitudeRef;
}

public String getMetaCreationDate() {
 return metaCreationDate;
}

public void setMetaCreationDate(String metaCreationDate) {
 this.metaCreationDate = metaCreationDate;
}

public String getExifFNumber() {
 return exifFNumber;
}

public void setExifFNumber(String exifFNumber) {
 this.exifFNumber = exifFNumber;
}

public String getgPSAltitudeRef() {
 return gPSAltitudeRef;
}

public void setgPSAltitudeRef(String gPSAltitudeRef) {
 this.gPSAltitudeRef = gPSAltitudeRef;
}

public String getExposureTime() {
 return exposureTime;
}

public void setExposureTime(String exposureTime) {
 this.exposureTime = exposureTime;
}

public String getgPSLongitude() {
 return gPSLongitude;
}

public void setgPSLongitude(String gPSLongitude) {
 this.gPSLongitude = gPSLongitude;
}

public String getCreationDate() {
 return creationDate;
}

public void setCreationDate(String creationDate) {
 this.creationDate = creationDate;
}

public String getiSOSpeedRatings() {
 return iSOSpeedRatings;
}

public void setiSOSpeedRatings(String iSOSpeedRatings) {
 this.iSOSpeedRatings = iSOSpeedRatings;
}

public String getMake() {
 return make;
}

public void setMake(String make) {
 this.make = make;
}

public String getOrientation() {
 return orientation;
}

public void setOrientation(String orientation) {
 this.orientation = orientation;
}

public String getMeteringMode() {
 return meteringMode;
}

public void setMeteringMode(String meteringMode) {
 this.meteringMode = meteringMode;
}

public String getTiffOrientation() {
 return tiffOrientation;
}

public void setTiffOrientation(String tiffOrientation) {
 this.tiffOrientation = tiffOrientation;
}

public String getgPSLongitudeRef() {
 return gPSLongitudeRef;
}

public void setgPSLongitudeRef(String gPSLongitudeRef) {
 this.gPSLongitudeRef = gPSLongitudeRef;
}

public String getTiffSoftware() {
 return tiffSoftware;
}

public void setTiffSoftware(String tiffSoftware) {
 this.tiffSoftware = tiffSoftware;
}

public String getExifFocalLength() {
 return exifFocalLength;
}

public void setExifFocalLength(String exifFocalLength) {
 this.exifFocalLength = exifFocalLength;
}

public String getFilename() {
 return filename;
}

public void setFilename(String filename) {
 this.filename = filename;
}

public String getxMPValueCount() {
 return xMPValueCount;
}

public void setxMPValueCount(String xMPValueCount) {
 this.xMPValueCount = xMPValueCount;
}

public String getGeoLong() {
 return geoLong;
}

public void setGeoLong(String geoLong) {
 this.geoLong = geoLong;
}

public String getFileOwner() {
 return fileOwner;
}

public void setFileOwner(String fileOwner) {
 this.fileOwner = fileOwner;
}

public String getSoftware() {
 return software;
}

public void setSoftware(String software) {
 this.software = software;
}

public String getExifImageHeight() {
 return exifImageHeight;
}

public void setExifImageHeight(String exifImageHeight) {
 this.exifImageHeight = exifImageHeight;
}

public String getTiffYResolution() {
 return tiffYResolution;
}

public void setTiffYResolution(String tiffYResolution) {
 this.tiffYResolution = tiffYResolution;
}

public String getyResolution() {
 return yResolution;
}

public void setyResolution(String yResolution) {
 this.yResolution = yResolution;
}

public String getgPSLatitude() {
 return gPSLatitude;
}

public void setgPSLatitude(String gPSLatitude) {
 this.gPSLatitude = gPSLatitude;
}

public String getDcDescription() {
 return dcDescription;
}

public void setDcDescription(String dcDescription) {
 this.dcDescription = dcDescription;
}

public String getGeoLat() {
 return geoLat;
}

public void setGeoLat(String geoLat) {
 this.geoLat = geoLat;
}

public String getFlashPixVersion() {
 return flashPixVersion;
}

public void setFlashPixVersion(String flashPixVersion) {
 this.flashPixVersion = flashPixVersion;
}

public String getDataPrecision() {
 return dataPrecision;
}

public void setDataPrecision(String dataPrecision) {
 this.dataPrecision = dataPrecision;
}

public String getWhiteBalance() {
 return whiteBalance;
}

public void setWhiteBalance(String whiteBalance) {
 this.whiteBalance = whiteBalance;
}

public String getTiffImageLength() {
 return tiffImageLength;
}

public void setTiffImageLength(String tiffImageLength) {
 this.tiffImageLength = tiffImageLength;
}

public String getDescription() {
 return description;
}

public void setDescription(String description) {
 this.description = description;
}

public String getDctermsCreated() {
 return dctermsCreated;
}

public void setDctermsCreated(String dctermsCreated) {
 this.dctermsCreated = dctermsCreated;
}

public String getDctermsModified() {
 return dctermsModified;
}

public void setDctermsModified(String dctermsModified) {
 this.dctermsModified = dctermsModified;
}

public String getLastModified() {
 return lastModified;
}

public void setLastModified(String lastModified) {
 this.lastModified = lastModified;
}

public String getFilePermissions() {
 return filePermissions;
}

public void setFilePermissions(String filePermissions) {
 this.filePermissions = filePermissions;
}

public String getExifExposureTime() {
 return exifExposureTime;
}

public void setExifExposureTime(String exifExposureTime) {
 this.exifExposureTime = exifExposureTime;
}

public String getLastSaveDate() {
 return lastSaveDate;
}

public void setLastSaveDate(String lastSaveDate) {
 this.lastSaveDate = lastSaveDate;
}

public String getgPSAltitude() {
 return gPSAltitude;
}

public void setgPSAltitude(String gPSAltitude) {
 this.gPSAltitude = gPSAltitude;
}

public String getAbsolutePath() {
 return absolutePath;
}

public void setAbsolutePath(String absolutePath) {
 this.absolutePath = absolutePath;
}

public String getColorSpace() {
 return colorSpace;
}

public void setColorSpace(String colorSpace) {
 this.colorSpace = colorSpace;
}

public String getFileSize() {
 return fileSize;
}

public void setFileSize(String fileSize) {
 this.fileSize = fileSize;
}

public String getMetaSaveDate() {
 return metaSaveDate;
}

public void setMetaSaveDate(String metaSaveDate) {
 this.metaSaveDate = metaSaveDate;
}

public String getFileCreationTime() {
 return fileCreationTime;
}

public void setFileCreationTime(String fileCreationTime) {
 this.fileCreationTime = fileCreationTime;
}

public String getDateTimeDigitized() {
 return dateTimeDigitized;
}

public void setDateTimeDigitized(String dateTimeDigitized) {
 this.dateTimeDigitized = dateTimeDigitized;
}

public String getFileName() {
 return fileName;
}

public void setFileName(String fileName) {
 this.fileName = fileName;
}

public String getContentType() {
 return contentType;
}

public void setContentType(String contentType) {
 this.contentType = contentType;
}

public String getApertureValue() {
 return apertureValue;
}

public void setApertureValue(String apertureValue) {
 this.apertureValue = apertureValue;
}

public String getxParsedBy() {
 return xParsedBy;
}

public void setxParsedBy(String xParsedBy) {
 this.xParsedBy = xParsedBy;
}

public String getFileModifiedDate() {
 return fileModifiedDate;
}

public void setFileModifiedDate(String fileModifiedDate) {
 this.fileModifiedDate = fileModifiedDate;
}

public String getTiffXResolution() {
 return tiffXResolution;
}

public void setTiffXResolution(String tiffXResolution) {
 this.tiffXResolution = tiffXResolution;
}

public String getFileLastModifiedTime() {
 return fileLastModifiedTime;
}

public void setFileLastModifiedTime(String fileLastModifiedTime) {
 this.fileLastModifiedTime = fileLastModifiedTime;
}

public String getExifDateTimeOriginal() {
 return exifDateTimeOriginal;
}

public void setExifDateTimeOriginal(String exifDateTimeOriginal) {
 this.exifDateTimeOriginal = exifDateTimeOriginal;
}

public String getDateTime() {
 return dateTime;
}

public void setDateTime(String dateTime) {
 this.dateTime = dateTime;
}

public String getExifImageWidth() {
 return exifImageWidth;
}

public void setExifImageWidth(String exifImageWidth) {
 this.exifImageWidth = exifImageWidth;
}

public String getImageHeight() {
 return imageHeight;
}

public void setImageHeight(String imageHeight) {
 this.imageHeight = imageHeight;
}

public String getImageWidth() {
 return imageWidth;
}

public void setImageWidth(String imageWidth) {
 this.imageWidth = imageWidth;
}

public String getUnknownTag0xc62f() {
 return unknownTag0xc62f;
}

public void setUnknownTag0xc62f(String unknownTag0xc62f) {
 this.unknownTag0xc62f = unknownTag0xc62f;
}

public String getResolutionUnit() {
 return resolutionUnit;
}

public void setResolutionUnit(String resolutionUnit) {
 this.resolutionUnit = resolutionUnit;
}

public String getTiffModel() {
 return tiffModel;
}

public void setTiffModel(String tiffModel) {
 this.tiffModel = tiffModel;
}

public String getExifIsoSpeedRatings() {
 return exifIsoSpeedRatings;
}

public void setExifIsoSpeedRatings(String exifIsoSpeedRatings) {
 this.exifIsoSpeedRatings = exifIsoSpeedRatings;
}

public String getMaxApertureValue() {
 return maxApertureValue;
}

public void setMaxApertureValue(String maxApertureValue) {
 this.maxApertureValue = maxApertureValue;
}

public String getExposureMode() {
 return exposureMode;
}

public void setExposureMode(String exposureMode) {
 this.exposureMode = exposureMode;
}

public String getModel() {
 return model;
}

public void setModel(String model) {
 this.model = model;
}

public String getFileLastAccessTime() {
 return fileLastAccessTime;
}

public void setFileLastAccessTime(String fileLastAccessTime) {
 this.fileLastAccessTime = fileLastAccessTime;
}

public String getTiffImageWidth() {
 return tiffImageWidth;
}

public void setTiffImageWidth(String tiffImageWidth) {
 this.tiffImageWidth = tiffImageWidth;
}

public String getWhiteBalanceMode() {
 return whiteBalanceMode;
}

public void setWhiteBalanceMode(String whiteBalanceMode) {
 this.whiteBalanceMode = whiteBalanceMode;
}


}

TensorFlow Code For Classifying Images

python classify_image.py
--image_file
/opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg

solar dish, solar collector, solar furnace (score
= 0.98316)
window screen (score = 0.00196)
manhole cover (score = 0.00070)
radiator (score= 0.00041)
doormat, welcome mat (score = 0.00041)

bazel-bin/tensorflow/examples/label_image/label_image--image=/opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg
tensorflow/examples/label_image/main.cc:204]
solar dish (577):
0.983162I tensorflow/examples/label_image/main.cc:204]
window screen (912): 0.00196204I tensorflow/examples/label_image/main.cc:204]
manhole cover (763): 0.000704005I tensorflow/examples/label_image/main.cc:204]
radiator (571): 0.000408321I tensorflow/examples/label_image/main.cc:204]
doormat (972): 0.000406186

Related Source Code and Slides

Reference

HDF / NiFi Flow

drones-meetup-14-dec-2016.xml

Cloudera Community

Community Articles

Drones (UAV) Data Ingest Methods

Apache Hadoop

Apache HCatalog

Apache Hive

Apache NiFi

Apache Phoenix

HDFS

Tensorflow