Creating a CSD / Parcel for Nifi - CDH 5.7.0

[#1]

I've installed apache nifi on one of my CDH 5.7 clusters (linux version CentOS 6.7), but I'd like to manage it from within Cloudera Manger.

I did some research on parcels and on CSDs.  It looks like this is something I can do, and it doesn't look like it should be too difficult.

I came across the github page https://github.com/prateek/nifi-parcel, which gives step-by-step instructions for creating a nifi parcel for Cloudera.  Unfortunately, I'm running into errors.  

The steps instruct me to execute the command to download cloudera/cm_ext and then build it.

cd /tmp
git clone https://github.com/cloudera/cm_ext
cd cm_ext/validator
mvn install

When I execute maven to install the validator, I ran into a build failure.

[WARNING] The POM for com.cloudera.cmf.schema:cloudera-manager-schema:jar:5.5.0 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

I assumed that maybe 5.5.0 stands for CDH 5.5.0, so I updated the pom.xml to 5.7.0.

Downloading: http://repo.maven.apache.org/maven2/com/cloudera/cmf/schema/cloudera-manager-schema/5.7.0/cloudera-manager-schema-5.7.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
...
[ERROR] Failed to execute goal on project schema-validator: Could not resolve dependencies for project com.cloudera.enterprise:schema-validator:jar:5.7.0: Could not find artifact com.cloudera.cmf.schema:cloudera-manager-schema:jar:5.7.0 in cloudera-external (https://repository.cloudera.com/artifactory/ext-release-local/) -> [Help 1]

I searched https://repository.cloudera.com/artifactory/ext-release-local/ and found that there's nothing there under the ./com/cloudera directory.

Is there a better way to do this?

[#2]

Does it also fail when you mvn install from the root dir?, ie:

cd /tmp
git clone -b cm5-5.5.0 https://github.com/cloudera/cm_ext
#not cd cm_ext/validator
cd cm_ext
mvn install

let me know if this helps.

[1] the cloudera-manager-schema is here https://github.com/cloudera/cm_ext/blob/cm5-5.5.0/cm-schema/pom.xml#L5-L6

[2] see "Running the Validator" https://github.com/cloudera/cm_ext/tree/cm5-5.5.0

[#3]

I'm new to maven, so when I ran mvn install from /root, I got an error:

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.130s
[INFO] Finished at: Mon May 23 17:00:31 EDT 2016
[INFO] Final Memory: 7M/964M
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/root). Please verify you invoked Maven from the correct directory. -> [Help 1]

The POM file was in the /tmp/cm_ext/validator directory.

[root ~]# cd /tmp/cm_ext/validator
[root validator]# ll
total 5048
-rw-r--r--. 1 root root 5144659 Feb 19  2013 apache-maven-3.0.5-bin.tar.gz
-rw-r--r--. 1 root root    9409 May 23 16:58 pom.xml
-rw-r--r--. 1 root root     476 May 19 14:07 README.md
drwxr-xr-x. 5 root root    4096 May 19 14:07 src

I was able to biuld the cm-schema package as suggested.  Once I did, I got a new error when I tried to build validator:

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /tmp/cm_ext/validator/src/main/java/com/cloudera/cli/validator/ApplicationConfiguration.java:[30,20] package java.nio.file does not exist
[ERROR] /tmp/cm_ext/validator/src/main/java/com/cloudera/cli/validator/ApplicationConfiguration.java:[31,20] package java.nio.file does not exist
[ERROR] /tmp/cm_ext/validator/src/main/java/com/cloudera/cli/validator/ApplicationConfiguration.java:[72,18] cannot find symbol
symbol  : variable Paths
location: class com.cloudera.cli.validator.ApplicationConfiguration
[ERROR] /tmp/cm_ext/validator/src/main/java/com/cloudera/cli/validator/ApplicationConfiguration.java:[71,14] cannot find symbol
symbol  : variable Files
location: class com.cloudera.cli.validator.ApplicationConfiguration
[INFO] 4 errors
[

I assume that I need to install java.nio.file or to make sure that it's in a path that can be accessed.  I'll look into this, but if anyone has any clues that I can follow, they will be greatly appreciated.

Thanks,

- DaveW

[#4]

Can you list the JDK or javac -version ?

[#5]

Sure.

# java -version
java version "1.7.0_95"
OpenJDK Runtime Environment (rhel-2.6.4.0.el6_7-x86_64 u95-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

[#6]

how about the compiler "javac -version" ?

[#7]

Sorry,

# javac -version
javac 1.6.0_38

[#8 · Accepted solution]

My initial suggestion is to update your javac to be the same as your java version 1.7, as the "package java.nio.file" is part of the JDK

[#9]

That fixed it.  I used alternatives to install a new alternative to javac, then used it again to configure javac to the new alternative.  

alternatives --install /usr/bin/javac javac /usr/java/jdk1.7.0_67-cloudera/bin/javac 1

# alternatives --config javac

There are 2 programs which provide 'javac'.

  Selection    Command
-----------------------------------------------
*  1           /usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/javac
 + 2           /usr/java/jdk1.7.0_67-cloudera/bin/javac

Enter to keep the current selection[+], or type selection number: 2

Thanks for your help!

DaveW

[#10]

Since the question was asked, the situation has changed. As soon as Hortonworks and Cloudera merged, NiFi became supported by Cloudera.

Shortly after the integrations with CDH were also completed, so that NiFi is now a fully supported and integrated component.

Please look into the documentation for the latest info at any time, but in general Cloudera Manager is now able to install NiFi.