Code Repositories
Find and share code repositories
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Rising Star
Repo Description

Preventive Maintenance Data Science Use-Case for Fleet Cost Avoidance.

Repo Info
Github Repo URL https://github.com/kirkhas/zeppelin-notebooks/tree/master/Preventive_maintenance
Github account name kirkhas
Repo name Preventive_maintenance
237 Views
0 Kudos
Comments
Rising Star

This demo requires a custom library from this project: https://github.com/hortonworks/fieldeng-scythe.To install custom Scala dependencies from github using IntelliJ IDEA on Mac, perform these steps (replace descriptions in <carets> with appropriate values.

From a shell prompt:

cd ~/IdeaProjects
git clone https://github.com/<path>/<projectname>.git

(mkdir ~/IdeaProjects if the directory does not exist)

(clone https://github.com/hortonworks/fieldeng-scythe.git in this case)

  • Create a new IntelliJ project.

Open IntelliJ, create a new project under File -> New -> Project From Existing Sources, import from existing sources cloned to new directory created above

Add dependencies when prompted

  • Set up the project to create a JAR file.

In IntelliJ, add a JAR file to the outputs using File -> Project Structure, Artifacts. Select Create new Artifact -> JAR -> From modules with Dependencies.

Check "Include in project build” checkbox.

Path should default to: …/IdeaProjects/<projectname>/out/artifacts/<jarname>_jar. Give the JAR file an appropriate name.

Build the project by selecting Build -> Build Project (will likely rebuild the project completely due to the changes above)

Verify creation of JAR in local directory

  • Add the JAR as a dependency in the Zeppelin interpreter.

Copy JAR to Zeppelin server. Zeppelin will handle distributing the dependency to the Spark nodes.

scp -i ~/ssh/<certificate file>.pem ~/IdeaProjects/<projectname>/out/artifacts/<jarname>_jar<username>@<zeppelin.host.ip.address>:~
ssh -i ~/ssh/<certificate file>.pem <username>@<zeppelin.host.address> sudo cp <jarname>.jar /usr/lib

In Zeppelin Interpreter config, edit %Spark or %Spark2 interpreter as needed. At bottom of configuration settings, add dependency to file location with absolute path:

/usr/lib/<jarname>.jar

  • Add dependency to Zeppelin notebook:
%spark2
import <com.your.libraryname>._
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎03-29-2018 01:39 PM
Updated by:
 
Contributors
Top Kudoed Authors