Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Cloudera Employee

Use S3 as storage for Zeppelin Notebooks.

Step 1. Use external storage to point to S3 bucket in Cloudbreak advance options. This will use S3access profile and AWS credentials. CB takes care of that set up. In addition to that

Step 2 Change these 3 inzeppelin-env.sh

export ZEPPELIN_NOTEBOOK_S3_BUCKET=yourBucketName

export ZEPPELIN_NOTEBOOK_S3_ENDPOINT="http://s3.amazonaws.com/yourBucketName"

export ZEPPELIN_NOTEBOOK_S3_USER=admin

Step 3 Change this 1 inzeppelin-site.xml

Point zeppelin.notebook.storage to org.apache.zeppelin.notebook.repo.S3NotebookRepo

Detailed Steps below

3. Launch HDP Cluster on AWS using Cloudbreak.

Not all the screenshots are included, only capturing screenshots that focus on some key Advance features that enable the required Zeppelin Storage.

  • a.Use the Advance tab on Cloudbreak.
  • b.Cloudbreak uses AWS credentials that will provide the necessary AWS Key and Secrect Access Key for S3 Storage setup.

80616-cb1.png

  • a.Provide an instance profile created in AWS that has access to your S3
  • b.Provide your bucket name for base storage

80617-cb2.png

4.Enable Zeppelin Storage on S3.

Once Ambari starts and all services are started we need to make some configuration changes to enable Zeppelin.

In the zeppelin-config change the following properties:

zeppelin_notebook.s3.bucket

zeppelin_notebook.s3.user

zeppelin_notebook.storage

80618-zeppelinconf.png

Or you could change them in zeppelin-env:

export ZEPPELIN_NOTEBOOK_S3_BUCKET=bucketName

export ZEPPELIN_NOTEBOOK_S3_ENDPOINT="http://s3.amazonaws.com/bucketName"

export ZEPPELIN_NOTEBOOK_S3_USER=admin


Here is an example path: bucket/user/notebook/2A94M5J1Z/note.json

80619-zeppenlinenv.png

Now when you create and save Notebooks in Zeppelin, it will save in S3.

80620-zeppelinnote.png

You will be able to see the notebooks in your AWS portal, in your S3 bucket. Zeppelin notebooks use 9 character hash as the name of the folder and note.json file in that folder.

80621-s3bucket.png

1,321 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 06:55 AM
Updated by:
 
Contributors
Top Kudoed Authors