Member since
09-10-2015
93
Posts
33
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
930 | 10-07-2016 03:37 PM | |
1150 | 10-04-2016 04:14 PM | |
892 | 09-29-2016 03:17 PM | |
622 | 09-28-2016 03:14 PM | |
936 | 09-09-2016 09:41 PM |
05-30-2017
07:00 PM
@Karl, have you tried running SparkR directly, without Zeppelin? That might help identify whether it's a Zeppelin or Spark(R) issue. Here's documentation for running SparkR with R from the command line: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/ch_spark-r.html
... View more
05-09-2017
09:43 PM
Also, the following info is for enabling user impersonation with %jdbc(phoenix): Make sure following the following configs are set up; In HBase, ensure that these three settings exist (if not, then add):
enable phoenix sql
In advance setting:
hbase.thrift.support.proxyuser=true
hbase.regionserver.thrift.http=true in HDFS config, ensure that these configs are set:
hadoop.proxyuser.hbase.groups=*
hadoop.proxyuser.hbase.hosts=*
hadoop.proxyuser.zeppelin.groups=*
hadoop.proxyuser.zeppelin.hosts=* Make sure the user has
access to HBase, which can be verified from "hbase shell", and then
running a command like user_permissions.
... View more
05-09-2017
09:18 PM
@PAVAN KUMAR JALLA, you can use the %jdbc interpreter to access Phoenix in HDP 2.6 and 2.5; use %jdbc(phoenix). Basics (without Kerberos) are similar to instructions for %jdbc(hive), at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_zeppelin-component-guide/content/using-jdbc-interp.html
... View more
05-08-2017
10:20 PM
1 Kudo
p.s. you can also access Zeppelin from the "Quick links" menu on the Zeppelin service page.
... View more
05-08-2017
10:18 PM
Hi @John Cleveland, my understanding is that the Zeppelin view has been removed (so the tutorial is out of sync). You should be able to access Zeppelin through the URL+port, as you mentioned. For another note about the view, see https://community.hortonworks.com/questions/68573/how-to-show-zeppelin-view-in-ambari-vmware-hdp-25.html
... View more
05-05-2017
05:13 PM
1 Kudo
The Spark Component Guide and Command Line Installation Guide were updated to reflect new Spark features. Here are links to several of the latest features: Support for Spark 2, documented in several topics including:
Installing Spark Using Ambari Installing and Configuring Apache Spark 2 (manual installation) Running Spark Configuring Spark2 for Wire Encryption Automating Spark Jobs with Oozie Spark Action Using Livy with Spark Versions 1 and 2 Livy API information, in Submitting Spark Applications Through Livy Enabling Spark SQL user impersonation for the Spark Thrift Server (doAs support), in Configuring the Spark Thrift Server The Zeppelin Component Guide was updated with additional details and examples for configuring Zeppelin with LDAP/AD and Kerberos security; see Configuring Zeppelin Security.
In addition, the documentation for interpreters and user impersonation
was extended. Portions of this information that apply to HDP 2.5 were
also added to the Security chapter in the HDP 2.5 Zeppelin Component
Guide. In the messaging area, the Kafka Component Guide has additional information in Configuring Kafka for a Production Environment.
... View more
- Find more articles tagged with:
- Data Science & Advanced Analytics
- data-science
- documentation
- How-ToTutorial
- Kafka
- Spark
- zeppelin
04-24-2017
07:41 PM
@Daniel Müller this is a general comment so it might not help, but there's a set of custom property boxes under the Spark service Configs tab, including one called Custom spark-hive-site-override. The Spark guide describes a similar custom property step (for doAs support) on the following page in the Spark guide, under the Ambari subsection: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/config-sts-user-imp.html
... View more
04-07-2017
04:43 PM
@Ken JiiiiIn case it's helpful, there are some examples (for 2.6) at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/spark-with-oozie-workflows.html and a community forum article at https://community.hortonworks.com/articles/84071/apache-ambari-workflow-manager-view-for-apache-ooz-2.html
... View more
04-07-2017
04:25 PM
Hi Ken, quick question--in your job.properties did you add two separate lines, they're just running together here?
... View more
03-13-2017
04:43 PM
Yes, documentation planned to sync with HDP 2.6 GA release.
... View more
03-10-2017
03:18 AM
1 Kudo
Not sure if you're using an HDP distribution with Zeppelin 0.7, but for HDP 2.5.x with Zeppelin 0.6 (supported) there's some info at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_zeppelin-component-guide/content/using-jdbc-interp.html
... View more
03-03-2017
07:22 PM
spark.apache.org has some tuning info for Spark streaming--not for Hive specifically, but maybe the general info will be helpful. The following link is for Spark 2.0.1: http://spark.apache.org/docs/2.0.1/streaming-programming-guide.html#performance-tuning
... View more
02-25-2017
11:12 PM
1 Kudo
@Raymond Xie, I hope Artem's info helped. As a side note, if you're working with Kafka 0.10 see the following links; the skc package (for using Spark 1.6.x with Kafka 0.10) has a couple of spark-submit examples. https://github.com/hortonworks-spark/skc http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/using-spark-streaming.html
... View more
02-22-2017
11:30 PM
1 Kudo
@Dinesh Chitlangia I'd also ask about your goals. If you plan to focus more on analytics, Python should support more statistical packages/libraries. There is also a Java API for Spark, which might get you started with Spark constructs more quickly; see https://spark.apache.org/docs/0.9.1/java-programming-guide.html. When I was thinking about a similar question the following article was helpful: https://datasciencevademecum.wordpress.com/2016/01/28/6-points-to-compare-python-and-scala-for-data-science-using-apache-spark/
... View more
02-17-2017
12:44 AM
1 Kudo
@zhixun he there is additional information and a spark-submit example at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/spark-add-libraries.html
... View more
02-15-2017
06:21 PM
1 Kudo
@Varun R
I'm not sure about saving a dashboard, but there's info about built-in visualizations on the following pages: apache.org overview at https://zeppelin.apache.org/docs/0.6.2/#data-visualization Tutorial at https://hortonworks.com/hadoop-tutorial/how-to-visualize-website-clickstream-data/ Helium might also help, in terms of being able to extend and save visualizations, https://zeppelin.apache.org/docs/0.7.0/development/writingzeppelinvisualization.html#writing-a-new-visualization-experimental
... View more
02-05-2017
04:35 AM
@Colin Cunningham, by "tab" do you mean the Interpreter option in the pulldown menu underneath your logon ID? There is an access control feature that hides settings on interpreter pages unless someone is authorized, but this sounds different (I haven't seen the pulldown option disappear). Here's the URL for the interpreter page if you want to try it directly: http://<host>:9995/#/interpreter
... View more
02-04-2017
08:26 PM
Not sure this'll help, but have you tried accessing the notebook UI from within Ambari? I think it's under the "Quick Links" menu. You'll probably get the same error, but maybe worth a try. Also see the following HCC post, re: 503 error-- https://community.hortonworks.com/content/kbentry/81471/zeppelin-ui-returns-503-error.html
... View more
02-04-2017
08:19 PM
p.s. is it this tutorial you're using? http://hortonworks.com/hadoop-tutorial/getting-started-apache-zeppelin/
... View more
02-04-2017
08:15 PM
I haven't seen it disabled, but what rguruvannagari says (below) makes sense. If you've tried that & it doesn't work, please post back. I'll also see if I can track down the sandbox owner to request an revision/clarification.
... View more
02-03-2017
10:22 PM
1 Kudo
Hi @Colin Cunningham, I'm wondering if Ambari could be writing over it, or at least conflicting with your settings, because you can specify shiro.ini settings from Ambari config tabs. For more info see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_zeppelin-component-guide/content/zepp-auth-getting-started.html
... View more
01-20-2017
06:13 PM
1 Kudo
@Edgar Daeds, from what I understand, not yet--you need to use multiple paragraphs. @jzhang, to confirm: did you mean (for now, in HDP 2.5.x) that they can run 10 queries in parallel, in 10 separate paragraphs?
... View more
01-17-2017
06:03 PM
@Christian Guegi I believe there are constraints for Kafka based on versions and on whether a cluster has Kerberos enabled or not. I'll try to find someone to respond for 2.4 to 2.5.3.
... View more
12-14-2016
04:49 PM
In general, you can find Spark-HDP version info in the Spark Component Guide. For example, see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/ch_introduction-spark.html. For HDP-Ambari version compatibility, see the stack compatibility section of the Ambari Installation Guide; for example, http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/determine_stack_compatibility.html
... View more
12-13-2016
11:57 PM
3 Kudos
@Andi Sonde , not sure if you came across the following in your research: http://kafka.apache.org/documentation.html#basic_ops_racks.
... View more
12-01-2016
03:21 AM
1 Kudo
There are many ways to run Hadoop on virtual machines. Earlier this year I tried several approaches, and ended up using a helpful Quick Start Guide written by Yusaku Sako. The Quick Start uses VirtualBox, Vagrant, and predefined scripts to set up a multi-node HDP cluster. You can choose which version of Ambari to install, and then choose and install an associated version of the HDP stack. For anyone new to virtual machines, there is now a Quick Start for New VM Users. The extended version adds background information and additional details for installing Ambari and the HDP stack. Topics include: Terminology Prerequisites Installing VirtualBox and Vagrant Starting Linux Virtual Machines Accessing Virtual Machines Installing Ambari Installing the HDP Stack Troubleshooting Reference information for basic Vagrant commands
... View more
- Find more articles tagged with:
- Ambari
- hdp-2.5.0
- How-ToTutorial
- Sandbox & Learning
- vagrant
- virtualbox
- VM
11-29-2016
11:37 PM
1 Kudo
It looks like you have R installed; is it on all nodes in your cluster? There is also a requirement to set JAVA_HOME. If you have access to Spark directly you might want to try accessing R from Spark first, to help isolate the issue. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/ch_spark-r.html
... View more
11-16-2016
06:38 PM
Thanks @Satish Duggana! For reference, this and related windowing articles have been incorporated into the Storm Component Guide; see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_storm-component-guide/content/storm-windowing.html
... View more
11-15-2016
11:53 PM
The HDP Spark Component Guide (versions 2.5.0+) has been updated per Bikas's clarification, http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/spark-encryption.html
... View more
10-12-2016
05:08 PM
1 Kudo
@Smart Solutions there is also some related info for Apache Spark version 1.6.2 (shipped with HDP 2.5) at https://spark.apache.org/docs/1.6.2/security.html#encryption.
... View more