About lgeorge

lgeorge · ‎05-30-2017

@Karl, have you tried running SparkR directly, without Zeppelin? That might help identify whether it's a Zeppelin or Spark(R) issue. Here's documentation for running SparkR with R from the command line: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/ch_spark-r.html

lgeorge · ‎05-09-2017

Also, the following info is for enabling user impersonation with %jdbc(phoenix): Make sure following the following configs are set up; In HBase, ensure that these three settings exist (if not, then add): enable phoenix sql In advance setting: hbase.thrift.support.proxyuser=true hbase.regionserver.thrift.http=true in HDFS config, ensure that these configs are set: hadoop.proxyuser.hbase.groups=* hadoop.proxyuser.hbase.hosts=* hadoop.proxyuser.zeppelin.groups=* hadoop.proxyuser.zeppelin.hosts=* Make sure the user has access to HBase, which can be verified from "hbase shell", and then running a command like user_permissions.

lgeorge · ‎05-09-2017

@PAVAN KUMAR JALLA, you can use the %jdbc interpreter to access Phoenix in HDP 2.6 and 2.5; use %jdbc(phoenix). Basics (without Kerberos) are similar to instructions for %jdbc(hive), at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_zeppelin-component-guide/content/using-jdbc-interp.html

lgeorge · ‎05-08-2017

p.s. you can also access Zeppelin from the "Quick links" menu on the Zeppelin service page.

lgeorge · ‎05-08-2017

Hi @John Cleveland, my understanding is that the Zeppelin view has been removed (so the tutorial is out of sync). You should be able to access Zeppelin through the URL+port, as you mentioned. For another note about the view, see https://community.hortonworks.com/questions/68573/how-to-show-zeppelin-view-in-ambari-vmware-hdp-25.html

lgeorge · ‎05-05-2017

The Spark Component Guide and Command Line Installation Guide were updated to reflect new Spark features. Here are links to several of the latest features: Support for Spark 2, documented in several topics including: Installing Spark Using Ambari Installing and Configuring Apache Spark 2 (manual installation) Running Spark Configuring Spark2 for Wire Encryption Automating Spark Jobs with Oozie Spark Action Using Livy with Spark Versions 1 and 2 Livy API information, in Submitting Spark Applications Through Livy Enabling Spark SQL user impersonation for the Spark Thrift Server (doAs support), in Configuring the Spark Thrift Server The Zeppelin Component Guide was updated with additional details and examples for configuring Zeppelin with LDAP/AD and Kerberos security; see Configuring Zeppelin Security. In addition, the documentation for interpreters and user impersonation was extended. Portions of this information that apply to HDP 2.5 were also added to the Security chapter in the HDP 2.5 Zeppelin Component Guide. In the messaging area, the Kafka Component Guide has additional information in Configuring Kafka for a Production Environment.

lgeorge · ‎04-24-2017

@Daniel Müller this is a general comment so it might not help, but there's a set of custom property boxes under the Spark service Configs tab, including one called Custom spark-hive-site-override. The Spark guide describes a similar custom property step (for doAs support) on the following page in the Spark guide, under the Ambari subsection: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/config-sts-user-imp.html

lgeorge · ‎04-07-2017

@Ken JiiiiIn case it's helpful, there are some examples (for 2.6) at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_spark-component-guide/content/spark-with-oozie-workflows.html and a community forum article at https://community.hortonworks.com/articles/84071/apache-ambari-workflow-manager-view-for-apache-ooz-2.html

lgeorge · ‎04-07-2017

Hi Ken, quick question--in your job.properties did you add two separate lines, they're just running together here?

lgeorge · ‎03-13-2017

Yes, documentation planned to sync with HDP 2.6 GA release.

lgeorge · ‎03-10-2017

Not sure if you're using an HDP distribution with Zeppelin 0.7, but for HDP 2.5.x with Zeppelin 0.6 (supported) there's some info at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_zeppelin-component-guide/content/using-jdbc-interp.html

lgeorge · ‎03-03-2017

spark.apache.org has some tuning info for Spark streaming--not for Hive specifically, but maybe the general info will be helpful. The following link is for Spark 2.0.1: http://spark.apache.org/docs/2.0.1/streaming-programming-guide.html#performance-tuning

lgeorge · ‎02-25-2017

@Raymond Xie, I hope Artem's info helped. As a side note, if you're working with Kafka 0.10 see the following links; the skc package (for using Spark 1.6.x with Kafka 0.10) has a couple of spark-submit examples. https://github.com/hortonworks-spark/skc http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/using-spark-streaming.html

lgeorge · ‎02-22-2017

@Dinesh Chitlangia I'd also ask about your goals. If you plan to focus more on analytics, Python should support more statistical packages/libraries. There is also a Java API for Spark, which might get you started with Spark constructs more quickly; see https://spark.apache.org/docs/0.9.1/java-programming-guide.html. When I was thinking about a similar question the following article was helpful: https://datasciencevademecum.wordpress.com/2016/01/28/6-points-to-compare-python-and-scala-for-data-science-using-apache-spark/

lgeorge · ‎02-17-2017

@zhixun he there is additional information and a spark-submit example at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/spark-add-libraries.html

lgeorge · ‎02-15-2017

@Varun R I'm not sure about saving a dashboard, but there's info about built-in visualizations on the following pages: apache.org overview at https://zeppelin.apache.org/docs/0.6.2/#data-visualization Tutorial at https://hortonworks.com/hadoop-tutorial/how-to-visualize-website-clickstream-data/ Helium might also help, in terms of being able to extend and save visualizations, https://zeppelin.apache.org/docs/0.7.0/development/writingzeppelinvisualization.html#writing-a-new-visualization-experimental

lgeorge · ‎02-05-2017

@Colin Cunningham, by "tab" do you mean the Interpreter option in the pulldown menu underneath your logon ID? There is an access control feature that hides settings on interpreter pages unless someone is authorized, but this sounds different (I haven't seen the pulldown option disappear). Here's the URL for the interpreter page if you want to try it directly: http://<host>:9995/#/interpreter

lgeorge · ‎02-04-2017

Not sure this'll help, but have you tried accessing the notebook UI from within Ambari? I think it's under the "Quick Links" menu. You'll probably get the same error, but maybe worth a try. Also see the following HCC post, re: 503 error-- https://community.hortonworks.com/content/kbentry/81471/zeppelin-ui-returns-503-error.html

lgeorge · ‎02-04-2017

p.s. is it this tutorial you're using? http://hortonworks.com/hadoop-tutorial/getting-started-apache-zeppelin/

lgeorge · ‎02-04-2017

I haven't seen it disabled, but what rguruvannagari says (below) makes sense. If you've tried that & it doesn't work, please post back. I'll also see if I can track down the sandbox owner to request an revision/clarification.

lgeorge · ‎02-03-2017

Hi @Colin Cunningham, I'm wondering if Ambari could be writing over it, or at least conflicting with your settings, because you can specify shiro.ini settings from Ambari config tabs. For more info see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_zeppelin-component-guide/content/zepp-auth-getting-started.html

lgeorge · ‎01-20-2017

@Edgar Daeds, from what I understand, not yet--you need to use multiple paragraphs. @jzhang, to confirm: did you mean (for now, in HDP 2.5.x) that they can run 10 queries in parallel, in 10 separate paragraphs?

lgeorge · ‎01-17-2017

@Christian Guegi I believe there are constraints for Kafka based on versions and on whether a cluster has Kerberos enabled or not. I'll try to find someone to respond for 2.4 to 2.5.3.

lgeorge · ‎12-14-2016

In general, you can find Spark-HDP version info in the Spark Component Guide. For example, see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/ch_introduction-spark.html. For HDP-Ambari version compatibility, see the stack compatibility section of the Ambari Installation Guide; for example, http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/determine_stack_compatibility.html

lgeorge · ‎12-13-2016

@Andi Sonde , not sure if you came across the following in your research: http://kafka.apache.org/documentation.html#basic_ops_racks.

lgeorge · ‎12-01-2016

There are many ways to run Hadoop on virtual machines. Earlier this year I tried several approaches, and ended up using a helpful Quick Start Guide written by Yusaku Sako. The Quick Start uses VirtualBox, Vagrant, and predefined scripts to set up a multi-node HDP cluster. You can choose which version of Ambari to install, and then choose and install an associated version of the HDP stack. For anyone new to virtual machines, there is now a Quick Start for New VM Users. The extended version adds background information and additional details for installing Ambari and the HDP stack. Topics include: Terminology Prerequisites Installing VirtualBox and Vagrant Starting Linux Virtual Machines Accessing Virtual Machines Installing Ambari Installing the HDP Stack Troubleshooting Reference information for basic Vagrant commands

lgeorge · ‎11-29-2016

It looks like you have R installed; is it on all nodes in your cluster? There is also a requirement to set JAVA_HOME. If you have access to Spark directly you might want to try accessing R from Spark first, to help isolate the issue. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/ch_spark-r.html

lgeorge · ‎11-16-2016

Thanks @Satish Duggana! For reference, this and related windowing articles have been incorporated into the Storm Component Guide; see http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_storm-component-guide/content/storm-windowing.html

lgeorge · ‎11-15-2016

The HDP Spark Component Guide (versions 2.5.0+) has been updated per Bikas's clarification, http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/spark-encryption.html

lgeorge · ‎10-12-2016

@Smart Solutions there is also some related info for Apache Spark version 1.6.2 (shipped with HDP 2.5) at https://spark.apache.org/docs/1.6.2/security.html#encryption.

Online	Offline
Last Visited	‎08-08-2019 07:47 PM

Date Registered	‎09-10-2015 08:46 PM
Last Visited	‎08-08-2019 07:47 PM
Total Messages Posted	93
Total Kudos Received	31

Re: how to install Solr using Ambari view?

Re: Dynamic allocation on Spark Standalone cluster

Re: spark2 not accessable

Re: apache spark 2.0

Re: Zeppelin: %hive and %phoenix interpreters in 2...

Re: Getting SparkR to work on Zeppelin (HDP2.6 San...

Re: how to access phoenix data from zepplein

Re: how to access phoenix data from zepplein

Re: Where is Zeppelin option in dropdown menu?

Re: Where is Zeppelin option in dropdown menu?

HDP 2.6 Documentation Updates for Data Science Com...

Re: Spark Settings with Ambari: Can't find corresp...

Re: How to run Oozie Spark action properly?

Re: How to run Oozie Spark action properly?

Re: Is Kerberos required for zeppelin hive identit...

Re: How to create hive interpreter in Zeppelin 0.7...

Re: Spark Performance Tuning

Re: Spark Streaming on Kafka data, question on fi...

Re: What language should I use to learn Spark?

Re: hadoop-lzo jar file is missing for spark servi...

Re: Create zeppelin notebook like kibana,hue,banan...

Re: How do I recover missing Zeppelin Interpreter ...

Re: my zepplin shiro.ini changes are repeatedly be...

Re: my zepplin shiro.ini changes are repeatedly be...

Re: my zepplin shiro.ini changes are repeatedly be...

Re: my zepplin shiro.ini changes are repeatedly be...

Re: Does zeppelin support multiple hive queries in...

Re: Kafka Rolling Upgrade

Re: Spark 1.5.2 installation

Re: Can anyone explain Kafka rack awareness featur...

Ambari and HDP Installation: Quick Start for new V...

Re: HDP, Zeppelin Notebook, Livy, Spark & R: "requ...

Re: Windowing in Apache Storm trident.

Re: Confusion in documentation : Configuring Spark...

Re: Confusion in documentation : Configuring Spark...