Cloudera Community
TimothySpann
user rank
user rank
Master Guru

Re: Adding NiFi Server to HDP 2.5 Sandbox

by Master Guru Master Guru in Support Questions
‎07-07-2016 08:35 PM
‎07-07-2016 08:35 PM
It is the HDP 2.5 Tech Preview Sandbox. Maybe experimental. ... View more

Re: Adding NiFi Server to HDP 2.5 Sandbox

by Master Guru Master Guru in Support Questions
‎07-07-2016 08:22 PM
‎07-07-2016 08:22 PM
I will download it myself, I know that works, thanks. I usually do 8090. I just saw it in the list when I was working on the 2.5 sandbox and thought that would be easier. ... View more

Adding NiFi Server to HDP 2.5 Sandbox

by Master Guru Master Guru in Support Questions
‎07-07-2016 08:01 PM
2 Kudos
‎07-07-2016 08:01 PM
2 Kudos
When I try to add the NiFi Service to the HDP 2.5 sandbox using Ambari, it fails. I have tried a few times on clean unchanged sandboxes and it always fails. There are a lot of warnings on configurations. Is there a specific set of configurations needed to make this work? ... View more
Labels:

Using Kafka Manager with HDP 2.5 Sandbox Kafka

by Master Guru Master Guru in Community Articles
‎07-07-2016 07:50 PM
2 Kudos
‎07-07-2016 07:50 PM
2 Kudos
Using Yahoo Kafka Manager Git clone the project (you need Java 8 to build). Then use SBT to do a clean distribution. This will take a while as it downloads a lot of jars. <code>kafka-manager.zkhosts="sandbox.hortonworks.com:2181" The build will produce a Zip file, unzip it, update configuration file (conf/application.conf) and then you can run it. ../kafka-manager/target/universal/kafka-manager-1.3.0.8.zip unzip ../kafka-manager/target/universal/kafka-manager-1.3.0.8.zip kafka-manager-1.3.0.8 git:(master) ✗ vi conf/application.conf kafka-manager-1.3.0.8 git:(master) ✗ bin/kafka-manager -Dconfig.file=conf/application.conf Access the Kafka Manager from Chrome http://localhost:9000/ Running Kafka Manager Resources https://github.com/yahoo/kafka-manager http://edbaker.weebly.com/blog/install-and-evaluation-of-yahoos-kafka-manager http://chennaihug.org/knowledgebase/yahoo-kafka-manager/ https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem Tools For Testing Kafka with a command-line client producer/consumer: https://github.com/edenhill/kafkacat (brew install kafkacat) For External Access You may need to set advertised.host.name http://stackoverflow.com/questions/31476679/send-kafkaproducer-from-local-machine-to-hortonworks-sandbox-on-virtualbox ... View more
Labels:

Create Kafka Topic and Use From Apache NiFi for HD...

by Master Guru Master Guru in Community Articles
‎07-07-2016 07:50 PM
1 Kudo
‎07-07-2016 07:50 PM
1 Kudo
From the Sandbox as Root /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper sandbox.hortonworks.com:2181 --replication-factor 1 --partitions 1 --topic people Test The Topic [root@sandbox kafka]# /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --topic people --zookeeper sandbox.hortonworks.com:2181 {metadata.broker.list=sandbox.hortonworks.com:6667, request.timeout.ms=30000, client.id=console-consumer-10628, security.protocol=PLAINTEXT} Resources: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_secure-kafka-ambari/content/ch_secure-kafka-create-topics.html https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_secure-kafka-ambari/content/ch_secure-kafka-produce-events.html ... View more

Accessing Hive on HDP 2.5 Sandbox

by Master Guru Master Guru in Community Articles
‎07-07-2016 06:04 PM
2 Kudos
‎07-07-2016 06:04 PM
2 Kudos
In HDP 2.5 Sandbox, I did a quick walk through. First thing I liked was the Visualization: The Data Explorer provided a nice query tool to view tables and graphs. The Data Visualization tab provides some nice graphing capabilities. After you run your queries you can look at the Tez results to see how it ran, it's a nice way to see what you may need to optimize. The Hive Ambari View is getting to be a very solid tool for working with Hive. From DDL (creating tables is easy) to viewing data, to updates and inserts. ... View more
Labels:

Re: storm: Start the storm service on sandbox

by Master Guru Master Guru in Support Questions
‎07-07-2016 03:39 PM
‎07-07-2016 03:39 PM
anything in the logs? ... View more

Re: NiFi OCR - Using Apache NiFi to read children’...

by Master Guru Master Guru in Community Articles
‎07-07-2016 03:01 PM
‎07-07-2016 03:01 PM
do you have the code available? ... View more

Re: Why does HDF come with Storm and not Spark?

by Master Guru Master Guru in Support Questions
‎07-07-2016 02:12 PM
‎07-07-2016 02:12 PM
Spark, Flink and other frameworks most likely will be added as the community rallies behind adding those. Connecting from NiFi to Kafka to Spark or to Flink or AKKA is very straight forward ... View more

In Sandbox, HDP 2.5, getting a Hive Cli error

by Master Guru Master Guru in Support Questions
‎07-06-2016 06:32 PM
‎07-06-2016 06:32 PM
FAILED: RuntimeException java.io.FileNotFoundException: /etc/hive/2.5.0.0-817/0/xasecure-audit.xml I tried a few different users (root, hdfs, hive) [hive@sandbox conf]$ hive cli WARNING: Use "yarn jar" to launch YARN applications. Logging initialized using configuration in file:/etc/hive/2.5.0.0-817/0/hive-log4j.properties hive> show functions; FAILED: RuntimeException java.io.FileNotFoundException: /etc/hive/2.5.0.0-817/0/xasecure-audit.xml (No such file or directory) hive> show tables; Exception in thread "main" java.lang.AssertionError: Authorization plugins not initialized! at org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1509) at org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1513) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:626) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:506) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:318) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1197) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1245) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1134) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1124) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) [hive@sandbox conf]$ ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎05-20-2024 05:42 PM
Community Statistics
Member Since ‎01-07-2019 11:58 AM
Last Visited ‎05-20-2024 05:42 PM
Posts 1,973
Kudos received 1122