Member since
05-19-2017
25
Posts
1
Kudos Received
0
Solutions
09-04-2019
07:01 PM
I installed HDF3.11.1 for VMWare, it down't even get up and running;
I installed HDF3.1.1 for VirtualBox, however, it prompts me 502 Bad Gateway, seems a show stopper here.
So I want to download old version but I am so frustrated that I couldn't even find a download link!!! all the download link points to Cloudera and only the latest 3.1.1 is available.
Does anyone know the secret link for HDP/HDF 2.5/2.6?
I really cannot imagine why the merge created so much messy: other than this sandbox issue, where is the old version hidden like a secret "treasury"? why so many discussions/threads/documents in previous hortonworks are showing broken images - it's been so long since the merge completed.
I am very upset to the merge. It would be greatly appreciated if someone can share a working download link. Thank you from bottom of my heart.
... View more
Labels:
- Labels:
-
Cloudera DataFlow (CDF)
09-03-2019
07:21 PM
My platform and installations:
Windows 7 Professional
HDF Sandbox 3.1.1
Virtual Box 6.0
Working:
localhost:4200
Not working:
localhost:8080/8888, both show me 502 Bad Gateway
Using the Windows7 built-in Resource Monitor, it shows VirtualBox is listening to a wide range of ports as shown below:
What I tried and not working:
1. incognito
2. FF and Chrome
3. Increased the memory from 8GB to 16GB
4. Reboot Windows 7
Can anyone please investigate the root cause and fix it please?
Thank you very much.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Cloudera DataFlow (CDF)
09-03-2019
05:26 PM
Hi @Shelton I followed your suggestion and modified HOSTS file in the C:\Windows\System32\drivers\etc on my Windows 7 machine where the VirtualBox/Sandbox is installed, it still prompts me the 502 error. Here are the hosts file: 127.0.0.1 localhost ::1 localhost 127.0.0.1 sandbox.hortonworks.com sandbox I used the Windows7 built-in Resource Monitor to inspect which application is using 8080, and it shows VirtualBox: Since VirtualBox is where the sandbox resides, there is no way for me to terminate it. How do I then access Ambari from localhost? Thank you very much if you can kindly help me to sort it out at your earliest convenience.
... View more
01-20-2019
02:29 PM
Is it possible to create a flow in NiFi that after all the data ingestion and ETL complete, the pipeline triggers a Jupyter Notebook directly to kick off the Data Science model on the completed dataset? Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
12-02-2018
03:40 PM
Impala rejected the change of:
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
... View more
12-02-2018
03:09 PM
doesn't work here, full script is as below:
CREATE TABLE sr.sr2013 (
creation_date STRING,
status STRING,
first_3_chars_of_postal_code STRING,
intersection_street_1 STRING,
intersection_street_2 STRING,
ward STRING,
service_request_type STRING,
division STRING,
section STRING )
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
WITH SERDEPROPERTIES (
'colelction.delim'='\u0002',
'mapkey.delim'='\u0003',
'serialization.format'=',',
'field.delim'=',',
'skip.header.line.count'='1',
'quoteChar'= "\"") ;
... View more
07-11-2018
02:23 PM
Thank you @Krish E, did you sort it out now? I am having the same issue. What is your table's size?
... View more
06-17-2018
12:00 PM
ERROR: CDH 5.12.0-1.cdh5.12.0.p0.29 conflicts with SPARK2 2.0.0.cloudera.beta2-1.cdh5.7.0.p0.110234.
... View more
02-12-2018
07:22 PM
I am doing some practices, for some questions, there could be various solutions, for example, I can use RDD operations to do some filtering, sorting, and grouping; with DataFrame and SparkSQL, it is even easier to me to get the same result. My question is will there be a requirement in the exam that some questions must be resolved using RDD, not DataFrame+SparkSQL. or vice versa? Thank you.
... View more
02-02-2018
08:20 PM
I am not able to list any sqoop jobs despite I run the following command during the sqoop job running time: sqoop job --list Here is a screenshot: Thank you.
... View more
Labels:
- Labels:
-
Apache Sqoop
01-30-2018
12:54 PM
1 Kudo
Thanks Mr. Jervis, There have been a lots complaint about the cluster/VM performance, do you have any update from the certification department, I know you have been talking to them about those complaints.
... View more
01-30-2018
03:31 AM
I understand this is an old thread, hopefully my reply here will still get attention. My questions are: 1. The above discussion is confusing: the "marked" answer says template will be provided (not mandate to be used), the other user says multiple channels confirmed to him/her that no template was provided in the actual exam, which is true? 2. Is Flume excluded from the exam? 3. Is Internet browsing like Google allowed in the exam? I ask this question because often the case we come to the need to check documentation user guide for each component like Spark, Hive, etc. Thank you.
... View more
05-27-2017
05:48 AM
Hello, Did you sort it out your nifi on CM? I am looking for a solution on how to install NiFi on Cloudera cluster and using it from CM but have not seen a working walkthrough. Can you share your procedure? Thank you very much in advance.
... View more
05-19-2017
07:20 PM
Hello, I just started using Cloudera Quickstart on my local VM. According to this article: https://www.cloudera.com/documentation/enterprise/5-9-x/topics/sg_sentry_service_config.html, I need to do: Enabling the Sentry Service for Hive
Go to the Hive service.
Click the Configuration tab.
Select Scope > Hive (Service-Wide).
Select Category > Main.
Locate the Sentry Service property and select Sentry.
Click Save Changes to commit the changes.
Restart the Hive service. But in my Cloudera Quickstart there is no Sentry service, and in Hue-> Configuration: I don't see Sentry option in Sentry Service. Is it because I have a incomplete Cloudera Quickstart? or Sentry is not included in Quickstart? or I need to start Sentry service somewhere first? Please see the screenshot for your reference. Thank you very much.
... View more
Labels:
- Labels:
-
Apache Sentry
02-25-2017
09:07 PM
In case anyone ask, here is a screenshot:
... View more
02-25-2017
08:56 PM
Thank you Artem, could you please post the complete correct script/command here? I tried the following with no luck: spark-submit --class /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar /root/hdp/kafka_wordcount.py 192.168.128.119:2181 test Error:
java.lang.ClassNotFoundException: /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar spark-submit --class /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar /root/hdp/kafka_wordcount.py 192.168.128.119:2181 test Error:
java.lang.ClassNotFoundException: /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar
... View more
02-25-2017
07:49 PM
Hello,
I am doing a practice of using Spark Streaming to process Kafka data on sandbox 2.5
Here is my script and it is not working, can anyone help? Thank you very much in advance.
spark-submit --jars /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar /root/hdp/kafka_wordcount.py 192.168.128.119:2181 test
Error: No main class set in JAR; please specify one with --class
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Spark
02-24-2017
03:12 PM
@Luis Figueroa: Thank you for your suggestion however it doesn't work for me: I am using HDP 2.5, and I need to transfer a file from my local windows host to sandbox, here is my command: scp -P 2122 /RXIE/..../test.csv root@localhost:root I received the following error message: ssh: connection to host localhost port 2122: Connection refused Thank you.
... View more
02-24-2017
03:49 AM
I tried again using the hortonworks sandbox's ip address, this time it says Connection refused (tried port 2222, failed with same error) scp C/RXIE/Learning/Github/spark-master/examples/src/main/python/streaming/kafka_wordcount.py root@192.168.128.119:/root
... View more
02-24-2017
03:46 AM
Hello, I am trying to copy a file and following the above working suggestion, however, it failed here, please see the screenshot. First time: scp C/RXIE/Learning/Github/spark-master/examples/src/main/python/streaming/kafka_wordcount.py root@localhost:/root Second time: scp C:/RXIE/Learning/Github/spark-master/examples/src/main/python/streaming/kafka_wordcount.py root@localhost:/root How do I fix it? Thank you very much.
... View more
02-21-2017
12:28 PM
here is my consumer which receives console input but no file input from producer ./kafka-console-consumer.sh –zookeeper
192.168.112.129:2181 –topic kafka-topic1
... View more
02-21-2017
11:41 AM
consumer doesn't receive data, how shall I create consumer? thank you
... View more
02-21-2017
01:10 AM
Hello,
I am trying to take a csv file as the input of my Kafka producer as below:
./kafka-console-producer.sh --broker-list sandbox.hortonworks.com:6667 --topic kafka-topic1 < test.csv
Nothing happens and I will get kicked off to shell.
Can anyone help?
Thank you. It works fine when the input is from console:
... View more
Labels:
- Labels:
-
Apache Kafka