Member since
01-25-2020
22
Posts
1
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2632 | 04-03-2020 10:49 PM | |
2670 | 03-31-2020 11:15 AM | |
865 | 03-30-2020 09:45 PM | |
2702 | 03-25-2020 02:48 AM |
04-21-2020
08:08 AM
Thanks Ferenc I did found that out and tried to setup Tez but ended up having much errors,there isn't any specific documentation that guides me how i need to set up Tez for Hive in CM 7+. Any details would be helpful
... View more
04-13-2020
01:25 PM
Hi Team
Can you please advise if i need to read avro files on the CCA-175 exam environment,will the below work
import com . databricks . spark . avro . _
val avroDF = sqlContext . read . avro ( "/user/test/emp-avro" )
... View more
Labels:
- Labels:
-
Apache Spark
-
Certification
04-03-2020
10:49 PM
Hi Although i managed to get that changed however it seems like it is not supported,see my other thread around the errors i faced,where Hive on Tez is only recommended. But to answer your question we can change this on Hive Configurations. Go to CM->Hive-> Configurations->search on execution engine-> change the parameter to Spark from MR.(Default would have been MR.It will get rid of the errors and your services will start all green but you still have runtime failures. https://community.cloudera.com/t5/Support-Questions/Hive-on-Spark-Cloudera-Manager-7-03-Unable-to-Proceed/m-p/293264#M216584
... View more
04-03-2020
10:42 PM
Hello Li thanks for getting back,are we saving the only option available for Hive runtime engine is Hive on Tez?As MR is also not supported? I didn't find enough documentation that could guide me with Hive on Tez configuratiion on CM,if you know any good resources i can follow do advise thank you
... View more
04-03-2020
07:25 AM
Any info on the above issue,please update
... View more
04-01-2020
11:32 PM
Hi All
I am badly stuck setting the execution engine to spark and running hive queries on CM 7.03.As i understood Mapreduce as execution engine isn't anymore supported and we have TEZ and SPARK available.
I have tried configuring spark as my execution engine as TEZ looked a bit complex.
Can anyone help please,i am stuck since 3 days
I am using set hive.execution.engine=spark;
and have alos checked the below parameters and values are as below.
yarn.scheduler.maximum-allocation-mb=10g
yarn.nodemanager.resource.memory-mb=40g
hive.spark.client.server.connect.timeout=90000ms
hive.spark.client.connect.timeout=90000ms
Spark Executor Memory + Overhead = 1 gb
**********************
Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create Spark client for Spark session 431866dd-3dc5-40c3-b8bc-a6ab1b5883f2)' 20/04/02 01:12:29 [HiveServer2-Background-Pool: Thread-47]: ERROR spark.SparkTask: Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create Spark client for Spark session 431866dd-3dc5-40c3-b8bc-a6ab1b5883f2)'
Caused by: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
-
Cloudera Manager
03-31-2020
11:15 AM
This is now solved. Seems like Hive execution as MR isn't supported in Hive Server 2 anymore,i changed it to Spark and restarted the server,it worked.
... View more
03-30-2020
09:45 PM
This is resolved now,the issue was with the selectionof old version of centOS OS..
... View more
03-30-2020
02:38 PM
Hii All
When i runt he wget command fr CDP birany file installation,i am getting the below error in the log.can you please confirm why this is happening.
Iam using the below commands.
$ wget https://archive.cloudera.com/cm7/7.0.3/cloudera-manager-installer.bin
$ chmod u+x cloudera-manager-installer.bin
$ sudo ./cloudera-manager-installer.bin
https://archive.cloudera.com/cm7/7.0.3/redhat6/yum/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found" Trying other mirror. Error: Cannot retrieve repository metadata (repomd.xml) for repository: cloudera-manager. Please verify its path and try again
... View more
- Tags:
- CDP
Labels:
- Labels:
-
Cloudera Data Platform (CDP)
03-30-2020
08:52 AM
Hi Team
When i click on any of the FREE Training links under
https://www.cloudera.com/about/training.html
I am getting 503 service Not Available error,can anyone please advise.
... View more
Labels:
- Labels:
-
Training
03-25-2020
02:48 AM
I Further checked as per the Cloudera Manager Error Advice Instructions to run the below command,however i don't seem to have the agent present and running,what is that i am missing? [User_XXX@cdh-mgr ~]$ /etc/init.d/cloudera-scm-agent status -bash: /etc/init.d/cloudera-scm-agent: No such file or directory
... View more
03-25-2020
02:37 AM
Hi All
In my CDP environment, Hive Server 2 is failing to start,Below is the log for the same
0-03-25 08:33:56: Starting HiveServer2 WARNING: Use "yarn jar" to launch YARN applications. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.0.3-1.cdh7.0.3.p0.1635019/jars/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.0.3-1.cdh7.0.3.p0.1635019/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Exception in thread "main" java.lang.IllegalArgumentException: mr execution engine is not supported!
of the failure,i am unable to figure out whats causing the failure of the service.
... View more
Labels:
03-25-2020
02:12 AM
Hi All My Hive Server 2 is failing to start,Below is the complete log file of the failure,i am unable to figure out whats causing the failure of the service. 2020-03-25 08:33:56: Starting HiveServer2
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.0.3-1.cdh7.0.3.p0.1635019/jars/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.0.3-1.cdh7.0.3.p0.1635019/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.IllegalArgumentException: mr execution engine is not supported!
at org.apache.hadoop.hive.conf.HiveConf.validateExecutionEngine(HiveConf.java:5594)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5578)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5434)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:100)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:84)
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
... View more
03-24-2020
05:22 AM
Hello Shelton CDP is running on GCP
... View more
03-24-2020
12:29 AM
Hi All
I am currently stuck in the attatched step while configuring the Cloudera Manager.Can anyone please advise or share any link that can guide through.
thanks
... View more
- Tags:
- cloudera manager
- CM
Labels:
03-23-2020
02:59 AM
Thank You Bill ,i was searching HDP deployment into GCP but the below link https://www.cloudera.com/tutorials/sandbox-deployment-and-install-guide.html only explains about VB,Vmware and Docker,is there any resource that can guide me how do i deploy the HDP Sandbox into GCP
... View more
03-22-2020
10:41 AM
I would like to know what is the difference between the HDP Sandbox VM and CDH.
I am looking to setup CDH on GCP,can anyone let me know would i still be able to use the services after the 60 days trial phase if not what is the cost like?This is for personal use.
Is it also possible with HDP Sandbox on GCP?Again for own use and not for any business account.
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
03-12-2020
03:57 AM
1 Kudo
Hi Bill Thanks for the details,is the HDP Sandbox just has the same softwares and tools as we get in the CDH versions?or these will be specific to Hotonworks. thanks Swastik
... View more
03-11-2020
03:06 PM
Hi All
I am looking to setup Quick Start VM trial version on virtual Box,i saw few links on internet and youtube they clearly shows the downloadable page where there is an option to download the cloudera VM for Virtual Box.
However the current downloadable link shows as below.and when i click Try Now it only shows a set of instructions as below but doesn't show anywhere where i can download a VM for virtual Box.The Virtual Box is setup on Windows 10 on my system.
https://www.cloudera.com/downloads/cdp-data-center-trial.html
[cut-and-paste of web page at the above URL elided —Moderator]
... View more
Labels:
- Labels:
-
Apache Hadoop