Member since
01-17-2023
3
Posts
0
Kudos Received
0
Solutions
01-18-2023
06:58 AM
Hey everyone, I am currently trying to run the hortonworks sandbox 3.0.1. I have installed it on VMware with 22GB of ram allocated and 2 processor cores set, as I cannot enable more than that. However, I have been facing a myriad of issues in this kind of install, ranging from servers such as YARN, HIVE and SPARK refusing to start fully. I am incredibly frustrated with my experience with Hortonworks Sandbox. Any help is appreciated. All I want to do is to complete this guide, https://www.cloudera.com/tutorials/using-hive-with-orc-in-apache-spark-repl.html. I just cannot get spark-shell started.
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
01-17-2023
02:30 PM
I would like to run hortonworks sandbox on 4 cores, but unfortunate I only have 2 that I can reasonably allocate. I have been having a lot of issues wityh hdfs not responsing, spark not responding and just overall system sluggishness. What can I do to improve the performance here? I can allocate up to 25GB or memory and the disk space isgreater than 100GB. So would that be sufficient?
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
01-17-2023
07:57 AM
I am currently going through this guide on ORC format table in Hive via Spark, https://www.cloudera.com/tutorials/using-hive-with-orc-in-apache-spark-repl.html I have been having a lot of issues with the VMWare HDP Sandbox since the start, which I assume is because I am running it with only 2 CPUs and 22GB of RAM allocated. The spark-shell command did execute fully once, but never after that till date. I have tried setting ZooKeeper, Oozie, and Ranger to maintanence mode to allow for some space to run, but this strategy is no longer working, Spark-shell command is always stuck after the log level setting prompt. I have ensured that YARN server is running, but to no avail. Any help is appreciated.
... View more
Labels:
- Labels:
-
Apache Spark