Member since
10-19-2016
151
Posts
59
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
998 | 03-22-2018 11:48 AM | |
1499 | 01-12-2018 06:25 PM | |
2630 | 01-12-2018 03:56 AM | |
4211 | 01-12-2018 03:38 AM | |
2290 | 01-02-2018 10:29 PM |
05-25-2018
10:26 PM
@Steve Rosenberry - Hmm, very weird - I haven't seen this myself. Aside from trying in private browsing mode, pinging @Peter LaPierre in case he's seen this before.
... View more
05-21-2018
01:20 AM
Heya @David Jones! Regarding the script stopping after starting the ambari server: Unfortunately, HDP's 2.6.1 start script is a bit before my time, but I'm pinging @glupu in case he can help. Regarding using the latest version (lots of sweet improvements/additions, *wink wink*): If you prefer running on Azure, here's a link to deploying the latest sandbox on your cloud: https://hortonworks.com/tutorial/sandbox-deployment-and-install-guide/section/4/ If you prefer running a VM on your personal machine (at least 8GB+ free to allocate to the sandbox), there are VirtualBox, VMWare, or Docker versions here: https://hortonworks.com/downloads/#sandbox The latest HDP sandbox doesn't come with Jupyter installed by default, but I whipped up the commands you can run inside the HDP sandbox (once deployed) in order to install Jupyter+Python3: https://gist.github.com/orendain/d4fe83c3c11ce77b146762eb3acac471 Good luck, amigo - let me know if I can help!
... View more
05-18-2018
10:57 PM
@Emmanuel Obayi Great question!! We have something in the works for the sandbox to meet this request that's easy for users to set up. I'll report back when I have some more news within the next week, or feel free to ping me and I can provide updates.
... View more
05-18-2018
10:21 PM
Hey @Andrew Riffle, wondering if you got around this. If not, could I suggest removing the docker container and re-creating it? I've deployed HDF a bunch of times and don't think I've come across this on the sandbox before.
... View more
05-18-2018
10:12 PM
Thanks @Eino Mäkitalo - we're working on some updates to that tutorial - should be updated soon! 🙂
... View more
05-18-2018
10:04 PM
@David Jones A few notes on issues you may be facing: - The 'du' output looks to have been made on the VM - so the 96% disk taken is okay, expected even. - What problem exactly are you running into? As far as I can tell, the warning in ambari-agent.out shouldn't stop any service from running - are any services not coming up? I hate to be that guy who asks "Have you tried the latest version?" ... but ... Have you considered checking out one of the newer versions of the HDP Sandbox? Problems may be easier to help debug.
... View more
05-18-2018
09:43 PM
Heya @Arthur Jaron, Your Azure VM may take a while to fully boot itself up and also the sandbox container within (hence the "System is booting up" message). Could you try again after a few minutes of getting this message? This seems like a good note to add to the tutorial - I'll get a ticket on this asap.
... View more
05-18-2018
09:24 PM
Hiya @Gwenael Dumont! Could you run "docker info" and see what your "Docker Root Dir" is set to? This is the directory that you'll have to check for free space. IIRC it's /var by default on Debian. The quick fix suggestion would be to clean/prune unused data to clear up disk that Docker takes up. You can 'docker rm', 'docker rmi', 'docker system prune' to run the built-in pruner, etc. Hope that helps!
... View more
03-22-2018
12:16 PM
1 Kudo
Heya @Steve Hofstra. There was recently an update to the HDF sandbox (v3.1.0), which changed what all services were included. As a result, some tutorials that pointed to HDFS, HBase, etc on HDF (pre-v3.1.0), had to be updated. I think most of them are, but there may be a couple still in the works that assume a pre-v3.1.0 sandbox (they may say they require some particular version on the tutorial itself). Hope that helps!
... View more
03-22-2018
12:13 PM
Hi @Sandeep Ranjan, did you ever get this resolved? If it still isn't going through, we can get you a sandbox through other means if you're still running into issues.
... View more
03-22-2018
11:48 AM
1 Kudo
Hey @PiAiYi I'm a bit confused as to what you're trying to do. I'll explain a bit how the sandbox is set up and hopefully that provides a bit of context into the IPs that are in play. When you ssh using port 22, you'll notice the hostname of the machine you log into is set to "sandbox-host". This hosts main job is to provide an environment for a Docker container (if you're new to Docker, no worries). There is a second VM (the Docker container) inside of this sandbox-host machine. When you ssh using port 2222, you are logging into the second VM (with hostname sandbox-hdp), which again, is a VM inside the sandbox-host machine. When you play with the sandbox (Ambari, HBase, Hive, etc.), all those services are in this second VM (sandbox-hdp). So when you are within the sandbox-hdp machine, you can ping 172.17.0.2 because that is it's own IP. The reason you may be having trouble pinging 128.222... is because that is the IP address of the outside VM. For security (and other) reasons, you're not able to ping outside machines in this way by default. What is it you're trying to do? There might be another approach we can suggest.
... View more
03-22-2018
11:36 AM
@rutuja jagtap To piggyback on what @Geoffrey Shelton Okot wrote, you'll need to set up a couple of things on MySQL. One is the Hive database, and the other is the Hive user and its permissions. The following should work: # Once SSH'd in to the HDF sandbox:
# mysql -u -phortonworks1
CREATE DATABASE hive DEFAULT CHARACTER SET utf8;
CREATE USER 'hive'@'%' IDENTIFIED BY "$pass";
GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%' WITH GRANT OPTION;
commit;
quit;
... View more
03-13-2018
10:32 PM
@Brendan Lyon Grr, that's the wrong path - how'd that happen. I'll get that patched up now. In the meantime. If you can change the contents of deploy-topology.sh from "...trucking-storm-topology/target/trucking-storm-topology-0.3..." to "trucking-storm-topology/target/scala-2.11/trucking-storm-topology-0.3..." That should do the trick.
... View more
03-13-2018
08:02 PM
@Brendan Lyon Heya - could you fire off the build-topology.sh script before the deploy script? ./scripts/build-topology.sh That should be part of the instructions. If they're not, I'll get that patched up! [There's also a nice little update to this tutorial coming in the next few days ;)]
... View more
03-10-2018
03:42 AM
@Brendan Lyon Alrighty, the github repo was updated. Kindly "git pull" once cd'd in the directory and then try the script again. In case you were also hitting a couple of other errors, I found a fixed a few other things while in there. Good luck!
... View more
03-10-2018
03:35 AM
@Brendan Lyon Thanks for reporting this! I'll get an update out asap. Hang in there - I'll reply again soon!
... View more
01-19-2018
06:30 PM
1 Kudo
@Diego Gimenez Heya, Looks like you have another service on your machine that's taking up the 8080 port, so requests aren't actually reaching Ambari. You can either stop the EDB Postgres service on your machine if it's not in use, or if you need to keep it on you can make a couple of changes to get the Ambari UI to show up via another port. What VM are you running the sandbox on (VirtualBox/VMware)?
... View more
01-17-2018
11:20 PM
Hi @Gerald Iakobinyi-Pich I wish I could give a more definitive answer, but this depends on a number of factors including your machine's available CPU and RAM. One thing I would suggest you do ahead of time is to increase the amount of memory that you dedicate to Docker (the default is pretty low) -- it should speed up any Docker operation. See here for a how-to: https://hortonworks.com/tutorial/sandbox-deployment-and-install-guide/section/3/#configure-docker-memory I've seen this load finish between 5min and (in an extreme case) 20-30min. If you ever find yourself throwing your hands up in frustration at Docker, you can also consider the VirtualBox version of the sandbox. Good luck!
... View more
01-15-2018
08:36 PM
Hiya @Touhid Alam A couple of suggestions: 1. Grab the latest version of HDP (2.6.3 instead of 2.5) from https://hortonworks.com/downloads/#sandbox 2. If you want to keep working with HDP 2.5, check out this fix: https://community.hortonworks.com/articles/98459/how-to-configure-networks-on-the-virtualbox-hdp-26.html 3. Keep in mind that we recommend having 8GB of RAM available for the sandbox itself, so it may run slow with only 6GB of RAM, and may crash if the 6GB you have is a total across your entire machine and not just allocated to the sandbox itself. Let me know how it goes! 🙂
... View more
01-13-2018
01:03 AM
@Luc Chapon Heya, check out the "Hortonworks Data Platform Add-Ons" section on Hortonworks' download page for JDBC drivers and some docs. https://hortonworks.com/downloads/ Regarding your JDBC url: I would refer to the documentation for the best answer, but if you want to connect to Hive by querying Zookeeper, here's an excerpt from the docs: jdbc:hive2://zk=192.168.0.1:2181/hiveserver2 Use this option to enable the Dynamic Service Discovery feature, which allows you to
connect to Hive servers that are registered against a ZooKeeper service by connecting
to the ZooKeeper service.
... View more
01-12-2018
06:53 PM
@Andres Urrego Regarding the VM failing, is it the services shutting down on their own and not staying up? One common cause of this is not enough memory - to reduce resource usage try turning off all services and starting only HDFS, Zookeeper, YARN and Spark. Also make sure that you give your VM at least 8GB of RAM (https://hortonworks.com/tutorial/sandbox-deployment-and-install-guide shows how). As far as documentation for Spark2/HDFS, here is a good Spark2 starter tutorial followed by a Spark2/HDFS project walkthrough. https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/#option-2-download-and-setup-hortonworks-data-platform-hdp-sandbox https://hortonworks.com/tutorial/sentiment-analysis-with-apache-spark/
... View more
01-12-2018
06:42 PM
@Matt Heya. @Jay Kumar SenSharma's is right on point, though it sounds like there's a bit of confusion as to what Hadoop's "copyFromLocal" command does and what it expects. You run the Hadoop command with the option "copyFromLocal" from within the sandbox. This command will look on the Sandbox's local filesystem and copy the files you specified into the Hadoop filesystem. I can definitely see how this may be a bit confusing - it sort of sounds like the command would pull in files from your machine into HDFS on the sandbox. Remember, though, that the sandbox exists on a virtual machine that by default is separate from your own and can't access your filesystem. So you can definitely use that command, but you'll first have to move your local files into the sandbox. There are a bunch of ways to do that in bulk, check out Jay's post regarding SCP. Hope that clarifies a bit, and good luck! 🙂
... View more
01-12-2018
06:30 PM
@Mihir Kothari I'm a bit late to this thread, but let me try and reproduce this and I'll report back!
... View more
01-12-2018
06:25 PM
@j van Arendonk Heya! Looks like there's a bit of a mixup! You are SSH'd into the "wrong" sandbox. The VM that you set up is actually a wrapper around the "real" sandbox. See how the hostname of the machine you are SSH'd into is "sandbox-host"? You are SSH'd into the VM, whereas you want to SSH into the sandbox within the VM. There are a few different ways of doing that, feel free to take your pick: Using SSH (notice that the port needs to be 2222): For VirtualBox: ssh -p 2222 root@localhost For VMWare: ssh -p 2222 root@<the-ip-of-your-vm> Using the browser: with the sandbox running, open a browser to http://localhost:4200 You'll know you are successfully SSH'd into the sandbox when the prompt you see is "[root@sandbox /]#" (notice there is no "-host"). Hope that helps! Let us know how it goes! 🙂
... View more
01-12-2018
06:10 PM
@Andres Urrego Glad you got it working! As you already linked below, here are a couple of good tutorials to get you started: https://hortonworks.com/tutorial/learning-the-ropes-of-the-hortonworks-sandbox/ and https://hortonworks.com/tutorial/hadoop-tutorial-getting-started-with-hdp/ Also, check out the splash page on the sandbox by navigating to http://localhost:8888 in your browser. If your goal is to play with Apache Spark, check out the Spark tutorials on this page: https://hortonworks.com/tutorials/ Good luck!
... View more
01-12-2018
03:56 AM
1 Kudo
@Andreas Danckert @Aziz Herch @Marco Almeida Yup - this issue was triggered by a Docker change. Please use this updated start script for deploying the HDP 2.6.3 sandbox: https://raw.githubusercontent.com/hortonworks/data-tutorials/f88da0da324e252996cf3201fc9b038e9564a665/tutorials/hdp/sandbox-deployment-and-install-guide/assets/start_sandbox-hdp.sh Note: Please only use this script for the HDP 2.6.3 sandbox. If you find yourself reading this thread in the future, a new process is in place and this script is likely much more simplified/streamlined in the future - so the link above will be obsolete.
... View more
01-12-2018
03:53 AM
1 Kudo
@Aziz Herch Yup - this issue was triggered by a Docker change. Please use this updated start script for deploying the HDP 2.6.3 sandbox: https://raw.githubusercontent.com/hortonworks/data-tutorials/f88da0da324e252996cf3201fc9b038e9564a665/tutorials/hdp/sandbox-deployment-and-install-guide/assets/start_sandbox-hdp.sh Note: Please only use this script for the HDP 2.6.3 sandbox. If you find yourself reading this thread in the future, a new process is in place and this script is likely much more simplified/streamlined in the future - so the link above will be obsolete.
... View more
01-12-2018
03:46 AM
Thanks for reporting back! 🙂
... View more
01-12-2018
03:45 AM
@Nikhil Jayaram Heya, this should answer your question: https://community.hortonworks.com/answers/158884/view.html
... View more
01-12-2018
03:44 AM
@Rafal Pronko The current version of the HDP sandbox may hang during boot, but underneath the covers services may be up and running. Try opening a browser and navigating to http://localhost:8080 - if Ambari shows up that you're good to go - the start script is just hanging a bit. That'll be fixed in a very near-future release. Another explanation is that your Docker wasn't allocated enough RAM. Did you do this step? https://hortonworks.com/tutorial/sandbox-deployment-and-install-guide/section/3/#for-mac
... View more