Member since
10-19-2016
151
Posts
59
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1001 | 03-22-2018 11:48 AM | |
1525 | 01-12-2018 06:25 PM | |
2644 | 01-12-2018 03:56 AM | |
4229 | 01-12-2018 03:38 AM | |
2300 | 01-02-2018 10:29 PM |
01-12-2018
03:38 AM
1 Kudo
@Andres Urrego @Julián Rodríguez The "loading bar" seeming to hang is an issue with the current sandbox and will be fixed in a very near-future release. Underneath the covers, though, services are started/starting - even though that screen appears for a while. Ty opening up a browser and navigating to: https://localhost:8080 (or SSH via the browser at localhost:4200). If these services load, then the sandbox is up and good to go - that loading screen is just hanging and should change a few minutes later. Newer releases on the HDP sandbox won't have this issue, and the current HDF sandbox is free of this already. Hope that helps!
... View more
01-12-2018
03:33 AM
2 Kudos
@Julián Rodríguez Heya Julián, On boot, select the first kernel (3.10 non-rescue). The rescue option won't start the system with the appropriate resources/state to run the services that need to run. If I understood your original post correctly, booting with the default kernel won't get you to a login screen and you may not immediately be able to visit http://<ip-address>:8888 (also try port 8080 [ambari] and port 4200 [ssh in the browser]). You're probably seeing a loading bar that seems to hang for a really long time. That's an issue that the current sandbox has and will be fixed in a very near-future release. Underneath the covers, though, services are started/starting - even though the screen appears for a while. After a minute of booting up your VM, you should be able to navigate to <ip-add>:4200, :8080, and lastly :8888. ... However ... The thing about VMWare (at least in most cases) is that it's not obvious what ip address the VM is given, so you have to wait until that Centos "loading bar" actually goes away, which depending on the machine could be upwards of 10minutes. I see your screenshots have an IP at the top - that *seems* to be it, though I've never used ESXi so I can't say for certain. TL:DR; boot the first kernel, and give the VM a few minutes to do it's thing. It's getting fixed in an upcoming build. Also, if you're okay not using ESXi for a bit, the VirtualBox/Docker versions of the sandbox. Let me know how it goes or if I misunderstood and something else is also going wrong.
... View more
01-11-2018
06:58 PM
@Amogh Suman Hey Amogh, your concerns are warranted. Docker is wonderful, though with moving parts and build requirements, there are issues that come up in different environments (filesystem requirements, kernel incompatibilities on linux, Docker for Mac/Windows needing to mature, diff. versions of Docker, etc.). The VirtualBox release of the sandbox uses the same Docker image you've tried, only that we set up the host environment (the VirtualBox VM) to be compatible. The short answer is yes, you can absolutely run the sandbox in the cloud. You'll want to deploy the Docker image on your VM once it's set up, making sure that you install the right environment (I'd recommend CentoOS 7, the latest 4.x linux Kernel, Docker 17.09) on that host machine. If you want to use Azure, there's a pre-built template you can use - walkthrough is here. We're currently working on templates for other cloud platforms, but no ETA that I can provide. Hope that's a good start - keep us updated!
... View more
01-03-2018
07:55 PM
@Andreas Danckert @Marco Almeida I don't have access to my linux box at this very moment but I'll give this a go when I can and report back. In the meantime, the latest version of Docker that we can verify the builds check out for are Docker 17.09 for both Linux and Mac. Sidenote: If ambari-server and ambari-agent are up, even if the rest of the start script hangs, the Ambari ui (http://localhost:8080) may still be accessible - could you see if that's the case?
... View more
01-03-2018
07:38 PM
@Kareem Phillip-Jackson To help better track where the issue may be, what step of the process are you at when you have trouble connecting? You said you did this numerous times, where any of those successful or were each met with the same issue? Also pinging the developer of that doc @jmedel
... View more
01-03-2018
07:03 PM
1 Kudo
@Chidambar Deshpande It's tough to say without knowing what all is running on your host machine, but the general recommendation is to give the VM as much RAM as possible without slowing your host machine down to the point that it becomes unusable. Stopping heavy host processes would also help. If you try the HDF sandbox instead of HDP, for data flow and streaming tasks, as long as you're not doing any heavy processing (running Druid jobs) 6-8GB would be more than enough in case you want to play around with that.
... View more
01-03-2018
06:59 PM
1 Kudo
For anyone stumbling upon this issue, solutions and/or discussions can be found here: https://community.hortonworks.com/answers/156749/view.html https://community.hortonworks.com/questions/156778/getting-the-following-error-message-when-running-i.html?childToView=156983#answer-156983
... View more
01-03-2018
06:54 PM
@Mike Blank Heya Mike, sorry for the delay this weekend - would you be able to download the new sandbox (either HDF 3.0.2 or HDP 2.6.3) and give that a try? First, try it as it comes out of the box. If it reports a connectivity issue, only then try the solution linked above.
... View more
01-03-2018
03:18 AM
1 Kudo
@fintan guihen Heya, good question indeed. The recommended way to shut down the sandbox is to first stop all services via the Ambari UI, and then sending the shutdown signal to the VM (powering off, not killing it). The HDF sandbox is built in a such a way that signaling a VM shutdown should attempt to gracefully end running services, helping end mid-transaction data corruption. Though the same isn't yet true for the current HDP sandbox, it should be in a very near-future release!
... View more
01-02-2018
10:29 PM
Turns out it was due to not calling maven's "clean" before re-packaging. It had magically disappeared from my sbt tasks during some update.
... View more
01-02-2018
09:55 PM
I'm building a custom processor (using the nifi-nar-maven-plugin plugin for assembling) and today, after updating a few dependencies, I noticed the produced nar file was almost twice of its original size. I extracted the nar file only to find that multiple versions of dependencies were packaged, rather than just the updated ones specified in the pom file. Is this a known issue? Any solution?
... View more
Labels:
- Labels:
-
Apache NiFi
12-29-2017
01:48 AM
@Sai Geetha M N I don't know of a Python version tutorial/post of the linked you shared, but here are some resources worth checking out that are Python/Spark/Hive-with-Spark related: https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/ https://hortonworks.com/tutorial/setting-up-a-spark-development-environment-with-python/ https://hortonworks.com/tutorial/using-hive-with-orc-from-apache-spark/
... View more
12-28-2017
11:58 PM
@Mike Blank Heya, here's a solution: https://community.hortonworks.com/articles/98459/how-to-configure-networks-on-the-virtualbox-hdp-26.html
... View more
12-27-2017
09:21 PM
Heya @oen oen @yHAM Hmm, odd - if the sandbox is starting up and Ambari is running then so should all of its services and Ambari agent should see them. The hostfile that you edited - did you edit your local machine's hostfile or the sandbox's? The sandbox's hostfile should not necessarily be touched - it depends on a specific hostname being used and needs to properly resolve to it. As a sanity check, if it's not too much trouble, could you redeploy the sandbox without making modifications to it and seeing if Ambari comes up on http://localhost:8080 after a minute?
... View more
12-27-2017
09:13 PM
Heya @Ahshan MD The current version of the HDP sandbox was not configured for straight out-of-the-box Kerberos use - just as a heads up. There may need to be some tender loving care put in to set up all of the . This 'security' track would be a good place to start, along with maybe these docs: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-security/content/use_manual_kerberos_setup.html If you get things going I'd be very interested in learning how it went.
... View more
12-27-2017
09:08 PM
Hey @Abhilash Anil Just as a sanity check, is HDFS and all of its components shown as running from within Ambari? http://localhost:8080 is the webui for Ambari.
... View more
12-27-2017
08:40 PM
Heya @Sai Manohar Boidapu A few different suggestions to get the ball rolling: 1. Is the HFDS service and it's components shown as running from within Ambari? There should be a green checkmark next to HDFS and its components (Namenode, etc.) 2. If it is, could you then see if the namenode is up and running: ps -ef | grep -i namenode
... View more
12-27-2017
08:31 PM
1 Kudo
@Chidambar Deshpande Heya Chidambar - could you give this a try and let me know how it goes? 🙂 https://community.hortonworks.com/articles/98459/how-to-configure-networks-on-the-virtualbox-hdp-26.html
... View more
12-22-2017
10:36 PM
@Lucjan Dudkiewicz Heya, my memory is a bit fuzzy on if the HDP 2.5 Sandbox comes with the option to install Hive View out of the box, but here's the process to follow to know for sure: In Ambari: Shortcut URL (http://localhost:8080/views/ADMIN_VIEW/2.6.0.0/INSTANCE/#/views) ... or ... to get there yourself, then from any Ambari page go to the top-right user menu -> "Manage Ambari" -> "Views" link on the left sidebar. There should be an entry under "View Name" named "HIVE". Expand this entry to expose the "Create Instance" button. Click on "Create Instance". The top two fields should say "HIVE", followed by a select box for "Version". If you have "2.0.0" as an option for "Version", this is the Hive 2.0 View. Fill out the top portion of the "Details" fieldset. You should be able to safely leave the defaults for the "Settings" fieldset. If you don't have "2.0.0", you likely only have "1.5.0", which is the non-2.0 Hive View that comes by default. Hope this helps!
... View more
12-22-2017
10:30 PM
@Harish Ingle Would you happen to have the latest HDP Sandbox (HDP 2.6.3)? https://hortonworks.com/downloads/#sandbox The version you have (2.5.x) should work as well, just mentioning in case you're interested. That said, how much RAM is being allocated to the Sandbox? By default it should be requesting 8GB from VirtualBox, was this changed? Could you try SSH'ing into the sandbox (ssh root@localhost.com -p 2222) and if that doesn't work, then into the wrapping VM (ssh root@localhost -p 2122).
... View more
12-22-2017
10:09 PM
@Divya Sodha It's been a while since I worked with MATLAB, so I may not be much help - but here are some resources I found and also a place on HCC to ask where the respective experts may be able to weight in. Exchange data with databases: https://www.mathworks.com/products/database.html Data Processing Track on HCC (might be a good place to ask): https://community.hortonworks.com/spaces/66/index.html Good luck!
... View more
12-22-2017
08:53 PM
@Ilya Brodetsky Heya, I noticed you got past this issue - mind sharing what you found? Thanks!
... View more
12-22-2017
08:53 PM
1 Kudo
Heya @Ilya Brodetsky , pinging @gdeleon as I think he may be able to provide some insight here.
... View more
12-22-2017
08:37 PM
1 Kudo
@Chidambar Deshpande Heya, 2 suggestions 🙂 Grab the latest Hortonworks Sandbox (HDP 2.6.3 or HDF 3.0.2) https://hortonworks.com/downloads/#sandbox If you want to stick with the HDP 2.5 build you have, try opening up a browser and navigating to: https://localhost:8080 - If Ambari loads, then the sandbox is up and good to go - the "Starting Sandbox..." prompt may just be hanging for a bit, though the sandbox is fully operational. Hope that helps, let me know how it goes 🙂
... View more
12-21-2017
12:06 AM
1 Kudo
@mark berman Yep, the link that @Andrew Lim links to would have been the one to use. Just as an update - that tutorial was revamped so that when using newer HDF sandboxes, necessary components are baked in and no external downloads/uploads should be necessary. So for people that stumbled upon this issue: The tutorial is updated in a way that everything you need comes pre-set up with your downloaded Hortonworks HDF Sandbox.
... View more
12-20-2017
11:22 PM
@sandeep Jain I'd advise against changing the sandboxes' hostname if you can. The hostname is specified as part of an under-the-hood "docker run" command, which you'd have to change and would require trashing your current sandbox instance. The sandbox comes with Ambari already set up, which means you'll have to perform some database-crobatics in order to change the hostname the different services already resolve. You can definitely decide on your own hostname when setting up HDP or HDF from scratch, but the Sandbox is pre-configured in a way to get you going fast and we don't recommend you try altering hostnames.
... View more
12-20-2017
10:49 PM
@Harish Ingle Try opening a browser and navigating to http://localhost:8080 If Ambari opens up, then the sandbox has started fine and the VM screen has just not been refreshed (and can be safely ignored).
... View more
12-20-2017
10:37 PM
Hiya @Lei Feng @glupu is referring to the VirtualBox and VMWare versions of the Sandbox, found on the Hortonworks downloads page: https://hortonworks.com/downloads/#sandbox
... View more
11-28-2017
04:35 PM
2 Kudos
@raouia Heya, if you want to change job retention configs, you'll want to change a few YARN configs. Search this page for "yarn.resourcemanager.max-completed-applications", and also "yarn.log-aggregation.retain-seconds" and similar configs. Also, here's a good blog post to check out: https://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/
... View more
11-28-2017
04:30 PM
1 Kudo
@Theyagarajan Kanakasabai In addition to Jay's suggestions, try opening a browser and navigating to http://localhost:8888 or http://localhost:8080. Does anything come up? It might be the case that the sandbox is indeed loaded, but the vmware display isn't updating. If that's the case, it sometimes looks like the CentOS loading bar on the bottom of the screen doesn't progress (but in reality things are probably already up and running).
... View more