Member since
08-23-2016
261
Posts
201
Kudos Received
106
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
921 | 01-26-2018 07:28 PM | |
767 | 11-29-2017 04:02 PM | |
27255 | 11-29-2017 03:56 PM | |
1690 | 11-28-2017 01:01 AM | |
396 | 11-22-2017 04:08 PM |
10-20-2017
09:43 PM
1 Kudo
Hi @Pradeep Sadashivamurthy I'm not sure if you are using the default ports, or whether you have customized some of the ports. 9090 is usually the port for NiFi and not the schema registry. Schema Registry's default port, I think, is 7788. In addition, the Schema Registry has a very cool built-in tool to help making working with the APIs much easier. Try http://host.name:7788/api/swagger in a web browser, and see if the tool comes up, and go from there. Hope this helps
... View more
10-19-2017
03:59 PM
1 Kudo
Hi @Rajesh Reddy I've tried it from S3 to a regular HDFS cluster, but I don't see why it wouldn't work. Instead of the hdfs:// prefix for the source/destinations, you can replace that with s3a:// If your S3a keys are defined elsewhere already, they don't necessarily need to be passed in-line like this example: hadoop distcp -Dfs.s3a.access.key="<key>" -Dfs.s3a.secret.key="<my Key>"
s3a://hostname/dir/file s3a://hostname/dir/file
... View more
10-16-2017
03:12 PM
1 Kudo
Hi @Ni Pal I just signed into my account (US West region) and searched for "Hortonworks" in the community AMIs and am able to see it - see attached
... View more
10-16-2017
02:38 PM
1 Kudo
Hi @Sonu Singh I haven't encountered a HADOOP_HOME environment variable problem on the Sanbox before, but see this post on config for it: https://community.hortonworks.com/questions/109738/haddops-environment-variables.html
... View more
10-13-2017
08:00 PM
I think you have this already, but here's the Sandbox tutorial link with additional detail and screenshots to the steps I've written above: https://hortonworks.com/tutorial/learning-the-ropes-of-the-hortonworks-sandbox/
... View more
10-13-2017
07:59 PM
1 Kudo
@Sonu Singh Step 1: Modify the hosts file on your machine and reboot if required. On Windows 10, the hosts file that should be edited is usually at c:\Windows\System32\Drivers\etc\hosts with a line similar to: 127.0.0.1 sandbox.hortonworks.com Step 2: Ensure the Sandbox VM is up and running (can you reach Ambari to confirm?) Step 3: Open a terminal, or putty or whatever SSH tool you desire try SSH'ing to: sandbox.hortonworks.com port 2222 with user root. Step 4: If Step 3 did not work, try using the WebSSH tool built into the Sandbox: http://sandbox.hortonworks.com:4200 or http://127.0.0.1:4200 I can't think of anything else that would be in the way, I doubt very much that your Windows firewall (if enabled) would block access to something running on the local host but it won't hurt to double-check antivirus or firewall tools to see if they are blocking access. Maybe attach some screenshots of the VM running, and your VM config to see if something else jumps out at us. I suspect once we figure out why you cannot connect to the Sandbox, it may shed light on the 3rd party tool connections to the HiveServer2 running on the Sandbox as well.
... View more
10-13-2017
04:24 PM
Hi @Héctor Pérez Arteaga https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_data-governance/content/ch_hdp_data_governance_install_atlas_ambari.html Authorizer Resource Types:
Operation Type Entity Taxonomy Term Unknown Operations_Allowed are r = read, w = write, u = update, d = delete
... View more
10-11-2017
10:21 PM
1 Kudo
hi @Sonu Singh You are very welcome. I'm not sure why in your specific case the splash screen is not loading. I've had that had on one specific version of the sandbox in the past where there was an issue with the page, not sure if that is the csae for you. The goal of that page is to provide a starting point for users to be able to get to Ambari, get to other public links like the hortonworks documentation/tutorials and also access a web SSH tool (127.0.0.1:4200). The splash page itself is not needed though, one can access each of those things individually as well. In fact, I would say that most people go directly to Ambari and not bother with the splash page once they have used the sandbox a couple of times. Are you using Windows or Mac or linux? On a *nix/Mac machine, /etc/hosts is the file that can map a hostname to an ip address, and where I pasted that snip from. My VirtualBox adapter remains in NAT mode btw. I do also usually create a second network interface for a vbox host only adapter, if it is not already present. You can ssh to the cluster using whatever tool you like, just be sure to use the correct post (2222 for cluster access, 2122 for the docker host access which is normally not needed). If a tool is just accessing Hive on the standard port, there is likely no port forwarding required. The default hiveserver2 ports (10000) are usually open by default on the sandbox machines. A connection string of something like: jdbc:hive2://sandbox.hortonworks.com:10000/default should work for the sandbox once your /etc/hosts file hostname to ip mapping issue is sorted good luck!
... View more
10-11-2017
03:32 PM
1 Kudo
Hi @nisha vittal this appears to be a vmware issue with the cpu. Could you try using VirtualBox instead of vmware? Unfortunately that will require downloading the virtualbox sandbox, but could be worth the download time if it gets you up and running faster. We typically see less issues with the sandbox on virtualbox.
... View more
10-11-2017
02:58 PM
1 Kudo
Hi @Sonu Singh (Great name btw! :)) Welcome to the Hortonworks Sandbox, it is a great tool to start learning the Hortonworks platform with. I'll try to answer your questions in order. 1) I wouldn't worry too much about the splash page. If you can hit Ambari (http://127.0.0.1:8080) then you are good. The rest can be done via the web at https://hortonworks.com/tutorials/ 2) on my laptop's hosts file, I have an entry as follows: 127.0.0.1 sandbox.hortonworks.com sandbox When I need to connect to the sandbox, I just use the hostname and haven't had any issues. In fact, you should probably try ssh to your sandbox on port 2222, login as root, and change the Ambari admin password now if you haven't already done so. ssh -p 2222 root@sandbox.hortonworks.com (default pw is hadoop) then as root, run: ambari-admin-password-reset 3) Most of the tutorial work on the Sandbox can be done through Ambari, so logging into Ambari and using the occasional terminal via SSH as shown above should be all you need to connect to the cluster to get started. 4) The Sandbox uses docker internally, in addition to the vmware/virtualbox layer on your machine. This can sometimes making connecting third party tools (such as Talend) to it a bit tricky because of port forwarding issues. The port not only has to be open on your virtualbox layer, but also within the Docker host internally. See this link for a good explaination and guide on opening other ports: https://hortonworks.com/tutorial/sandbox-port-forwarding-guide/section/1/ For Talend specifically, there is a good picture of how to configure the Talend side here: https://www.talendforge.org/forum/img/members/256670/Capture-decran-2016-03-11-a-12_17_58.png Original conversation here: https://www.talendforge.org/forum/viewtopic.php?pid=173639
... View more
10-11-2017
02:38 PM
2 Kudos
Hi @Ganesh Ganjare NiFi does have a REST API that can be invoked using something like curl for CLI based execution. You can use the API to start process groups and processors. You can do a PUT call on the processor and update the state to RUNNING within the ProcessorEntity String. The Nifi API docs are here (look at Processors) https://nifi.apache.org/docs/nifi-docs/rest-api/index.html There is a good example here on using it for processors: https://community.hortonworks.com/questions/49338/using-the-nifi-rest-api-to-execute-a-data-flow-cre.html
... View more
09-22-2017
06:08 PM
Hi @Sree Kupp There is a great write up here on this topic: https://community.hortonworks.com/articles/65837/migrate-namenode-from-one-machine-to-another-via-a.html Ensure you adequately test this in a non-production environment.
... View more
09-21-2017
04:05 PM
Hi @Dinesh Das There is a feature for setting policies on resources accessed together. You may want to review this post, which walks you though setting it up (please note the API call required to enable the feature): https://community.hortonworks.com/articles/63664/how-to-create-a-ranger-policy-that-prohibits-combi.html
... View more
09-20-2017
08:48 PM
1 Kudo
Hi @Faruk Berksoz That is definitely a very good fit for Minifi and Nifi. There is a good article here by a colleague describing such a use case: https://community.hortonworks.com/articles/67756/ingesting-log-data-using-minifi-nifi.html
... View more
09-20-2017
08:34 PM
Hi @Jacqualin jasmin Yes you should still install MapReduce for the history server, and the client tools. Certain components still use MapReduce under the hood, and having it installed does not usually cause harm or take up too many resources. The history server can be installed on any master node you like and makes sense to do so, while the clients are typically installed on edge (or data) nodes where users will have access.
... View more
09-20-2017
08:26 PM
Hi @Juan Manuel Nieto There is no hard requirement to have the HDF and HDP clusters managed by the same instance of Ambari. There can be pros/cons either way depending on your particular use case. You should be just fine having a separate Ambari instance for HDF, and another Ambari instance for the existing HDP cluster.
... View more
09-19-2017
08:28 PM
Hi @David Williamson You may want to consult your local Hortonworks team for what the best way is for your specific use case. There are many ways to get data into HDP, including raw file uploads via tools, CLI uploads, or even SQL to Hadoop tools like Sqoop. Perhaps the most flexible, and easiest to maintain method is to use Hortonworks Data Flow: https://hortonworks.com/products/data-center/hdf/ Downloading the HDF Sandbox, and going through some of the tutorials online would be a great way to become a little more familiar with it.
... View more
09-19-2017
04:11 PM
Hi @Dhiraj Here is a good write up on using Knox as a reverse proxy for Ambari. https://community.hortonworks.com/articles/78361/configure-knox-to-access-ambari-ui.html
... View more
09-18-2017
07:47 PM
@ajay vembu Awesome, glad to hear. Best of luck with the remainder of the installation.
... View more
09-15-2017
07:14 PM
1 Kudo
Hi @sudi ts We do have some basic guidance on cluster planning in our documentation here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_cluster-planning/content/ch_hardware-recommendations_chapter.html However, for specific application utilization planning, I would recommend getting in touch with your local Hortonworks team and asking for some time with the local solution engineer. The solution engineer can work with you to uncover additional detail that helps with sizing beyond just the data sizing.
... View more
09-15-2017
05:43 PM
2 Kudos
Hi @ajay vembu I haven't seen this yet, are there any details in the ambari log files that would help narrow down the issue? If you want to try again, you can perhaps stop/start the ambari-server. If it happens again, it might be a good idea to clear it and start over: ambari-server stop ambari-server reset, then follow prompts ambari-server start
... View more
09-14-2017
06:56 PM
2 Kudos
Hi @M R I think if you are using the built-in database, which it appears are you trying to use, you likely do not need to install the postgres db manually. Ambari can do this for you. I suspect because you have installed it outside of Ambari, there is a conflict. I would try removing the manual postgres rpm based installation, and re-run the ambari-server setup
... View more
09-14-2017
04:33 PM
1 Kudo
Hi @R M I see from the URL in the screenshot that you are using the HDF Sandbox to do this work? The HDF Sandbox is optimized for the streaming components. You might want to try the HDP sandbox instead for services such as Hive. I'd recommend downloading the latest HDP sandbox, where the services will likely work out of the box.
... View more
09-14-2017
01:51 PM
1 Kudo
@Kenneth Molenda You might want to try the virtual box version instead, I see a lot more issues with the vmware side. The error message you are getting is related to vmware, and not to the hortonworks sandbox. You can try checking to make sure that CPU virtualization is enabled in your BIOS settings, and if not, try again.
... View more
09-13-2017
06:20 PM
1 Kudo
Hi @sagi levy Double check that the delimiters in the Hive View 2.0 were set correctly. I've run into one bug in Hive View 2.0 where the tab delimeter did not work correctly, but does work in Hive View 1, you can try using Hive View 1 as well to see if it is the same issue.
... View more
09-12-2017
03:31 PM
1 Kudo
Hi @Balachandra Pai There is a very good tutorial here showing how to use the Ambari APIs to download client configs: https://community.hortonworks.com/articles/76303/download-client-configs.html
... View more
09-12-2017
03:24 PM
1 Kudo
Hi @ASIF Khan Hadoop 2.6.0 was HDP 2.2.x You can view the chart here near the bottom that shows the HDP versions, Hadoop versions, and component versions: https://hortonworks.com/products/data-center/hdp/
... View more
09-11-2017
03:23 PM
1 Kudo
Hi @Brian Andrus I'm following up internally about the link not working. In the meantime, a slightly older version is working that might get you going again: http://public-repo-1.hortonworks.com/HDP/tools/2.6.0.3/hdp_manual_install_rpm_helper_files-2.6.0.3.8.tar.gz
... View more
09-11-2017
03:22 PM
1 Kudo
Hi @Brendan Smith I've sent an internal inquiry to follow up on this. I'll post back here once I hear back. In the mean time, this one should work: http://public-repo-1.hortonworks.com/HDP/tools/2.6.0.3/hdp_manual_install_rpm_helper_files-2.6.0.3.8.tar.gz
... View more
09-08-2017
09:31 PM
1 Kudo
Hi @April Collins are you using the HDP or HDF sandbox? The reason I ask is because you are using the Ambari port of 8080 in the HTTP request from the HDP Sandbox, but the SSH port of the docker host for the HDF Sandbox of 12122 If you are indeed using the HDF Sandbox, try the HTTP request with the port 18888: http://127.0.0.1:18888 for the welcome screen, and http://127.0.0.1:18080 for Ambari If you are using the HDP Sandbox, try SSH again using port 2222 and see if you can get in there. If so, check to ensure Ambari is up and running with: ambari-server status and if it is not running, try: ambari-server start
... View more