Setup HDP on Azure using Cloudbreak

mardiris · ‎06-03-2019

Hello, where I can find updated information upon how to install a small cluster in Azure? Preferably a video. I understand that using cloudbreak looks like the most promising way. After I have my cluster, is it possible to store the parameters and be able to launch the cluster on Azure at will, without having to spend a lot of time configuring the same things? Or even better do you know if there is a way to launch already preconfigured clusters from a blueprint? Say for example that I have a script which automatically provides 5 or 6 VMs on Azure and then launce HDP automatically? Thank you

mardiris · ‎11-16-2017

use <env-var>HADOOP_USER_NAME=${wf:user()}</env-var> in the workflow.xml I found the solution here: https://community.hortonworks.com/content/kbentry/26593/permission-denied-useryarn-accesswrite-oozie-shell.html

mardiris · ‎11-15-2017

hello, I use an oozie shell action to create a file aaaknowledge.txt in /tmp and then copy it to HDFS as per script.sh: echo "`date -u`" >> /tmp/aaaknowledge.txt hadoop fs -copyFromLocal /tmp/aaaknowledge.txt hdfs://sandbox.hortonworks.com:8020/user/maria_dev/oozie/log However the created directory has yarn as owner, thus the user cannot access it. How can I setup user rights for oozie actions?

mardiris · ‎07-24-2017

thanks guys, in my case, it is 192.168.163.134 port 2222 It works, but after I had to download and install again the sandbox

mardiris · ‎07-20-2017

Hello, as a matter of fact, it seems that I cannot use the command line for any Hadoop related commands... I have an HDP 2.5 sandbox and I cannot login as hdfs user. I connect through ssh at 192.168.163.134 (as per my VM display), port 22 when I type: su hdfs I get : su: user hdfs does not exist I tried also port 2222 , but I get access denied when I try to login as root/hadoop I also tried to use the built-in web client (http://192.168.163.134:4200) but I receive : 192.168.163.134 refused to connect. any ideas?

mardiris · ‎06-09-2017

Thank you!

mardiris · ‎06-09-2017

Hello Is there an archive from where I can download a HDP 2.5 sandbox? currently in https://hortonworks.com/downloads/#sandbox I can find only HDP 2.6 thank you

mardiris · ‎06-08-2017

thank you Mark

mardiris · ‎06-08-2017

Hello, I currently use spark submit to execute an SBT created jar. How can I automate the process in order to let it start periodically (i.e. every night or every few hours?) thank you

mardiris · ‎06-08-2017

Hello, I currently use spark submit to execute an SBT created jar. How can I automate the process in order to let it start periodically (i.e. every night or every few hours?) Thank you

mardiris · ‎07-27-2016

Thanx Simon, do you know perhaps of a tool that can enable me to read the ORC files in windows environment (I am not supposed to have access to Linux terminal)? and even hopefully automate the data (I have hundrends of ORC files)?

mardiris · ‎07-27-2016

Hello community, How can I upload ORC files to Hive? I was given an ORC file to import into hive. The upload table functionality in Ambari, which I always used, supports only csv, json and xml. I thought of saving the file in HDFS in the respective database directory and then create table in hive and load the data. Unfortunately, I cannot open the the file (I have only notepad) to see it's schema. Thank you.

mardiris · ‎06-10-2016

Hello @Ryan Cicak I cannot find the calculator either... Also the self paced course from the university has a broken link: "Hortonworks provides some basic help in an online Hadoop Cluster Configuration Guide available at http://hortonworks.com/cluster-sizing-guide/ " 404 error

Online	Offline
Last Visited	‎06-27-2019 04:06 PM

Date Registered	‎06-10-2016 08:27 AM
Last Visited	‎06-27-2019 04:06 PM
Total Messages Posted	18
Total Kudos Received	2

Setup HDP on Azure using Cloudbreak

Re: oozie how to change owner from yarn to user

oozie how to change owner from yarn to user

Re: I cannot manage files on HDFS with the command...

I cannot manage files on HDFS with the command lin...

Re: How can I download sandboxes from past release...

How can I download sandboxes from past releases?

Re: How can I execute Spark jobs periodically?

How can I execute Spark jobs periodically?

How can I execute Spark jobs periodically?

Re: How can I upload ORC files to Hive?

How can I upload ORC files to Hive?

Re: Cluster Sizing calculator