I'm not sure what exactly do you want to do. What do you mean by "related parameters"? If you want to save all cluster configuration properties before the upgrade you can extract a blueprint curl -u admin:admin -H "X-Requested-By: ambari" http://<ambari-server-fqdn>:8080/api/v1/clusters/:clusterName?format=blueprint and convert its json output to csv and import in Excel. You can do the same after the upgrade and compare output. Some people do that. However, you can check all config property changes using the "History" feature of Ambari by comparing config versions before and after the upgrade for each service. As for the upgrade itself just follow the Ambari Upgrade guide (and don't skip any step!). ... View more

This error means that something is wrong with your Derby database. Can you check is it created? The path do Derby files is given by "Oozie Data Dir" (oozie_data_dir) in Ambari->Oozie->Oozie server and OOZIE_DATA in oozie-env.sh. Check permissions to that path and retry. You can also try to create the DB manually if ooziedb.bat is available, but better go through Ambari. ... View more

Okay, great, yes, the error was about "no channel configured". Regarding the path in hdfs, I edited my answer to include the full path in hdfs including the Name node: hdfs://sandbox.hortonworks.com:8020/user/Revathy/Flume/%y-%m-%d/%H%M/%S. It's good to organize your folders in HDFS in some way, here I put your home directory in HDFS. How do one know that the Flulme agent works? Well, if it keeps on running, there are no errors in logs, and if data written to sinks is as expected. You can find a lot of details here. You can also run Flume from Ambari, in which case Ambari will let you know whether Flume process in healthy and running. However, one still has to incepct sinks to be sure. ... View more

Hi @Ran Postar You can reduce "Minimum user ID for submitting job" (min_user_id) in yarn-env in Ambari->Yarn from default 1000 to a smaller value, for example 500. The value is referenced as min.user.id={{min_user_id}} in container-executor.cfg.j2 and it should work. ... View more

Check the following two lines in your sink block source_agent.sinks.avro_sink.hdfs.filetype = Datastream source_agent.sinks.avro_sink.hdfs.a1.sinks.k2.hdfs.path = /Revathy/Flume/%y-%m-%d/%H%M/%S in the first one capitals are not correct, and in the second one the property name on the left side is incorrect. Change them and retry: source_agent.sinks.avro_sink.hdfs.fileType = DataStream source_agent.sinks.avro_sink.hdfs.path = hdfs://sandbox.hortonworks.com:8020/user/Revathy/Flume/%y-%m-%d/%H%M/%S ... View more

Hi Adi, the threshold means that the utilization of storage on each node after balancing will be (ACU +- threshold) where I use ACU to denote "average cluster utilization". Example: (1) Before adding new nodes: Let's say you have 10 data nodes, each has capacity of 20T, and your data size is 100T. In this case ACU=50% and if all nodes are perfectly balanced, each stores 10T of data. (2) After: Let's say you add 4 large nodes, each with capacity of 50T, and you still have 100T of data. Your total capacity is now doubled to 400T, and therefore ACU=25%. However, your new nodes are empty. Running the balancer with threshold th=10% will ensure that utilization of all nodes is between ACU-th and ACU+th, in this case between 15% and 35%. We are starting with old nodes at 50% and new nodes at 0% of utilization. Balancer will keep on moving data until old nodes' utilization is <= 35% and new nodes' utilization is >= 15%, which means old nodes keeping less than 20*0.35=7T and new nodes keeping more than 50*0.15=7.5T. As you can see, in this particular case data-per-node amounts are not so far away from each other, but as you keep on adding more data the differences will grow up little by little. If you are interested in more details about the balancer, please refer to HADOOP-1652 and the Balancer design document. ... View more

Are you on CentOS/RHEL-7? I did recently two upgrades, one using Ambari-2.2.0 and one using 2.2.1.1 (both from HDP-2.2.x) and I had no issues. However, both were on RHEL-6.x ... View more

Well, I thought my answer of Apr. 5 ... 🙂 ... View more