Thank you for the explanation! BTW, where do you see the 'Comments'?? I don't have that!   << The comment on the setting in CM should have explained it for you:       Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx.>>   ... View more

I have been using CM API python scripts for adding Hadoop services into a CDH cluster. I would like to add the Spark History Server role by calling a script. Could you please provide me with some samples/links/docs to create it.   Thank you!   ... View more

Actually, here is what I have deployed/confgured for Spark: <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> scm=> select * from services where service_id = 24;  service_id | optimistic_lock_version |  name  | service_type | cluster_id | maintenance_count | display_name | generation ------------+-------------------------+--------+--------------+------------+-------------------+--------------+------------          24 |                      34 | spark0 | SPARK        |         25 |                 0 | spark0       |          1 (1 row)   scm=> select role_type, configured_status, host_id from roles where service_id = 24;   role_type   | configured_status | host_id --------------+-------------------+---------  SPARK_WORKER | RUNNING           |       1  GATEWAY      | NA                |       4  GATEWAY      | NA                |       5  GATEWAY      | NA                |       6  GATEWAY      | NA                |       3  GATEWAY      | NA                |       1  GATEWAY      | NA                |       2  SPARK_WORKER | RUNNING           |       2  SPARK_WORKER | RUNNING           |       6  SPARK_WORKER | RUNNING           |       8  SPARK_WORKER | RUNNING           |       5  SPARK_WORKER | RUNNING           |       7  SPARK_WORKER | RUNNING           |       3  SPARK_MASTER | RUNNING           |       4 (14 rows) <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>   That tells me that the 'Spark History Server' role is not installed. Do I have to install it, and if so how??   Thank you!   ... View more

Thank you for the clarification! ... View more

Hi JM, thank you again!   The issue (under-replicated & corrupt blocks) started when I added 2 new nodes into an existing CDH 5.4 cluster. I went and selectively removed and restored files back into HDFS. HDFS now is HEALTHY.   However, I haven't pinpointed the root cause! I have opened up another thread listing more details about the corrputed blocks issues. I'll close this one amd continue with the other one.   Thanks for all your help.   Happy Mother's Day 🙂   ... View more

Hi JM, that worked!! I am now able to startup HBase with all its servers!   Thanks for your assistance and persistance 🙂   Cheers, TS   ... View more