You can refer https://community.hortonworks.com/questions/9790/orgapachehadoopipcstandbyexception.html for this issue. ... View more

The binaries downloaded by Ambari will install HDP for sure but you need to delete them and place your custom binaries before running the cluster setup. Thanks, Puneet ... View more

Hi Praveen, Here are a few points to help: 1. Try running your API without options like "--driver-memory 15g --num-executors 25 --total-executor-cores 60 --executor-memory 15g --driver-cores 2" and check logs for memory allocated to RDDs/DataFrames. 2. Driver doesn't need 15g memory if you are not collecting data on driver. Try setting it to 4g rather. I hope u r not using .collect() or similar operations which collect all data to driver. 3. The error needs fine tuning your configurations between executor memory and driver memory. The total number of executors(25) are pretty much higher considering the memory allocated(15g). Reduce number of executors and consider allocating less memory(4g to start with). Thanks, Puneet ... View more

Gaurav, While installing the Hadoop stack through Ambari it fetches install packages from hortonworks repo, even if you install apache Ambari distribution. Check Repo URLs at: https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+2.2.2+from+Public+Repositories However, if you want to experiment with installation below are a few things that could help you: 1. Ambari downloads the binaries to be installed at "/var/lib/ambari-server/resources/stacks/HDP/2.4/services/" directory. 2. You can copy the required services binaries at "/var/lib/ambari-server/resources/stacks/HDP/2.4/services/" on Ambari server machine and try running the installation. NOTE: the approach is not verified but worth a try. Thanks, Puneet ... View more

Could you share more details like command used to execute and input size? ... View more

Could you please share the snippets which reproduces the issue. ... View more

1. For NULL issue u need to map columns in HBase and Hive. See example at below link. https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-ColumnMapping 2. The default value for rowkey is '\002'(Ctrl B) character. ... View more

Answer 2: You can use HBaseStorageHandler for Hive-HBase Integration. Please refer: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz", "hbase.mapred.output.outputtable" = "xyz"); ... View more