you can try ALTER TABLE table_name [PARTITION (partition_key = 'partition_value' [, ...])] CONCATENATE; https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionConcatenate ... View more

Did you check if the partition table is creating small files on HDFS. Ideally you would want the files to be close to the HDFS block size. Also, check if the table stats are computed and you are using the right filter key to read only the required partitions. ... View more

You should be able to drop table using phoenix sqlline. https://phoenix.apache.org/language/#drop_table sqlline.py hostname.com /location/of/file/drop_table.sql ... View more

https://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/ Above link has good overview of how YARN works and the algorithms used ( capacity and fair scheduler ) by Resource manager for scheduling. Yarn capacity scheduler config tutorial is available at https://hortonworks.com/hadoop-tutorial/configuring-yarn-capacity-scheduler-ambari/ Does this help? ... View more

@uri ben-ari you need to look at those services that are failing and look at the log file to understand why they are failing and fix those issues. you can always revert back to the working config from the amabri UI. you can also perform restart of all the components required from the Ambari UI under Add Services -> Restart All affected ... View more

@Rohit Khose YARN is resource negotiator for your cluster. Spark ( like other hadoop application) requests YARN for resources specified by the user and if available it will use them. You can enable spark dynamic allocation so the spark application can scale up/down executors depending on the need. https://spark.apache.org/docs/1.6.1/configuration.html#dynamic-allocation ... View more

that log file does not tell much there should be some other stack trace in the log file indicating the actual error. are you running on one node? how much memory have you allocated? what else are you running on that node? ... View more

There could be number of reasons for that error, can you post the actual error for the log file?can you access the api elastic host on port 9200? ... View more

you can look into turning on `spark.dynamicAllocation.enabled` setting, this setting will release any un-unsed executors back to the cluster and request when they are needed link https://spark.apache.org/docs/latest/configuration.html#dynamic-allocation or after you have completed your analysis, you can restart the spark interpreter in zeppelin, due to lazy evaluation zeppelin will only start the spark context when you need it. ... View more

@Geetika Guleria you might consider re-constructing your original data to have all intervals e.g. Ray123 1 2015 2017 -> Ray123 1 2015 2016 and Ray123 1 2016 2017 assign rank based on the other rows by either mapping on the start_end timestamp. e.g. 2015_2016 -> [Ray123] , 2016_2017 -> [Ray123,Kate] but the problem could be to preserve the sequence of the rows, do you have other cols to track that e.g sequence, or id ... View more

@Krishna Kumar you dont need to stop spark context when you are using zeppelin. can you remove sc.stop() line and try again? ... View more