Default Page Size seems to be 200 on most APIs. Use query parameters pageSize and startIndex to page through the results ... View more

Hi Please check http://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.1/index.html Once you enable apache nifi Ranger, you will need to add each user and the node identities in Ranger and apply policies. https://community.hortonworks.com/articles/60001/hdf-20-integrating-secured-nifi-with-secured-range... You can also check: https://community.hortonworks.com/articles/57980/hdf-20-apache-nifi-integration-with-apache-ambarir... http://bryanbende.com/development/2016/08/22/apache-nifi-1.0.0-using-the-apache-ranger-authorizer ... View more

@Shelton Any update on this? looks like it is looking for some java packages  java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path, no leveldbjni-1.8 in java.library.path, no leveldbjni in java.library.path, /var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir/libleveldbjni-64-1-4657625312215122883.8 (Permission denied)] can we install it externally? ... View more

@saivenkatg55    This "Exiting with status 1: java.io.IOException: Problem starting http server" error should be linked to your other question I just have  responded to  https://community.cloudera.com/t5/Support-Questions/Unable-to-start-the-node-manager/td-p/286013 If this is resolved then the java.io.IOException shouldn't occur   HTH ... View more

Follow this https://github.com/ehiggs/spark-terasort ... View more

In /etc/yum.repos.d, remove all .repo files pointing to the Internet and copy only .repo files from other servers which are already using your local repo. For HDP nodes, initially you need only 2 .repo files, one for the OS, and ambari.repo. When Ambari adds a new node to the cluster it will copy there HDP.repo and HDP-UTILS.repo. Also, have you set your repository URLs in Ambari-> Admin-> Stack and versions-> Versions -> Manage Versions -> [click on your current version] ? ... View more

You need to follow these as those are for spark thrift Configuring Cluster Dynamic Resource Allocation Manually To configure a cluster to run Spark jobs with dynamic resource allocation, complete the following steps: Add the following properties to the spark-defaults.conf file associated with your Spark installation (typically in the $SPARK_HOME/conf directory): Set spark.dynamicAllocation.enabled to true . Set spark.shuffle.service.enabled to true . (Optional) To specify a starting point and range for the number of executors, use the following properties: spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.minExecutors spark.dynamicAllocation.maxExecutors Note that initialExecutors must be greater than or equal to minExecutors , and less than or equal to maxExecutors . For a description of each property, see Dynamic Resource Allocation Properties. Start the shuffle service on each worker node in the cluster: In the yarn-site.xml file on each node, add spark_shuffle to yarn.nodemanager.aux-services , and then set yarn.nodemanager.aux-services.spark_shuffle.class to org.apache.spark.network.yarn.YarnShuffleService . Review and, if necessary, edit spark.shuffle.service.* configuration settings. For more information, see the Apache Spark Shuffle Behavior documentation. Restart all NodeManagers in your cluster. ... View more

Thanks it help to resolve issue. I was missing with username principal ... View more

@chennuri gouri shankar This is a known issue with the Ambari version if the database used is MySQL. Manually create the required table by using the following create table statement: CREATE TABLE DS_JOBIMPL_<REPLACE THIS WITH THE NUMBER IN THE ACTUAL TABLE NAME> ( ds_id character varying(255) NOT NULL, ds_applicationid character varying(2800), ds_conffile character varying(2800), ds_dagid character varying(2800), ds_dagname character varying(2800), ds_database character varying(2800), ds_datesubmitted bigint, ds_duration bigint, ds_forcedcontent character varying(2800), ds_globalsettings character varying(2800), ds_logfile character varying(2800), ds_owner character varying(2800), ds_queryfile character varying(2800), ds_queryid character varying(2800), ds_referrer character varying(2800), ds_sessiontag character varying(2800), ds_sqlstate character varying(2800), ds_status character varying(2800), ds_statusdir character varying(2800), ds_statusmessage character varying(2800), ds_title character varying(2800) ); ... View more