@Sanaz Janbakhsh, Check maximum-applications and maximum-am-resource-percent properties in your cluster. Try increasing values for below properties to allow more applications to be running at a time. yarn.scheduler.capacity.maximum-applications / yarn.scheduler.capacity.<queue-path>.maximum-applications Maximum number of applications in the system which can be concurrently active both running and pending. Limits on each queue are directly proportional to their queue capacities and user limits. This is a hard limit and any applications submitted when this limit is reached will be rejected. Default is 10000. This can be set for all queues with yarn.scheduler.capacity.maximum-applications and can also be overridden on a per queue basis by setting yarn.scheduler.capacity.<queue-path>.maximum-applications. Integer value expected. yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent Maximum percent of resources in the cluster which can be used to run application masters - controls number of concurrent active applications. Limits on each queue are directly proportional to their queue capacities and user limits. Specified as a float - ie 0.5 = 50%. Default is 10%. This can be set for all queues with yarn.scheduler.capacity.maximum-am-resource-percent and can also be overridden on a per queue basis by setting yarn.scheduler.capacity.<queue-path>.maximum-am-resource-percent ... View more

By default, processor runs same SQL every 0 seconds. Please set Scheduling > Run Schedule to a higher value and make sure that SQL is created in such a way that upon execution it picks only new/updated records else you will see "already processed" data. Also, if NiFi is clustered then please set Scheduling > Execution to Primary node, else all nodes will run same query and you will see each record being processed "n" times where "n" = No. of nodes in Cluster. ... View more

If you mean Solr on HDFS, the answer is "it depends." If you have a high number of frequent updates to your index, I usually recommend local storage. On the other hand, if your updates are more batch, and not a constant stream, then using HDFS is a convenient option. If you mean installing Solr on HDF, the only supported option and use case is installing Ambari Infra. The Ambari Infra component is Solr under the covers, but it is only supported for use with HDP and HDF components such as Ranger for User Audit records. There's no support to use Ambari Infra for indexing your own data. ... View more