Hi, We have preemption enabled for all queues in our prod cluster. We need to run a big spark job without preemption enabled. Can someone help me on running a spark job without preemption. ... View more

Hi, We need to create data flow between two kafka clusters (cluster1 and cluster2). We are using Nifi sitting at cluster1 to send the messages from cluster1 kafka to cluster2 kafka. Here we wanted to use the external load balancer in front of cluster2 Kafka. However, it is not working properly. The Nifi is unable to send the data to cluster2 kafka if we use LB url in the kafka brokers section. If we put the actual kafka brokers, it is working properly. Can any one help me on this asap. ... View more

Hi, When dynamic allocation is enabled, most of the times we are facing issues while fetching the blocks RetryingBlockFetcher: Retrying fetch (1/3) for 1 outstanding blocks after 5000ms Error RetyringBlockFetcher: Exception while beginning fetch of 1 outstanding blocks (after 1 retries) java.io.IOException: Failed to connect to <host>:<some port> Caused by java.net.ConnectException: Connection refused: <host>:<some port> We are seeing these errors continuously in the executors when we run a big spark jobs. During this time nothing is being processed and after some time these errors are getting disappeared and the processing gets resumed. This is impacting our job SLAs. Can any one help me on this ... View more