Hi, I am getting below exception, while I try to restart kafka service (which is running over HDF environment- Kafka 0.10.2). I was getting this exception, as I had the disk space consuming 100% of the log directory and Kafka service went down because of that. As, I did cleared the space for the logs and restarted the kafka service, I was getting the below exception: 2017-09-09 16:56:36,751] FATAL (kafka.Kafka$) java.lang.ClassNotFoundException: org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at kafka.utils.CoreUtils$.createObject(CoreUtils.scala:174) at kafka.metrics.KafkaMetricsReporter$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:65) at kafka.metrics.KafkaMetricsReporter$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:64) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34) at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:64) at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:27) at kafka.Kafka$.main(Kafka.scala: 6] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable) java.lang.NumberFormatException: For input string: "logs.bk" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229) Any ideas here, like how can I overcome this issue? Thanks in advance, Sravanthi ... View more

Hi Everyone, I have a issue with Kafka here. On a cluster I have installed Zookeeper-client and Zookeeper server and are up and running. Along the same I have Kafka-server up and running and on Ambari it shows the health as good and fine. But, while I do execute the producer command for kafka as below, I am getting the below error: Kafka Producer Command: bin/kafka-console-producer.sh --broker-list kafka.broker:6667 --topic testTopic Error Message: [2017-09-05 12:22:02,638] ERROR Error when sending message to topic testTopic with key: null, value: 8 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for testTopic-0: 1535 ms has passed since batch creation plus linger time e 345[2017-09-05 12:22:05,439] ERROR Error when sending message to topic testTopic with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for testTopic-0: 1536 ms has passed since batch creation plus linger time Not getting the actual issue here. Any help will be appritiated. Thanks, Sravanthi ... View more

Hi, I am running a spark job which I have ran the command in yarn cluster mode. But, while running the job I got the below logs and it was in hung state printing the same set of messages: 17/09/01 09:58:17 INFO Client: Application report for application_1503983924258_0018 (state: ACCEPTED) 17/09/01 09:58:18 INFO Client: Application report for application_1503983924258_0018 (state: ACCEPTED) running-spark-job-issue.png Below is the spark command that I am running: spark-submit --class com.spark.sparkStreaming --master yarn --deploy-mode cluster SparkStreaming-0.0.1-jar-with-dependencies.jar --executor-memory 16g --driver-memory 4g --executor-cores 4 --num-executors 8 Below is the spark server configuration: Memory ranges from 64GB - 256GB and CPU is 16 cores. Thanks, Sravanthi ... View more

Hi, We have installed Hive in Hortonworks environment (within cluster). The Hive service is shown as started up and running in Ambari UI, but could not able to access Hive UI. Below is the error it shows for Hive view: hive-view-error-in-ambari-ui.png Tried opening Hive from command line even, but the same error: hive-command-line-error-with-tez-engine.png We tried changing the Tez engine with mapreduce engine and restarting ambari-server, but seems the issue still persist. Also, we do get HDFS Block health alert in Ambari UI. Below is the error: ambari-error-for-hdfs-health.png Not sure, of how to resolve this issue. Hence, requesting for any open suggestion. Thanks, Sravanthi ... View more

Hi, Following is the issue we are facing when we are trying publish data into Kafka topic using PublishKafka0.10.1.2 processor of Nifi 1.2.0 in HDF 3.0.1.0-43 : Error: Failed to process session due to org.apache.nifi.processor.exception.FlowFileHandlingException. Transfer relationship not specified. Attached is the Image for a more clarified detail about the error. I have got this error in the following scenarios: 1)When I check in the checkbox for both success and failure in Automatically Terminate Relationships. 2)When I check in the checkbox for success and route the data to a local file for failure in Automatically Terminate Relationships. processorconfiguration.png Are we missing out any relationships apart from failure and success?How to fix this?Any Suggestions for would be highly appreciated. ... View more