@Jona  Great article...may i ask can you use "custom spark2-metrics-properties" in spark config instead of advanced spark2-metrics-properties? ... View more

@saqie,  as this thread was marked 'Solved' in 2018, you would have a better chance of receiving a useful response by starting a new thread. This will also provide you with the opportunity to provide details specific to your issue that could aid others in providing a more tailored answer to your question. ... View more

Hi All,   I am trying to connect from Local machine to kafka cluster(kerberized Cluster) through python. Can anyone help what are the properties to specify for the krb5.conf file and other properties.   your help is appreciated. ... View more

@Tzaco Are you trying to download from this page?   https://www.cloudera.com/downloads/hortonworks-sandbox.html ... View more

Excellent article @Pedro Andrade ! ... View more

@Jonathan Sneep Could you please check if the below metrics queries are correct : Driver Memory Driver Heap Usage aliasByNode($application.driver.jvm.heap.usage, 1) Driver JVM Memory Pools Usage aliasByNode($application.driver.jvm.pools.*.used, 4) Executor & Driver Memory Used aliasByNode($application.*.jvm.heap.used, 1) Executor Memory Used aliasByNode(exclude($application.*.jvm.heap.used, '.driver.jvm.heap'), 1) alias(sumSeries(exclude($application.*.jvm.heap.used, '.driver.jvm.heap')), 'total') Task Executor Active Tasks Per Executor aliasByNode(summarize($application.*.executor.threadpool.activeTasks, '10s', 'sum', false), 1) Completed Tasks per Executor aliasByNode($application.*.executor.threadpool.completeTasks, 1) Completed Tasks/Minute per Executor aliasByNode(nonNegativeDerivative(summarize($application.*.executor.threadpool.completeTasks, '1m', 'avg', false)), 1) Read/Write IOPS Read IOPS alias(perSecond(sumSeries($application.*.executor.filesystem.hdfs.read_ops)), 'total') aliasByNode(perSecond($application.*.executor.filesystem.hdfs.read_ops), 1) Write IOPS alias(perSecond(sumSeries($application.*.executor.filesystem.hdfs.write_ops)), 'total') aliasByNode(perSecond($application.*.executor.filesystem.hdfs.write_ops), 1) HDFS Bytes Reads/Writes Per Executor Executor HDFS Reads aliasByMetric($application.*.executor.filesystem.hdfs.read_bytes) Executor HDFS Bytes Written aliasByMetric($application.*.executor.filesystem.hdfs.write_bytes) Also please let me know the queries for the below : HDFS Read/Write Byte Rate HDFS Read Rate/Sec HDFS Write Rate/Sec Looking forward to your update regarding the same. ... View more

@ Jonathan Sneep Thank you so much for help it is working now..... ... View more

Yes, you are right, That is what i meant. I haven't played with the sandbox, but the key would be to make sure that the user NiFi uses to run has access and permissions to the resources you are adding to the processor. Executestreamprocessor will have the same issue if the path is wrong or non existent in the server. I would first try to find out what user is used to run nifi (ps -ef would be your friend here). Then i would make sure that that user has access to the path in the console. (Use 'ls -l path') from the home directory. Path in this case would be both the path of the executable, and the path of the working directory (Make sure both are accessible) Lastly, try to execute your script from the command line. Thanks! Regards ... View more

@Michael Bronson Did it work OK without the underscore? I'm curious and hope I didn't make you go through making a new lab for nothing 🙂 let me know if you still have any errors. ... View more