I created a dashboard in Banana and saved the dashboard to solr. It works fine. I can see the entry in solr banana-int collection. But when I try to access the saved dashboard from Banana, I get a blank page and some errors in browser debug console. The url to saved dashboard looks like below http://<SolrInstance>:8983/solr/banana/index.html#/dashboard/solr/test_dashboard?server=%2Fsolr%2F I am unable to understand what is going wrong and I could get much information from documentation what to check when blank screen is seen for saved dashboard. Can I get help to resolve the issue. Thanks. ... View more

Hi, I would like to know where the email notifications created from Ambari are stored into and how can we access them. The goal is to use same notification configuration defined in ambari to send other custom notifications from another application. Can anyone help me to understand the above. Thanks, Anil. ... View more

I am trying to develop an usecase with metron. The end-to-end usecase is, we need to calculate the count of number of events happened for each ip’s. Based on the count reaching above the threshold count generate an alerts in the merton alerts-UI. For that, I have the events data coming to “indexing” kafka topic. I created a profile in ${METRON_HOME}/config/zookeeper/profiler.json as below. { "profiles": [ { "profile": "demo_iplogon_failed", "foreach": "ip_address", "onlyif": "source.type == 'demo_windowsnxlog' and event_id == '4625'", "init": { "count": "0" }, "update": { "count": "count + 1" }, "result": { "profile" : "count", "triage": { "logon_failed_count": "count" } } } ]} After reloading the zookeeper config files, I see the indices created in ElasticSearch with format profile_index_yyyy.MM.dd.HH and the snapshot of record is in attached snapshot I see all the records in the profiler_index_* pattern have field is_alert=”true”. But I don’t see these records turn up as alerts in the metron-alerts UI. I would like to figure out the reason for alerts not turning up in the metron-alerts UI ? I am seeing is_alerts=”true” for all the records under profiler_index_*. How can I configure to set is_alert=”true” only when the count exceeds the threshold value ? @nallen @asubramanian: tagging you as you can help me quickly. Please don't mind! ... View more

I am trying to convert the epoch in milliseconds to TimeStamp of specific timezone. I tried using "format" of nifi expression language as below in updateAttribute processor ${closed_epoch:format("yyyy", "GMT")} But updateAttribute processor is showing error saying "unexpected token ','. Is it the bug with Nifi expression language? and what is the timezone that Nifi uses by default when I try ${closed_epoch:format("yyyy")}? Is there any other way to convert epoch to timestamp of specific timezone ? ... View more

I would like to know if there is a way to encrypt the non-empty HDFS directory using RangerKMS?? I am getting below exception while trying to create encryptZone for non-empty directory. ************** hdfs crypto -createZone -keyName demokey -path /user/demo RemoteException: Attempt to create an encryption zone for a non-empty directory. *************** ... View more

Hi, I am observing the below behavior when there huge number of flow files are in processing by the data flow. All the nifi processors in the data flow are executing once per 10 secs. For example: I have a updateAttribute processor which is configured to perform 1 concurrent task. I see that it is processing 6 records per minutes even though there are thousands of flow files in the queue. At the start of loading huge data, the flow works fine. It processes thousands of records in just seconds. But I see the above kind of behavior after sometime. I am unable to understand what is happening in the backend of nifi that is causing for this kind of behavior. Can I get some help to understand/troubleshoot the issue. Thanks. ... View more

Hi, I am facing an issue with consumeKafka_0_10 processor in my local nifi instance. I have seen the Jira created for my issue and was resolved long time back. So, I believe my local nifi instance does not have the fix for the ConsumeKafka_0_10 processor. I would like to know 1) where can I get the latest nar file for consumeKafka_0_10 processor 2) what is the process to update the consumeKafka_0_10 processor with latest in my local nifi instance ... View more

Hi, I am trying to read the data from kafka topic on the cluster where kerbaros is enabled. I am getting errors like below for all the topics that I am trying to read the data from. ***************Error logs******************* 2017-08-17 06:18:26,505 INFO [test-consumer-group_storagev4-1c-1502965058303-e4ac2072-leader-finder-thread] kafka.consumer.ConsumerFetcherManager [ConsumerFetcherManager-1502965058340] Added fetcher for partitions ArrayBuffer() 2017-08-17 06:18:26,599 WARN [test-consumer-group_storagev4-1c-1502965084565-3028bb7a-leader-finder-thread] k.c.ConsumerFetcherManager$LeaderFinderThread [test-consumer-group_storagev4-1c-1502965084565-3028bb7a-leader-finder-thread], Failed to find leader for Set([demo_unix_indexing,0]) java.lang.NullPointerException: null 2017-08-17 06:18:26,599 INFO [test-consumer-group_storagev4-1c-1502965084565-3028bb7a-leader-finder-thread] kafka.consumer.ConsumerFetcherManager [ConsumerFetcherManager-1502965084750] Added fetcher for partitions ArrayBuffer() 2017-08-17 06:18:26,710 WARN [test-consumer-group_storagev4-1c-1502965058303-e4ac2072-leader-finder-thread] k.c.ConsumerFetcherManager$LeaderFinderThread [test-consumer-group_storagev4-1c-1502965058303-e4ac2072-leader-finder-thread], Failed to find leader for Set([indexing,0]) java.lang.NullPointerException: null 2017-08-17 06:18:26,710 INFO [test-consumer-group_storagev4-1c-1502965058303-e4ac2072-leader-finder-thread] kafka.consumer.ConsumerFetcherManager [ConsumerFetcherManager-1502965058340] Added fetcher for partitions ArrayBuffer() 2017-08-17 06:18:26,805 WARN [test-consumer-group_storagev4-1c-1502965084565-3028bb7a-leader-finder-thread] k.c.ConsumerFetcherManager$LeaderFinderThread [test-consumer-group_storagev4-1c-1502965084565-3028bb7a-leader-finder-thread], Failed to find leader for Set([demo_unix_indexing,0]) java.lang.NullPointerException: null ******************** I have done the necessary configurations but I could not able to figure out the cause of the issue. Note: I am able to read the data from kafka topic with consumeKafka processor. Nifi details: Nifi is running on standalone single node. Zookeeper and kafka are running on 3 node cluster. ***********jass file configiuration******* Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/nifi.keytab" storeKey=true useTicketCache=false principal="nifi@TESTSSTECH.LOCAL"; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true renewTicket=true serviceName="kafka" useKeyTab=true keyTab="/etc/security/keytabs/nifi.keytab" principal="nifi@TESTSSTECH.LOCAL"; }; ************************** ... View more

I would like to extract the property "feedts" from the flowfile JSON using EvaluateJSONProcessor. JSON is as below **********************JSON**************** {"latestFeedJobExecutionContext":{"demo.msexchange":{"jobExecutionId":87,"startTime":1501760142595,"endTime":1501760304816,"executionContext":{"feedts":"1501760190824"}}},"feedName":"test.test_trigger_data_transition","feedId":"474a06bc-7d47-4404-9935-7f7edea55979","dependentFeedNames":["demo.msexchange"] ********************* I would like to know how can I overcome the character "." in the property name. In this case I have its "demo.msexchange". Is there anyway I can extract the property "feedts" from above JSON. ... View more