@Masi  Load Balancing related issues are fixed in NiFi 1.10, so you should not be hitting any known issue in this version. On LB queue how many flowfiles you pass at any given point? How many nodes you have?  If you are passing one flowfile and till the time it processed/passed to next processor there is no flowfile in queue then it is possible that NiFi not load balancing the flowfiles at all as it doesn’t have enough flowfiles. Also on load balancing queue you should have an icon representing LB queue, do you see that icon color changing from green to grey? If icon is green means it is load balancing the flowfiles and if grey then it finished the balancing.   If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here.   -Akash ... View more

@Delio It would be bit complicated flow design as direct support for delete events is not available for GetHDFSEvents processor yet. But you can refer this solution, which can help you to move further.   If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here.   -Akash ... View more

@Law  serveraddress is hostname of host where nifi service is installed. E.g you have installed it on samplehost.com then it would be "http://samplehost.com:8080/nifi" .If you enable SSL for nifi later then the URL will change to "https://samplehost.com:<secure_port>/nifi". Also you should check that host and port are reachable from your browser to access NiFi UI.       If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here.   -Akash ... View more

@dzbeda You can use one of below approach. 1. Install minifi on windows machine and send logs from minifi to NiFi [Push]  2. You can use ConsumeWindowsEventLog to get event logs in NiFi [Pull] ... View more

@Delio  What do you mean by "But what about file delete?" Do you want to delete the data at source HDFS as soon as you fetch it in NiFi?  ... View more

@sarithe  It is expected behaviour as Ranger has to pull resources from NiFi and as NiFi is secured it will look for authentication token from Ranger but as Ranger is un-secure it can not present the token and get 403 as response. You should enable SSL [or kerberise the cluster] for Ranger so Ranger can authenticate using SSL [or tgt]certificate. Meanwhile you can use https://community.cloudera.com/t5/Community-Articles/NiFi-Ranger-based-policy-descriptions/ta-p/246586 as reference to setup policies manually.   -Akash ... View more

@kumar79  I am not very sure with your complete use case, if you explain more would be better. Why you want to take date from shell/cron? you can use UpdateAttribute processor with custom property and use NiFi Regular Expression language to set date for you. E.g https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates The attribute later you can use in your flow to satisfy QL condition.   Hope it helps. -Akash ... View more

@scoe74  If it is just token you have to take from SiteA and then using that token you can download data from SiteB then you can use 1st  InvokeHTTP to take the token, add the token as flowfile attribute and then using that token you can use 2nd InvokeHTTP to download the data.   Hope it helps you. -Akash ... View more

@dzbeda  You should extend the Content repo as it will store the content of your flowfiles. Please go through the best practices for setup https://community.cloudera.com/t5/Community-Articles/HDF-CFM-NIFI-Best-practices-for-setting-up-a-high/ta-p/244999   Also note that NiFi should not be used as backup for any failover mechanism as it is meant for processing and not for storage. I would recommend to store the data to may be some centralised location from where you can store and retrieve data as per use case and in case of failure as well you can have backup on that location. NiFi can be used to hold data for a day or 2 but if it is not processed then it will hold data at content repository and can fill the content repo storage. ... View more

Do you see any Warns, Errors for "CN=admin, OU=ApacheNiFi" user in nifi-user.log file when you login to the ui?  ... View more

@Vapper  Looks like there are issues with authorization, can you please check your authorizations.xml file and check if the user you have mentioned in INITIAL_ADMIN_IDENTITY is assigned to all expected policies? ... View more

@fgarcia  Cloudera NiFi Kafka processors should work with all Kafka distributions be it HDP, CDH, CDP, Confluent or Apache Kafka. We generally recommend to use *Kafka_0_10 with Broker version as 0.10 and likewise. Hope it helps 🙂 ... View more

@rstewart  Counter is in-memory content for NiFi service. There is no direct way to get value of particular counter. But you can try below call to get a json response from NiFi about all counters. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/counter" -H "Authorization: Bearer $TOKEN" -H "Connection: close" -k Once you get the Json response you can parse it as per your use case ... View more

@varun_rathinam  You can use MergeContent processor if it fits for your use case. It is better way to handle small file issue in NiFi. Please refer below link for more details. https://community.cloudera.com/t5/Support-Questions/Merge-Content-for-small-content-issue/td-p/167660   BR, Akash ... View more

@abhinav_joshi  NiFi supports " /access/kerberos" for kerberos/spnego negotiation and "/access/token" for username/password to provide token to use with " Authorization: Bearer" in api request. So you can try to use below calls 1. TOKEN=$( curl -k "https://$HOSTNAME:9091/nifi-api/access/token" -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data 'username=xxx@REALM&password=secret' --compressed ) 2. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/flow/status" -H "Authorization: Bearer $TOKEN" -H "Connection: close" and so on. ... View more

@bsivalingam83  This error generally occurs when you copied the flow.xml.gz from another NiFi cluster and tries to load it in new NiFi setup. This happens because the "nifi.sensitive.props.key" from first NiFi setup is different from "nifi.sensitive.props.key" value from second NiFi setup.   If flow.xml.gz is not copied and CFM is 1.0.x version then you may need to check  https://docs.cloudera.com/cfm/1.0.1/upgrading-cfm/content/upgrading-cfm.html to resolve the issue ... View more

Hi Inam, Please find below link for reference : https://community.hortonworks.com/articles/178747/mqtt-with-apache-nifi.html (I have tested it and worked fine for me) Also as you are using psk-identity please check for "use_identity_as_username" value in your mosquitto.conf file. I am not very much sure about psk-identity but you can use mosquitto_passwd to set password for particular user and redirect the password to a file which file can be used for authentication to publish MQTT messages. ... View more