Hello, I'm a learner & i would like to use the method you made mentioned here to collect logs in a remote server & send to Nifi. Please, can you put me through because i have been battling with how to build a msi before the real implementation. Thank you so much. ... View more

@dzbeda, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.   ... View more

Hello   You may refer to this thread for the monitoring options for NiFi https://community.cloudera.com/t5/Support-Questions/NIFI-Monitoring-processor-and-nifi-Service/td-p/128485 ... View more

@dzbeda  Have you tried specifying the index in your configured Query within the GetSplunk NiFi processor component? NiFi is not going to be able to provide you a list of indexes from your Splunk to choose from, you would need to know what indexes exist in your Splunk server. Hope this helps, Matt ... View more

Look at everything that has *Record in the name for anything like CSV, JSON, Parquet, AVRO, Logs.   https://www.datainmotion.dev/2020/07/ingesting-all-weather-data-with-apache.html ... View more

I recommend not using nifi and working with the console. Using nifi is not recommended because additional logs are generated. It is recommended to divide and compress the files that you want to move from the console into appropriate sizes and send them. For HDFS, you can use the distcp command.   https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html   ... View more

@dzbeda    Seeing "localhost" in your shared log output leads to what may be the issue. When you configure the URL in the Remote Process Group (RPG), it tells that NiFI RPG to communicate with that URL to fetch the Site-To-Site (S2S) details.  Included in those returned details are things like: - number of nodes in target NiFi cluster (if standalone returns only one host) - The hostnames of those node(s) (in this case it looks like maybe localhost is being returned - Configured RAW port if configured - Whether HTTP transport protocol is enabled - etc... So when you RPG actually tries to send FlowFiles over S2S it is trying to send to localhost which results in itself rather than the actual target linux NiFi it fetched the S2S details from. When some properties are left unconfigured, NiFi returns whatever the OS resolves. I am going to guess your linux server is returning localhost rather than the actual hostname. You will want to verify your S2S configuration setup in the target NiFi (linux server): http://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#site_to_site_properties Try setting the " nifi.remote.input.host" to see if that helps you. Hope this helps, Matt ... View more

Problem solved.  Someone changed system1 UID ... View more

You can use the config API to update a service configuration - Document is in link https://cloudera.github.io/cm_api/apidocs/v11/path__clusters_-clusterName-_services_-serviceName-_roleConfigGroups_-roleConfigGroupName-_config.html   I am quoting an example for service monitor below You can find the list of all roleconfiggroup by running command curl -u 'username:password' http://xxx:7180/api/v14/cm/service/roleConfigGroups/ Next I am updating the heap of service monitor to 650MB curl -u 'username:password' -X PUT -H "Content-Type:application/json" -H "Accept:application/json" -d '{"items":[{ "name": "firehose_heapsize","value": "681574400"}]}' http://xxxx:7180/api/v14/cm/service/roleConfigGroups/mgmt-SERVICEMONITOR-BASE/config Then restarted it curl -X POST -u 'username:password' http://xxxx:7180/api/v14/cm/service/commands/restart ... View more