Member since
06-26-2015
515
Posts
138
Kudos Received
114
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2271 | 09-20-2022 03:33 PM | |
| 6048 | 09-19-2022 04:47 PM | |
| 3260 | 09-11-2022 05:01 PM | |
| 3734 | 09-06-2022 02:23 PM | |
| 5824 | 09-06-2022 04:30 AM |
03-30-2022
03:58 AM
@chitrarthasur , The processor that you're using is not the ListenHTTP processor, but the HandleHttpRequest, correct? When you use HandleHttpRequest, you must always connect that processor to a HandleHttpResponse processor so that NiFi can send a response back to the client. With this you can ingest the content of the request, process it in NiFi and send a response according to the results of your processing. If your only requirement is to ingest the content of the HTTP request, you should use a ListenHTTP instead. Cheers, André
... View more
03-30-2022
02:14 AM
1 Kudo
@PabloO , In Kafka's terminology, a topic is a "distributed logs". The data for each topic's partitions is saved in what's called "log segment files". So, the "log.dirs" and "log.dir" parameters point to the directories where the actual messages are saved, *not* the "error logs". For example, if your "log.dirs" is set to "/data1" and you have a topic named "mytopic". The data for the partition 0 of that topic will be saved in files under the directory "/data1/mytopic-0". Cheers, André
... View more
03-29-2022
06:00 PM
Hi @inyongkim , At the moment this connector has no controls to adjust the flushing mechanism. We're aware of that and Cloudera is working on making that more configurable so that it does not create a small file problem in your destination cluster. Cheers, André
... View more
03-29-2022
05:50 PM
1 Kudo
@PabloO , The logs are written to the directories configured in the log.dirs and/or log.dir properties of the Kafka broker. You can modify those properties to configured to the broker to use the directory that you want. Cheers, André
... View more
03-29-2022
05:39 PM
1 Kudo
Hi @ep_gunner , You can use the QueryNiFiReportingTask reporting task to report on the status of any processors, connections, process groups, etc. This reporting task allow you to use a SQL query NiFi status. For example, if you're interested in any stopped processor, you can use the query: select *
from PROCESSOR_STATUS
where runStatus = 'Stopped' You could also filter that by processor name or id if you need to be more specific: select *
from PROCESSOR_STATUS
where runStatus = 'Stopped'
and <YOUR_FILTER>
where YOUR_FILTER could be any of the below:
id = '<YOUR_PROCESSOR_ID>'
groupId = '<YOUR_PROCESSOR_GROUP_ID>'
name = '<YOUR_PROCESSOR_NAME>'
processorType = '<YOUR_PROCESSOR_TYPE>'
or any combination of the above The results of this reporting task can be forwarded to any available ReportSink in NiFi (Database, Kafka, Email, etc). You can also send the results to a NiFi Input Port using the SiteToSiteReportingRecordSink and them you can process those reports with NiFi itself and take any other action you want. Cheers, André
... View more
03-29-2022
04:44 PM
@sheep , You need to add an EvaluateJsonPath processor before your PutHBaseJson to extract the value that you need and save it as an attribute in the flowfile. For example, you could get the value from $.field1.nestedfield and save that as the attribute mynestedfieldvalue. You can then refer to that attribute in your PutHBaseJson processor as ${mynestedfieldvalue}. Please check out this other answer to a similar question: https://community.cloudera.com/t5/Support-Questions/Hash-key-value-missing-putdynamodb-nifi/m-p/339622/highlight/true#M233139 Cheers, André
... View more
03-29-2022
03:54 PM
Hi @templarian , You can use something like the example below: Cheers, André
... View more
03-27-2022
03:42 PM
@mystefied_ , You can download the CDP Trial version from the Cloudera website below: https://www.cloudera.com/downloads/cdp-private-cloud-trial/cdp-private-cloud-base-trial.html Cheers, André
... View more
03-27-2022
03:39 PM
1 Kudo
@bdworld2 , We currently don't support any docker-based sandbox/quickstart clusters. The one available in GitHub is of a very old CDH version. The best approach is to use a CDP Trial installation if you want to evaluate the latest product. Please check the links below for more information: https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-trial-download-information.html https://community.cloudera.com/t5/Community-Articles/How-to-create-a-Centos7-CDP-DC-Base-VM-for-sandbox-learning/ta-p/300837 Cheers, André
... View more
03-27-2022
03:21 PM
@azg , Yes, you can do this with the ListenFTP processor. Please take a look at the documentation pages below. They explain quite clearly how this processor works. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.0/org.apache.nifi.processors.standard.ListenFTP/additionalDetails.html https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.0/org.apache.nifi.processors.standard.ListenFTP/ Cheers, André
... View more