Member since
06-26-2015
505
Posts
129
Kudos Received
114
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
873 | 09-20-2022 03:33 PM | |
2734 | 09-19-2022 04:47 PM | |
1573 | 09-11-2022 05:01 PM | |
1688 | 09-06-2022 02:23 PM | |
2587 | 09-06-2022 04:30 AM |
05-18-2022
04:19 PM
@rafy The extra character in the response message is not a special character. It's just some extra double-quotes. However, when double-quotes appear in a string in JSON they must be escaped. The correct valid JSON would've been: {
"responseMessage": "hdhdhdhdh:dgdgdgdg \"fdfdf\" bbbbbbb",
"result": "failed",
"code": "34"
} The application that's generating those messages should be fixed to generate valid JSON representations. Cheers André
... View more
05-18-2022
04:13 PM
@SAMSAL , If your NiFi cluster was secured only with TLS (no Kerberos and/or LDAP external providers for authentication), you must have generate a client TLS certificate that you can use to authenticate with NiFi and register that certificate in your browser before you can make any calls to it. If you have an external authentication provider configured, you can make a call to POST /nifi-api/access/token, passing username and password as form parameters to perform the authentication. If you're using Postman, this call will save the returned token in a cookie and you'll be able to perform the next calls as usual. Otherwise, is you're using an external script, you can get the returned token and pass that as a bearer token for the subsequent calls. For example: token=$(curl \
-X POST \
-H 'Content-Type: application/x-www-form-urlencoded') \
-d 'username=admin&password=supersecret1' \
"https://nifi.example.com:8443/nifi-api/access/token"
curl \
-X GET \
-H "Authorization: Bearer $token" \
"https://nifi.example.com:8443/nifi-api/processors/d95f5430-0180-1000-ffff-ffff96c5d76f" Cheers, André
... View more
05-18-2022
03:46 PM
@VinayKumar , In the site where you downloaded the parcel, there's also a ".sha1" file that you need to download and save in the same directory where you saved the parcel, with the same permissions (444) and ownership (cloudera-scm:cloudera-scm). IMPORTANT: The "*.sha1" file, however, must be renamed to "*.sha" for it to be recognized. Cheers, André
... View more
05-18-2022
03:40 PM
@OptOut , Could you please provide the link to the doc you followed? André
... View more
05-17-2022
05:36 PM
1 Kudo
@Threepwood , What you're proposing here is something called Stateless NiFi execution. In a normal NiFi flow data is always shared in the form of flowfiles and there's no way to work around that. I/O is a tax that NiFi pays to make it more flexible. It's know to be I/O heavy but it still performs very well and can handle huge volumes of data if you follow the best practices when building your flows. Stateless NiFi execution can be used for a subset of NiFi flows that don't need to store state. In a NiFi Stateless execution, nothing is written to disk. Data is sent from one processor to another through function calls. It can achieve much faster performance this way, but it's limited to stateless flows. Mark Payne has another video where he talks about this and how to use it to achieve Exactly-Once delivery from Kafka to Kafka: https://www.youtube.com/watch?v=VyzoD8eh-t0 Cheers, André
... View more
05-17-2022
05:24 PM
@shrikantbm , How old is the oldest log segment on those partitions? What's the default log retention set for your cluster? Have you set any retention for the __consumer_offsets topic specifically? Cheers, André
... View more
05-16-2022
08:26 PM
1 Kudo
@kotopes , The API endpoint that you're looking for is the same you use to reconfigure pool parameters, but using a GET method instead of PUT: GET /cmf/clusters/cluster-name/queue-manager-api/api/v1/environments/dev/clusters/cluster-name/resources/scheduler/partitions/default/queues/root.pool_name Cheers, André
... View more
05-16-2022
07:22 PM
1 Kudo
@JimAtWork , The endpoint you're looking for is /api/v1/projects/{user}/{project_name}/files/{filename}. For example, you can upload a file to a project with the following curl command: curl \
-u myuser:mypwd \
-X PUT \
"http://cdsw.example.com/api/v1/projects/myuser/myproject/files/myfile.txt" \
-F "files[]=@myfile.txt" Cheers, André
... View more
05-16-2022
04:09 PM
1 Kudo
@Threepwood , Flowfiles in NiFi are always persisted on disk. It doesn't mean, though, that they are always written for every processor. Flowfiles only need to be written when they changed. If they don't NiFi will use the same flowfile on disk across processors. An excessive number of flowfiles, though, does affect performance and it's an anti-pattern to have too many very small flowfiles. It's a best practice to use record-based processor to avoid this. Please check out Mark Payne's YouTube series on NiFi Anti-Patterns for more on this: https://www.youtube.com/watch?v=RjWstt7nRVY&t=302s Cheers, André
... View more
05-16-2022
03:55 PM
@loridigia , The best way to avoid this problem is to configure and use Knox Gateway to access those services. Knox will handle the authentication and will ensure you can use those services without requiring any specific browser configuration. Cheers, André
... View more