Member since
02-01-2022
288
Posts
103
Kudos Received
60
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1266 | 05-15-2025 05:45 AM | |
| 5297 | 06-12-2024 06:43 AM | |
| 8296 | 04-12-2024 06:05 AM | |
| 6138 | 12-07-2023 04:50 AM | |
| 3417 | 12-05-2023 06:22 AM |
05-12-2023
09:33 AM
@rafy I got the same issues when I tried to create a flow using RPATH. However, here is a solution i found to dial into the data array and match on the room = A: SELECT * FROM FLOWFILE WHERE room = 'A' I used QueryRecord With JSONTreeReader (see below) and JSONRecordSetWriter (default). NiFI Flow Definition here: @gitHub Screenshots:
... View more
05-12-2023
08:40 AM
Can you please show the source json in a : code box Also, try a manual test, without attribute, such as: SELECT *
FROM FLOWFILE
WHERE RPATH(data, '/room') = 'A' This will ensure the query is correct to the json payload. Once that works, start testing adding the attribute.
... View more
05-12-2023
06:59 AM
@rafy I would try this: SELECT * FROM FLOWFILE WHERE RPATH_STRING(data, '/room')='${ip}' Assuming ip is an attribute (${ip}) ofcourse.
... View more
05-11-2023
05:42 AM
1 Kudo
@ushasri I believe the solution here is to use the Record based processors with a specified schema. This allows you to provide the correct schema to the Reader and the Writer regardless of the field names in original excel data source.
... View more
05-10-2023
05:56 AM
@zzeng Great article. Reach out to me on internal channels. I would love to show you my oracle to kudu demo, using kafka and schema registry.
... View more
05-10-2023
05:42 AM
1 Kudo
@SandyClouds ^^ that is how you do it. One important thing to mention. That processor is not meant to be deployed in a set it up and forget it situation. CDC against another database requires a lot more technical attention to how the data is changing over time AFTER you grab data for first time. In my opinion, this processor is meant to be a conversation starter, or a way to take a 1 time shot of data source, where you may be watching it run, but not expecting it to run indefinitely and keep 2 systems synced.
... View more
05-10-2023
05:24 AM
@mwblee HDP is no longer supported platform. You cannot access the final HDP artifacts without cloudera subscription. I would highly recommend that you take a look at CDP and modern supported versions of original HDP components.
... View more
05-08-2023
07:24 AM
@sridharavulapat You should be able to find the required hive values in hive-site.xml file. You can get this file from Cloudera Manager. Additionally, use Cloudera Manager to download hive drivers and get the fully qualified jdbc url.
... View more
05-04-2023
05:47 AM
1 Kudo
@ushasri You should check out NiFI Registry: https://nifi.apache.org/registry.html Using NiFi Registry you are able to version control flows during development. Using same Nifi Registry you can now deploy these flows to other environment(s) such as your Cloudera licensed version (HDF,CDF,CFM). Additionally, XML templates are going away. In modern versions of nifi, you should use the Create Flow Definition, and transfer these JSON definition files manually between environments and/or nifi developers. Other CI/CD DFLC (Data Flow Lifecycle) concepts for deploying flows across environments are using NIFI CLI API programmatically to do manual deployment operations. We also often seen some level of integration between Github and NiFi Registry.
... View more