Member since
11-17-2021
1167
Posts
262
Kudos Received
30
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 444 | 04-23-2026 02:02 PM | |
| 1042 | 03-17-2026 05:26 PM | |
| 6505 | 11-05-2025 10:13 AM | |
| 1223 | 10-16-2025 02:45 PM | |
| 2082 | 10-06-2025 01:01 PM |
09-27-2024
10:22 AM
1 Kudo
@Shalexey Your query does not contain a lot of detail around your use case, but i'll try to provide some pointers here. NiFi processor components have one or more defined relationships. These relationships are where the NiFi FlowFile is routed when a processor completes its execution. When you assign a processor relationship to more then 1 outbound connection, NiFi will clone the FlowFile how ever many times the relationship usage is duplicated. So looking at dataflow design shared, I see you have what appears to be "success" relationship route twice out of the UpdateAttribute processor (this means the original FlowFile is sent to one of these connections and a new cloned FlowFile is sent to the other connection). So you can't simply route both these FlowFiles back to your QueryRecord processor as each would be executed against independently. If I am understanding your use case correctly you ingest a csv file that needs to be updated with an additional new column (primary key). The value that will go into that new column is fetched from another DB via the ExecuteSQLRecord processor. Problem is that the ExecuteSQLRecord processor would overwrite your csv content. So what you need to build is a flow that get the enhancement data (primary key) and adds it to the original csv before the putDataBaseRecord processor. Others might have different solution suggestions, but here is one option that comes to mind: MattWho_0-1727456293238.png GetFile --> Gets the original CSV file UpdateAttribute --> sets a correlation ID (corrID = ${UUID()}) so that when FlowFile is cloned later both can be correlated to one another with this correlation ID that will be same on both. ExecuteSQL --> query max key DB QueryRecord --> trim output to just needed max key ExtractText --> Extract the max key value from the content to a FlowFile Attribute (maxKey). ModifyBytes --> set Remove all Content to true to clear content from this FlowFile (does not affect FlowFile attributes. MergeContent - min num entries = 2, Correlation Attribute name = corrID, Attribute strategy = Keep All Unique Attributes. (This will merge both FlowFiles original and clone with same value in FlowFile attribute "corrID" into one FlowFile containing only the csv content) UpdateRecord --> Used to insert the max key value from the max key FlowFile attribute into the original CSV content. (Record reader can infer schema; however, record writer will need to have a defined schema that includes the new "primary key" column. Then you will be able to add a dynamic property to insert maxkey flowfile attribute into the "primaryKey" csv column. MattWho_1-1727457156154.png PutDatabaseRecord --> write modified csv to destination DB. Even if this does not match up directly, maybe you will be able to apply the NiFi dataflow design concept above to solution your specific detailed use case. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
09-26-2024
08:02 AM
I finally had success using HTTP transport. My .odbc.ini looks something like this: ``` [ODBC] ; Specify any global ODBC configuration here such as ODBC tracing. [ODBC Data Sources] Impala = Cloudera ODBC Driver for Impala [Impala] Driver=/opt/cloudera/impalaodbc/lib/universal/libclouderaimpalaodbc.dylib Description=Cloudera Impala ODBC Driver DSN Host=[datahub-name]-master0.hiaa-cdp.uvmh-kdle.a4.cloudera.site Port=443 Schema=default AuthMech=3 UseSASL=1 SSL=1 TransportMode=http httpPath=[datahub-name]/cdp-proxy-api/impala ```
... View more
09-26-2024
01:14 AM
1 Kudo
Hi @szhao , based on the symptoms this may be a problem within the ODBC driver. Can you please open a support ticket through the MyCloudera support portal so that our support team can forward these information to our driver team? Thanks, Miklos
... View more
09-25-2024
08:35 AM
1 Kudo
@Twelve @aLiang The crypto.randomUUID() issue when running NiFi over HTTP or on localhost has been resolved via https://issues.apache.org/jira/browse/NIFI-13680. The fix will be part of next release after NiFi-2.0.0-M4. Thanks, Matt
... View more
09-23-2024
04:25 PM
@Ivoz Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
09-18-2024
04:27 PM
1 Kudo
@meow00110 Welcome to the Cloudera Community! To help you get the best possible solution, I have contacted our Security team on your behalf and will be in contact soon. Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
... View more
09-18-2024
11:29 AM
Here are some highlights from the month of August
Upcoming Events
DianaTorres_11-1726683527742.png
Upcoming on October 30th: ClouderaNOW: Master the Art of Delivering Trusted Data for Business Insight and Enterprise AI
DianaTorres_12-1726683538181.png
Upcoming on September 26th: WEBINAR
Navigating the Future of Data Management and Analytics
Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details!
Top Activities
511 new support questions
1249 new members
Our Leaderboads
We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions.
See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page.
DianaTorres_14-1726683849992.png
@MattWho @vaishaakb @Scharan @vamsi_redd @vats @SAMSAL @Shelton @Juanes @yagoaparecidoti
Unanswered Questions
Share your expertise and answer some of the below open questions. Also, be sure to bookmark the unanswered question page to find additional open questions.
Unanswered Community Post
Components/ Labels
impala forces full table scan
Apache Impala
Migrating NiFi Registry from one server to another
Apache NiFi NiFi Registry
NiFi PutSlack Proxy error
Apache NiFi
kafka + undertsanding better the kafka controller.log logs
Apache Kafka
Custom Utilization Report
Cloudera Data Platform (CDP)
Top Trending Labels
DianaTorres_15-1726684124312.png
... View more
09-17-2024
11:19 AM
@moonspa Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
09-11-2024
06:17 AM
@moshell281 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more