Member since
06-26-2015
515
Posts
138
Kudos Received
114
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2257 | 09-20-2022 03:33 PM | |
| 6009 | 09-19-2022 04:47 PM | |
| 3236 | 09-11-2022 05:01 PM | |
| 3702 | 09-06-2022 02:23 PM | |
| 5770 | 09-06-2022 04:30 AM |
07-31-2022
06:35 PM
@AbhishekSingh , 1. No, the NiFi Toolkit is not mandatory. If you're familiar with TLS certificates and NiFi administration you can set up and manage the cluster without it. The toolkit makes some of the activities easier, though. 2. What do mean by "flowfile change restriction"? Cheers, André
... View more
07-31-2022
06:33 PM
@Domo , I'm not familiar with Cassandra to answer the question. I see, though, that there's a Jira for the upgrade of the Cassandra driver to version 4, but that hasn't got traction yet. You might want to enquire in the Jira itself. Cheers, André
... View more
07-31-2022
06:29 PM
@AbhishekSingh , The JSON datatype in MySQL is stored in an optimized binary format. When you do CDC you only get the binary changes associated with that column. If you are processing this within MySQL, you have functions to deal with the JSON datatype. In NiFi, though, I'm afraid this is not possible. If you want to get the JSON changes as text you should convert the column data type to a string-type. Cheers, André
... View more
07-31-2022
06:23 PM
@samrathal , Are the permissions of your credentials file correct? Ensure the file is owned by the nifi user and that user has permissions to read it. Cheers, André
... View more
07-31-2022
06:21 PM
@KhASQ , Besides @SAMSAL solution, you can also use ReplaceText to eliminate the need of extracting the entire content as an attribute. You'd still have to set a large enough buffer, though, to ensure your largest message could be processed. Cheers, André
... View more
07-31-2022
06:13 PM
@Caliber , Not sure if this is the problem, but beeline has a known issue with terminal settings when running in the background. Try adding this to your script, before the for loop: export HADOOP_OPTS=-Djline.terminal=jline.UnsupportedTerminal Cheers, André
... View more
07-31-2022
06:09 PM
1 Kudo
As @mkohs mentioned, using the RPATH function is the way to go here. It can be used on the SELECT clause, for projection, as well as on the WHERE cause, for filtering. You can see a similar use of it in this post here: https://community.cloudera.com/t5/Support-Questions/QueryRecord-processor-issue-with-nested-JSON/td-p/338556 Cheers, André
... View more
07-31-2022
06:05 PM
@SandyClouds , The CaptureChangeMySQL processor acts as a MySQL replication slave. As such, I believe it requires the REPLICATION_SLAVE privilege in MySQL. Cheers, André
... View more
07-31-2022
05:54 PM
2 Kudos
@kellerj , Cloudera Manager does not manage the registry-aliases.xml file for you but you can create this file manually in a common location (e.g. /etc/nifiregistry/conf/registry-aliases.xml) and configure NiFi Registry to point to that file using the following safety valve configuration in Cloudera Manager: After that, you just need to restart the NiFi Registry service and... voilà! Cheers, André
... View more
07-31-2022
05:20 PM
1 Kudo
@Brenigan , "&" on the RHS returns the value found at the specified path position. When used inside "[]" the meaning is the same and the returned value is used as an index of that array. For example, take this expression: "@1": "outer[&4].inner[&2].t", The different matches for "&" expressions are shown below: [{
"operation": "shift",
"spec": {
"events": {
"*": { <-- &4 matches the position of the item in the events array (e.g. 0, 1, 2)
"transports": { <-- &3 matches "transports"
"*": { <-- &2 matches the position of the item in the transports array (e.g. 0, 1)
"*": { <-- &1 matches the same as &0
"@1": "outer[&4].inner[&2].t", <-- &0 matches the value of the item in the transports array (e.g. PUSH, SMS)
"@(3,eventTypeCode)": "outer[&4].inner[&2].etc"
}
}
}
}
},
"*": "&"
}
}] So, the expression "outer[&4].inner[&2].t" will create an "outer" array that has as many elements as the "events" array. Each element of the "outer" array will have an "inner" attribute whose value will be another array. This array will have as many elements as the respective "transports" array, each one with the "t" and "etc" attributes. Hope this helps. Cheers, André
... View more