About tono425

tono425 · ‎12-17-2024

@Shelton Thank you for your advice. As I use the latest version of NiFi and it requires Java 21, I added the following line in bootstrap.conf and confirmed the warning messages disappeared. java.arg.EnableNativeAccess=--enable-native-access=ALL-UNNAMED I appreciate your help. Thank you,

tono425 · ‎12-15-2024

@MattWho Thank you for your advice. I installed Java 21 (21.0.5) but similar warnings still persist. Dec 16 11:09:10 myserver nifi.sh[253936]: WARNING: A restricted method in java.lang.foreign.Linker has been called Dec 16 11:09:10 myserver nifi.sh[253936]: WARNING: java.lang.foreign.Linker::downcallHandle has been called by the unnamed module Dec 16 11:09:10 myserver nifi.sh[253936]: WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for this module Thank you,

tono425 · ‎12-12-2024

Hello, Recently I setup NiFi 2.0. Everytime I start NiFi, I get below warning messages in /var/log/messages. Dec 13 12:07:58 myserver nifi.sh[208489]: WARNING: A restricted method in java.lang.foreign.Linker has been called Dec 13 12:07:58 myserver nifi.sh[208489]: WARNING: java.lang.foreign.Linker::downcallHandle has been called by org.apache.lucene.store.PosixNativeAccess in an unnamed module Dec 13 12:07:58 myserver nifi.sh[208489]: WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module Dec 13 12:07:58 myserver nifi.sh[208489]: WARNING: Restricted methods will be blocked in a future release unless native access is enabled Is there any way to supress these messages? Here are product versions. Red Hat Enterprise Linux 8.8 NiFi 2.0.0 Java OpenJDK 23.0.1.0.11 Regards,

tono425 · ‎12-04-2024

Hello, After trials and errors, I succeeded to sort records by the following steps. 1. Convert flow file to json by ConvertRecord processor Convert flow file from parquet format to json format so that we can modify it in next step. 2. Format records by ExecuteStreamCommand processor As double quotation marks outside the {} were preventing us from processing records as json, I removed them and escape characters by sed then sorted records by jq from script. Sample script #!/bin/bash /usr/bin/sed -e 's/\"{/{/g' -e 's/}\"/}/g' -e 's/\\"/"/g' $1 | /usr/bin/jq '. | sort_by(.Src_obj__event_metadata.timestamp)' 3. Convert flow file to parquet by ConvertRecord processor Convert flow file from json format to parquet format. @ywu Your advice was very helpful in resolving the issue. Thanks a lot.

tono425 · ‎11-11-2024

Hello @ywu , Thank you for your advice. Yes, as you noted I don't think JSON_EXTRACT is working as expected in QueryRecord processor. I converted my parquet file to json format and tried with EvaluateJsonPath processor as you advised. When I set $.Src_obj__event_metadata.timestamp as timestamp, EvaluateJsonPath processor terminated by unmatched relationship. When I changed timestamp value to .Src_obj__event_metadata.timestamp, only [] was genarated as output. [] When I changed the value to .Src_obj__event_metadata, the following output was genarated. ["{\"timestamp\":\"2024-11-01T00:23:58.440995\",\"severity\":\"Info\"}","{\"timestamp\":\"2024-11-01T00:23:58.429579\",\"severity\":\"Info\"}","{\"timestamp\":\"2024-11-01T00:23:08.441709\",\"severity\":\"Info\"}","{\"timestamp\":\"2024-11-01T00:23:08.428501\",\"severity\":\"Info\"}","{\"timestamp\":\"2024-11-01T00:23:48.440624\",\"severity\":\"Info\"}"] From these results, it seems we need another consideration to correctly specify timestamp field. Do you have any idea or insights? Regards,

tono425 · ‎11-06-2024

@ywu Thank you for your prompto reply. I tried with that query but unfortunately the same result. Both of 2 queries you provided passed the validation in the processor. Thanks.

tono425 · ‎11-06-2024

@ywu Thank you for the advice. I tried with the query you suggested but the situation has not changed. Thanks.

tono425 · ‎11-06-2024

Hello, I'm triyng to sort record in parquet file like below in order of timestamp. {"Src_obj__event_metadata":"{\"timestamp\":\"2024-11-01T00:23:58.440995\",\"severity\":\"Info\"}","Src_obj__user_data":"{\"message\":\"Message AAA\"} {"Src_obj__event_metadata":"{\"timestamp\":\"2024-11-01T00:23:58.429579\",\"severity\":\"Info\"}","Src_obj__user_data":"{\"message\":\"Message BBB\"} {"Src_obj__event_metadata":"{\"timestamp\":\"2024-11-01T00:23:08.441709\",\"severity\":\"Info\"}","Src_obj__user_data":"{\"message\":\"Message CCC\"} {"Src_obj__event_metadata":"{\"timestamp\":\"2024-11-01T00:23:08.428501\",\"severity\":\"Info\"}","Src_obj__user_data":"{\"message\":\"Message DDD\"} {"Src_obj__event_metadata":"{\"timestamp\":\"2024-11-01T00:23:48.440624\",\"severity\":\"Info\"}","Src_obj__user_data":"{\"message\":\"Message EEE\"} I confugured the following query in QueryRecord processor and it passed validation in the processor. SELECT * from flowfile ORDER BY JSON_EXTRACT(Src_obj__event_metadata, "$.timestamp") ASC But when I run it, it failed with the following error. It seems to fail to find timestamp field. error Could someone please point out what is wrong with my query? Thanks.

Online	Offline
Last Visited	‎12-19-2024 10:37 AM

Member Since	‎10-22-2024 05:41 PM
Last Visited	‎12-19-2024 10:37 AM
Posts	8
Kudos received	6

Cloudera Community

Re: Getting java warning after staring NiFi

Re: Getting java warning after staring NiFi

Getting java warning after staring NiFi

Re: How can I sort record in parquet file?

Re: How can I sort record in parquet file?

Re: How can I sort record in parquet file?

Re: How can I sort record in parquet file?

How can I sort record in parquet file?