Member since
11-18-2021
6
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1190 | 12-15-2021 05:23 AM |
02-17-2022
09:05 AM
I have a process that uses the PutBigQueryBatch processor, in which I would like it to truncate the table before inserting the data. I defined an AVRO schema, and previously created the table in BigQuery specifying how I wanted the fields. I am aware that if I change the "Write Disposition" property to the value "WRITE_TRUNCATE", it will truncate the table. However, when I use this option, the schema of the table in BigQuery ends up being deleted, which I would not like to happen, and a new schema is created to record the data. I understand that the "Create Disposition" property exists, and that if the "CREATE_NEVER" option is selected, the schema should be respected and not deleted. When I run this processor with the "Write Disposition" property set to "WRITE_APPEND", the schema I created in BigQuery is respected, but with the "WRITE_TRUNCATE" not. Is there any way to use the "WRITE_TRUNCATE" option and the table schema not be deleted? Am I doing something wrong? Below I forward the configuration that I am using in the PutBigQueryBatch processor:
... View more
Labels:
- Labels:
-
Apache NiFi
-
Schema Registry
12-21-2021
05:14 AM
I have a stream in which I generate a json file with GenerateFlowFile, then I use Base64EncodeContent to encode the json file. I need to take the base64 encode result and concatenate it with a sentence, to finally move to invokehttp. I tried MergeContent for this and was not successful. I'm not able to find a way to get the result of the Base64EncodeContent encode and concatenate it with the sentence. I would like suggestions on how to carry out this flow. Below is the flow I drew:
... View more
Labels:
- Labels:
-
Apache NiFi
12-15-2021
05:23 AM
2 Kudos
I managed to solve it, I used ${field.value:substring(0,23):toDate("yyyy-MM-dd'T'HH:mm:ss.SSS"):format("yyyy-MM-dd HH:mm: ss.SSS")}
... View more
12-13-2021
12:40 PM
I get a file with multiple dates in the following format: 2021-08-25T12:34:48.984704 which is in micro format. How could I change these dates on the nifi to stay in millis? I tried the following: Apache Nifi Expression Language - toDate formatting formatting but I was not successful in the configuration. I believe that the answer is cohort, but I am not managing to make it work.
... View more
Labels:
- Labels:
-
Apache NiFi