How to validate/ignore optional value while validating it using 'Validate CSV' processor in Apache NiFi?

[#1]

I'm new to Apache Nifi Community and I got stuck at one point - please help me for the below problem :

In 'Validate CSV' processor I'm giving below schema(schema_used) to validate the CSV input. But, processor is showing error(err_screenshot).

Validate CSV -

Schema_Used - StrNotNullOrEmpty(), ParseDate("MM/dd/yyyy"), StrNotNullOrEmpty(), StrNotNullOrEmpty(), Optional(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), Optional(), Optional(), StrNotNullOrEmpty()

The above schema is showing below error -

err_screenshot -

Note: The Schema is working totally fine, if I remove 'Optional()'.

Problem - Please tell me how can I use 'Validate CSV' processor in Apache Nifi and validate Mandatory/Optional Fields of CSV. Also, suggest/share any script(groovy/python) if that can validate the CSV.

[#2 · Accepted solution]

Hi @AidenMartin ,

According to the documentation of the validate csv apply processors, the Optional has to take  a type parameter :

"

...

Schema property: Null, ParseDate("dd/MM/yyyy"), Optional(ParseDouble())
Meaning: the input CSV has three columns, the first one can be null and has no specification, the second one must be a date formatted as expected, and the third one must a double or null (no value).

...

"

If you are not sure of the data type then just use Null instead of Optional, which means the value can be provided or not.

reference: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.23.2/org.apach...

If you find this is helpful please accept solution.