https://community.cloudera.com/t5/Support-Questions/How-to-validate-ignore-optional-value-while-validating-it/m-p/384637#M245441 <DIV class="postcell post-layout--right"><DIV class="s-prose js-post-body"><P>I'm new to Apache Nifi Community and I got stuck at one point - please help me for the below problem :</P><P>In 'Validate CSV' processor I'm giving below schema(<STRONG>schema_used</STRONG>) to validate the CSV input. But, processor is showing error(<STRONG>err_screenshot</STRONG>).</P><P><STRONG>Validate CSV</STRONG> -</P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="AidenMartin_0-1709718794627.png" style="width: 400px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/39982i89B7DE62E6E78737/image-size/medium?v=v2&amp;px=400" role="button" title="AidenMartin_0-1709718794627.png" alt="AidenMartin_0-1709718794627.png" /></span><P><STRONG>Schema_Used</STRONG> - <EM>StrNotNullOrEmpty(), ParseDate("MM/dd/yyyy"), StrNotNullOrEmpty(), StrNotNullOrEmpty(), <FONT color="#FF0000">Optional()</FONT>, StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), <FONT color="#FF0000">Optional()</FONT>, <FONT color="#FF0000">Optional()</FONT>, StrNotNullOrEmpty()</EM></P><P>The above schema is showing below error -<BR /><BR /><STRONG>err_screenshot</STRONG> -</P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="AidenMartin_1-1709718875207.png" style="width: 400px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/39983i17B4A0FF100ADD11/image-size/medium?v=v2&amp;px=400" role="button" title="AidenMartin_1-1709718875207.png" alt="AidenMartin_1-1709718875207.png" /></span><P><STRONG>Note:</STRONG> The Schema is working totally fine, if I remove 'Optional()'.</P><P><STRONG>Problem</STRONG> - Please tell me how can I use 'Validate CSV' processor in Apache Nifi and validate Mandatory/Optional Fields of CSV. Also, suggest/share any script(groovy/python) if that can validate the CSV.</P></DIV></DIV> Wed, 06 Mar 2024 09:57:42 GMT AidenMartin 2024-03-06T09:57:42Z https://community.cloudera.com/t5/Support-Questions/How-to-validate-ignore-optional-value-while-validating-it/m-p/384637#M245441 <DIV class="postcell post-layout--right"><DIV class="s-prose js-post-body"><P>I'm new to Apache Nifi Community and I got stuck at one point - please help me for the below problem :</P><P>In 'Validate CSV' processor I'm giving below schema(<STRONG>schema_used</STRONG>) to validate the CSV input. But, processor is showing error(<STRONG>err_screenshot</STRONG>).</P><P><STRONG>Validate CSV</STRONG> -</P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="AidenMartin_0-1709718794627.png" style="width: 400px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/39982i89B7DE62E6E78737/image-size/medium?v=v2&amp;px=400" role="button" title="AidenMartin_0-1709718794627.png" alt="AidenMartin_0-1709718794627.png" /></span><P><STRONG>Schema_Used</STRONG> - <EM>StrNotNullOrEmpty(), ParseDate("MM/dd/yyyy"), StrNotNullOrEmpty(), StrNotNullOrEmpty(), <FONT color="#FF0000">Optional()</FONT>, StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), <FONT color="#FF0000">Optional()</FONT>, <FONT color="#FF0000">Optional()</FONT>, StrNotNullOrEmpty()</EM></P><P>The above schema is showing below error -<BR /><BR /><STRONG>err_screenshot</STRONG> -</P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="AidenMartin_1-1709718875207.png" style="width: 400px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/39983i17B4A0FF100ADD11/image-size/medium?v=v2&amp;px=400" role="button" title="AidenMartin_1-1709718875207.png" alt="AidenMartin_1-1709718875207.png" /></span><P><STRONG>Note:</STRONG> The Schema is working totally fine, if I remove 'Optional()'.</P><P><STRONG>Problem</STRONG> - Please tell me how can I use 'Validate CSV' processor in Apache Nifi and validate Mandatory/Optional Fields of CSV. Also, suggest/share any script(groovy/python) if that can validate the CSV.</P></DIV></DIV> Wed, 06 Mar 2024 09:57:42 GMT https://community.cloudera.com/t5/Support-Questions/How-to-validate-ignore-optional-value-while-validating-it/m-p/384637#M245441 AidenMartin 2024-03-06T09:57:42Z https://community.cloudera.com/t5/Support-Questions/How-to-validate-ignore-optional-value-while-validating-it/m-p/384644#M245445 <P>Hi <a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/109438">@AidenMartin</a> ,</P><P>According to the documentation of the validate csv apply processors, the Optional has to take&nbsp; a type parameter :</P><P>"</P><P>...</P><P><EM><STRONG>Schema property:</STRONG>&nbsp;</EM>Null, ParseDate("dd/MM/yyyy"), Optional(ParseDouble())<BR /><EM><STRONG>Meaning:</STRONG> the input CSV has three columns, the first one can be null and has no specification, the second one must be a date formatted as expected, and the third one must a double or null (no value).</EM></P><P><EM>...</EM></P><P>"</P><P>If you are not sure of the data type then just use Null instead of Optional, which means the value can be provided or not.</P><P>reference: <A href="https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.23.2/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html" target="_blank" rel="noopener">https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.23.2/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html</A></P><P>If you find this is helpful <STRONG>please</STRONG> accept solution.</P><P>&nbsp;</P> Wed, 06 Mar 2024 10:54:05 GMT https://community.cloudera.com/t5/Support-Questions/How-to-validate-ignore-optional-value-while-validating-it/m-p/384644#M245445 SAMSAL 2024-03-06T10:54:05Z