ValidateRecord - Do not process valid records when invalid records found

[#1]

Hello ,

I need help with designing a flow such that when there are invalid records found then do not process the valid records. I want to use validateRecord processor to determine the records validity 

Thanks,

Hemanth

[#2]

Hi,

That depends on your input and what kind of validation you are trying to do. can you provide more information on that? For example the blow flow validate json input against Avro schema:

The input Json in the GenerateFlowFile processor looks like as follows:

{

	"records":
	[
		{
			"name":"John",
			"age": 25
		},
		{
			"name":"Smith",
			"age": 33
		}
	]

}

The configuration for the ValidateRecord Processor looks like:

Where the SchemaText Value:

{
	"type": "record",
	"name": "Record",
	"fields": [
		{
			"name": "records",
			"type": {
				"type": "array",
				"items": {
					"type": "record",
					"namespace": "Record",
					"name": "records",
					"fields": [
						{
							"name": "name",
							"type": "string"
						},
						{
							"name": "age",
							"type": "long"
						}
					]
				}
			}
		}
	]
}

According to the Schema the input is valid json , however if you change age value to null or some string it will be invalid. The "Validation Details Attribute Name" Property in the ValidationRecord will store the validation message in the specified attribute name.

Hope that helps.

[#3]

Thank you for the response.

I would like to not process the valid records when there are invalid records found

[#4]

what is the format of the records ? Are they json array of records ? Can you provide sample data or an example of how your records are structured in the flowfile?

[#5]

the input files are csv