Hello, This may be a silly question but I'm in a bit of a loss and have been stuck for a couple days now trying to find a solution. I have a flow design where I have to validate a record against an avro schema and insert that record in a Kudu table. The avro validation schema is: {
"type" : "record",
"name" : "tbcertificati",
"fields" : [
{ "name" : "num_caso", "type" : "string" },
{ "name" : "num_rch", "type" : "string" },
{ "name" : "num_cert", "type" : "string" },
{ "name" : "seqno", "type" : "string" },
{ "name" : "data_inizio_kudu", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "num_vis", "type" : "double" },
{ "name" : "data_compilaz_cert", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "data_acquisiz_cert", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "cod_matr_med", "type" : "double" },
{ "name" : "medico_struttura", "type" : "string" },
{ "name" : "flag_int_ext", "type" : "string" },
{ "name" : "cod_cert", "type" : "string" },
{ "name" : "cod_prognosi", "type" : "double" },
{ "name" : "data_iniz_prognosi", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "data_fine_prognosi", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "desc_cert", "type" : "string" },
{ "name" : "data_ripresa_lav", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "flag_postumi", "type" : "string" },
{ "name" : "flag_medico_stru", "type" : "string" },
{ "name" : "note", "type" : "string" },
{ "name" : "prg", "type" : "double" },
{ "name" : "crt_time", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "id_usr_crt", "type" : "string" },
{ "name" : "mod_time", "type" : {
"type":"long",
"logicalType":"timestamp-millis"}},
{ "name" : "id_usr_mod", "type" : "string" },
{ "name" : "fl_annullato", "type" : "string" },
{ "name" : "dm_timestamp", "type" : "string" },
{ "name" : "dm_txid", "type" : "string" },
{ "name" : "dm_operation_type", "type" : "string" }
]
} A record can arrive like this: {
"DM_TIMESTAMP" : "1970-01-01 00:00:00.000",
"DM_TXID" : "1",
"DM_OPERATION_TYPE" : "A",
"DM_USER" : "STRING",
"NUM_CASO" : "1",
"NUM_RCH" : "1",
"NUM_CERT" : "1",
"NUM_VIS" : "",
"DATA_COMPILAZ_CERT" : "1970-01-01 00:00:00",
"DATA_ACQUISIZ_CERT" : "",
"COD_MATR_MED" : "",
"MEDICO_STRUTTURA" : "",
"FLAG_INT_EXT" : "A",
"COD_CERT" : "A",
"COD_PROGNOSI" : "1",
"DATA_INIZ_PROGNOSI" : "",
"DATA_FINE_PROGNOSI" : "",
"DESC_CERT" : "",
"DATA_RIPRESA_LAV" : "1970-01-01",
"FLAG_POSTUMI" : "",
"FLAG_MEDICO_STRU" : "A",
"NOTE" : "",
"PRG" : "1",
"CRT_TIME" : "1970-01-01 00:00:00",
"ID_USR_CRT" : "STRING",
"MOD_TIME" : "1970-01-01 00:00:00",
"ID_USR_MOD" : "STRING",
"FL_ANNULLATO" : "A",
"SEQNO" : "1",
"DATA_INIZIO_KUDU" : 1673274352029
} The putKudu processor is as following: Validation record always fails, and if trying to directly insert the record with the PutKudu processor, I get the error: Failed to write due to Can't set primary key column foo to null : java.land.IllegalArgumentException: Can't set primary key column num_caso to null I have trying capitalizing/decapitalizing the field names, with no success. The field clearly has a value associated to it, I don't understand why it is said to be null. Thanks in advance.
... View more
