Created 02-27-2018 03:21 PM
Hello,
I'm trying to put a CSV file on my table using PutDatabseRecords. And I have this error :
none of the fields in the records map to the columns defined warranty table
My table is like :
Create table Warranty( id int, Train_Id varchar(255), Siemens_Nr varchar(255), UIC_Nr varchar(255), Configuration varchar(255), Warranty_Status varchar(255), Warranty_Data_Type varchar(255), _of_progression varchar(255), Delivery_Date varchar(255), Warranty_on_Delivery_Date varchar(255), Customer_Status varchar(255), Commissioning_Date varchar(255), Preliminary_acceptance_date varchar(255), Warranty_Start_Date varchar(255), Warranty_End_Date varchar(255), Effective_End_Warranty_Date varchar(255), Level_2_in_function varchar(255), Baseline varchar(255), RELN_revision varchar(255), TC_report varchar(255), Last_version_Date varchar(255), ETCS_ID_NID_Engine float, Item_Type varchar(255), Path varchar(255) )
and my avro schema generated from InferAvroSchema is :
{ "type" : "record", "name" : "Warranty", "doc" : "Schema generated by Kite", "fields" : [ { "name" : "id", "type" : "long", "doc" : "Type inferred from '1'" }, { "name" : "Train_Id", "type" : "long", "doc" : "Type inferred from '21691'" }, { "name" : "Siemens_Nr", "type" : "string", "doc" : "Type inferred from 'Loco-001'" }, { "name" : "UIC_Nr", "type" : "long", "doc" : "Type inferred from '193901'" }, { "name" : "Configuration", "type" : "string", "doc" : "Type inferred from 'ZP28'" }, { "name" : "Warranty_Status", "type" : "string", "doc" : "Type inferred from 'Out_of_Warranty'" }, { "name" : "Warranty_Data_Type", "type" : "string", "doc" : "Type inferred from 'Real_based_on_preliminary_acceptance_date'" }, { "name" : "of_progression", "type" : "long", "doc" : "Type inferred from '100'" }, { "name" : "Delivery_Date", "type" : "string", "doc" : "Type inferred from '18/12/2009'" }, { "name" : "Warranty_on_Delivery_Date", "type" : "string", "doc" : "Type inferred from '18/12/2013'" }, { "name" : "Customer_Status", "type" : "string", "doc" : "Type inferred from 'homologation'" }, { "name" : "Commissioning_Date", "type" : "string", "doc" : "Type inferred from '6/10/2010'" }, { "name" : "Preliminary_acceptance_date", "type" : "string", "doc" : "Type inferred from '6/01/2011'" }, { "name" : "Warranty_Start_Date", "type" : "string", "doc" : "Type inferred from '6/01/2011'" }, { "name" : "Warranty_End_Date", "type" : "string", "doc" : "Type inferred from '6/01/2013'" }, { "name" : "Effective_End_Warranty_Date", "type" : [ "null", "string" ], "doc" : "Type inferred from 'null'", "default" : null }, { "name" : "Level_2_in_function", "type" : "string", "doc" : "Type inferred from '17/07/2015'" }, { "name" : "Baseline", "type" : "string", "doc" : "Type inferred from '2.10.23.4'" }, { "name" : "RELN_revision", "type" : "string", "doc" : "Type inferred from '0434-26.3'" }, { "name" : "TC_report", "type" : "string", "doc" : "Type inferred from 'A480140'" }, { "name" : "Last_version_Date", "type" : "string", "doc" : "Type inferred from 'A-23/09/2015'" }, { "name" : "ETCS_ID_NID_Engine", "type" : [ "null", "long" ], "doc" : "Type inferred from '13001'", "default" : null }, { "name" : "Item_Type", "type" : "string", "doc" : "Type inferred from 'Item'" }, { "name" : "Path", "type" : "string", "doc" : "Type inferred from 'sites/TrWMTISnerc_Community/Lists/X4Trains'" } ] }
Can someone help me to find the error plz !
Thank you
Created 02-27-2018 03:34 PM
Can you please provide some more info about what you are trying to do?
Created 02-27-2018 04:42 PM
can you post some logs and example data
Created 02-27-2018 04:43 PM
include template, screen shot, example CSV
Created 02-27-2018 04:56 PM
See this: https://community.hortonworks.com/questions/45426/convertjsontosql-problemsjsontosql-issues.html
Could be this:
What is the "Translate Field Names" property of ConvertJSONtoSQL set to? If your columns are identical to the json field names, be sure to set that property to false.
None of the fields are matching by name: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-proce...
I am thinking it's case sensitivity issues. This is a difficult name and not supported by AVRO: _of_progression varchar(255),
If you have any weird or non-AVRO safe names, try my Attributename cleaner https://community.hortonworks.com/articles/154760/generating-avro-schemas-and-ensuring-field-names-m...
Created 02-27-2018 05:10 PM
I don't use ConvertJsonToSQL.
I'm using GetFile -> InferAvroSchema -> PutDatabaseRecord.
Created 02-27-2018 05:29 PM
Ignore unmatched fields for both attributes
check the names
quote column identifiers -true
translate field names true
Created on 02-27-2018 05:59 PM - edited 08-17-2019 05:28 PM
I have always the same error!
I share my processor configuration
Created 02-28-2018 08:46 AM
Created 02-28-2018 01:58 PM
1. Turn translate fields name to true
2. You must specify a schema registry
3. Change your schema access strategy. It's usually not from header
Where is your schema name? Where is schema stored?