Created 09-24-2023 07:24 AM
I am building a workflow to extract data from a MS SQL server table using the ExecuteSQL processor with this simple SQL:
select
[06 QQQ],
[11 JJJ],
[12 KKK],
[13 JJJ],
from DB_TEST.dbo.FGRID
But I get the error below.
routing to failure: org.apache.nifi.processor.exception.ProcessException: org.apache.avro.SchemaParseException: Illegal initial character: 06 QQQ - Caused by: org.apache.avro.SchemaParseException: Illegal initial character: 06 QQQ
Apparently, NiFi can't recognize column names with spaces and other special characters. I have tried using single and double quotes but still get same error. Question is - what can I use to reference column names with special characters. In DBeaver, I am able to query those column names with []. Any assistance will be appreciated. Thanks
Created 09-25-2023 01:35 AM
@LKB, you could try encapsulating your column names in single or double quotes or backticks in NiFi, instead of using the square brackets.
Something like:
select
`06 QQQ`,
`11 JJJ`,
`12 KKK`,
`13 JJJ`
from DB_TEST.dbo.FGRID
or like this:
select
"06 QQQ",
"11 JJJ",
"12 KKK",
"13 JJJ"
from DB_TEST.dbo.FGRID
In SQL, column names and aliases should not start with numeric digits or contain spaces unless they are enclosed in backticks (``) or double quotes (") depending on the database system. Using DBeaver, those backticks are replaced by double quotes.
Created on 09-25-2023 12:03 PM - edited 09-25-2023 12:06 PM
I have tried all the suggestions above and none of them worked. I have also tried to use the ExecuteSQL settings to normalize column and table but it messed up the actual values in those columns. Not sure why presenting column names with spaces (or alphanumeric characters) should be so painful as I have been on this for days now. I'll appreciate any solution out there from anyone who has encountered similar issue
Created on 09-25-2023 11:17 PM - edited 09-25-2023 11:19 PM
@LKBand if you try the below select, what error do you encounter? Unfortunately I have no MS SQL database with which I could test so I can only somehow help you debug your situation, one step at a time.
select
"06 QQQ" as 66_QQQ,
"11 JJJ" as 11_JJJ,
"12 KKK" as 12_KKK,
"13 JJJ" as 13_JJJ
from DB_TEST.dbo.FGRID
-- or with ` instead of "
-- or QQQ_66, sending the number at the back of the column, instead of the front.
I assume that you have your Record Writer configured as Inherit Record Schema.