Support Questions

Find answers, ask questions, and share your expertise

How do I reference column names with spaces in SQL server table from NiFi ExecuteSQL processor?

avatar
Explorer

I am building a workflow to extract data from a MS SQL server table using the ExecuteSQL processor with this simple SQL:

select
[06 QQQ],
[11 JJJ],
[12 KKK],
[13 JJJ],
from DB_TEST.dbo.FGRID

But I get the error below.

routing to failure: org.apache.nifi.processor.exception.ProcessException: org.apache.avro.SchemaParseException: Illegal initial character: 06  QQQ
- Caused by: org.apache.avro.SchemaParseException: Illegal initial character: 06  QQQ

 Apparently, NiFi can't recognize column names with spaces and other special characters. I have tried using single and double quotes but still get same error. Question is - what can I use to reference column names with special characters. In DBeaver, I am able to query those column names with []. Any assistance will be appreciated. Thanks

3 REPLIES 3

avatar

@LKB, you could try encapsulating your column names in single or double quotes or backticks in NiFi, instead of using the square brackets.

Something like:

select
`06 QQQ`,
`11 JJJ`,
`12 KKK`,
`13 JJJ`
from DB_TEST.dbo.FGRID


or like this:

select
"06 QQQ",
"11 JJJ",
"12 KKK",
"13 JJJ"
from DB_TEST.dbo.FGRID

 In SQL, column names and aliases should not start with numeric digits or contain spaces unless they are enclosed in backticks (``) or double quotes (") depending on the database system. Using DBeaver, those backticks are replaced by double quotes.

avatar
Explorer

@cotopaul

I have tried all the suggestions above and none of them worked. I have also tried to use the ExecuteSQL settings to normalize column and table but it messed up the actual values in those columns. Not sure why presenting column names with spaces (or alphanumeric characters) should be so painful as I have been on this for days now. I'll appreciate any solution out there from anyone who has encountered similar issue

avatar

@LKBand if you try the below select, what error do you encounter? Unfortunately I have no MS SQL database with which I could test so I can only somehow help you debug your situation, one step at a time.

 

select
"06 QQQ" as 66_QQQ, 
"11 JJJ" as 11_JJJ,
"12 KKK" as 12_KKK,
"13 JJJ" as 13_JJJ
from DB_TEST.dbo.FGRID

-- or with ` instead of "
-- or QQQ_66, sending the number at the back of the column, instead of the front. 

 

I assume that you have your Record Writer configured as Inherit Record Schema.