PutBigQuery not evaluating flowfile attributes

[#1]

I'm attempting to transition from using PutBigQueryBatch to PutBigQuery. However, the latter doesn't seem to evaluate the dataset and table attributes as expected.

In the provided example, one processor has the values hardcoded, which works correctly, while the other processor relies on retrieving those values from the flowfile's attributes. However, it appears that the attributes are not being properly evaluated.

The example flow (using Nifi 1.21.0):

The flowFile used:

The left PutBigQuery processor has hardcoded values, and it works:

The right PutBigQuery processor fails because it seems it's using the EL definition but it's not evaluating it:

Any help would be appreciated.

[#2 · Accepted solution]

@jisaitua Thank you for making such a well written post with all of the right screen shots, etc.  

I am working on duplicating this issue and so far i have confirmed the same experience. 

INVALID_ARGUMENT: Invalid project resource name projects/${test}; Project id: ${test}

So this is a known JIRA bug:

https://issues.apache.org/jira/projects/NIFI/issues/NIFI-11608?filter=allissues

I will work on getting some traction here!!  

[#3]

@jisaitua Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @cotopaul  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#4]

Thank you @DianaTorres for your comment. Let's see what the experts have to say.

Regards!

[#5 · Accepted solution]

@jisaitua Thank you for making such a well written post with all of the right screen shots, etc.  

I am working on duplicating this issue and so far i have confirmed the same experience. 

INVALID_ARGUMENT: Invalid project resource name projects/${test}; Project id: ${test}

So this is a known JIRA bug:

https://issues.apache.org/jira/projects/NIFI/issues/NIFI-11608?filter=allissues

I will work on getting some traction here!!  

[#6]

Thank you for helping with this, @steven-matison!

[#7]

Some progress:

 resource 'projects/gcp-se-cdp-sandbox-env/datasets/dataset/tables/tablename'

With datasets = ${bq.dataset} tables = ${bq.table.name}

[#8]

@jisaitua I am not sure how long its going to take, but i should have a github commit ready w/ a new PutBigQuery nar that you can use with 1.21.   I will update again tomorrow morning.

[#9 · Accepted solution]

Pull Request is in!!   

https://github.com/apache/nifi/pull/7316

This will be fixed in future releases of NiFi.

@jisaitua send me a DM with your email if you need to use the new processor right away.

[#10]

Wow, that was fast, thank you very much @steven-matison!