@MattWho    Exactly, the advanced feature in UpdateAtrributes is how we tag all in the incoming data.  Flowfile comes in and the schema.name attribute gets set.  Then later down the line we do some validation on the data as it gets read into a NiFi Recordset object in conjunction with the AvroSchemaRegistry.  However, we would like to switch to using JSON Schema, but the ValidateJSON processor doesn't support attribute references so the "JSON Schema" parameter needs to either the whole JSON Schema itself, or a hard-coded Parameter Context parameter reference.   What this means is that we need to use a separate ValidateJson processor for each JSON Schema (i.e. 20 schemas = 20 processors).  Ugh!   This is a very significant shortcoming of the ValidateJson processor and makes it unusable except for simple flows with homogenous data.  Hopefully there are plans to expand out its features.   I attached a couple of screenshots to help illustrate the problem.       ... View more

The concept is EXACTLY the same concept as using the AvroSchemaRegistry, which holds a bunch of schemas, and as each flowfile gets read by a single processor that uses a NiFi Recordset Reader/Writer service (e.g. ConvertRecord), they will individually reference their own schema.  Some are Red data, some are Blue data, and some are Purple data, and they each dynamically reference their corresponding Avro schema.  In fact, you can specify the "schema.name" attribute, which references said schema in the AvroSchemaRegistry.   The premise of my inquiry is that I want to achieve something similar using JSON Schemas despite the fact there is no JSON Schema Registry.  So I want to use Parameter Context parameters to hold the schemas, then dynamically refer to them in a similar manner as we can do with Avro schemas. ... View more

Thanks for taking the time to reply Matt, but this doesn't address my question.  Let me use an example and hopefully that will clarify what I'm trying to do.   Within a process group named "transit", there is a "Process Group Parameter Context" named "transit_variables".   This parameter context (transit_variables) includes all the high level variables used throughout the process group, but it also contains a large number of Avro schemas which are referenced in an AvroSchemaRegistry controller service, which is defined within the main process group.     For the sake of organizational purposes I'd like to move these Avro schemas into their own parameter context, BUT (this is the catch), I want access to them from the AvroSchemaRegistry whose scope is associated with the process group, whose scope is set to the parent parameter context.   I can "solve" this problem by creating a child process group (e.g. transit_data_parser) with a new child parameter context (e.g. transit_schemas), then add additional AvroSchemaRegistry, JSONTreeReader, JSONRecordsetWriter controller services, which are coupled with the child parameter context "transit_schemas".  The process group would contain a single processor with an input port and and output port, and would be used solely for the purpose of reading a flowfile into a NiFi Recordset object with Avro validation.      This so-called solution is very tightly coupled and as such not very flexible.  It means that every time I want to use a schema I need to create one of these tightly couple process groups, which defeats the whole purpose of the context parameter concept.   What would be better is if I could reference these child schemas using a hierarchal object syntax, e.g.  #{transit_variables.transit_schemas}.  Using this syntax from within a single AvroSchemaRegistry controller service I can reference parameters at the parent level and all parameters at the child levels.  ... View more

There were some changes in 1.20 to address better schema inference. See the references below for release notes for 1.20 and specifically the issue you are experiencing.    [NIFI-10956] - Schema Inference returns incorrect datatype for records where some arrays are empty   https://issues.apache.org/jira/browse/NIFI-10956       ... View more

What you are doing is mostly correct.  I see a couple of items that may be creating some issues, e.g. mis-assigned values.   For example, you are assigning accel.x to all three attributes values ax, ay, az.  But it seems like you probably meant to assign ax = $.accel.x , ay =$.accel.y , az =$.accel.z   It seems you've mixed up "random" and "rand".  The field name in your JSON is "random" so your attribute assignment should be rand = $.random   Where is "btn" in your JSON? ... View more

It sounds like you are using OAuth, which means you ought to be using an OAuth2AccessTokenProvider controller service.  See my screen shot, which shows that inside the InvokeHTTP there is a configuration line that lets you specify this controller service.  This service automatically handles getting the token for you and embedding it in the header so your flowfile remains intact.  This will greatly simplify your current flow because you can get rid of the map cache and that whole process cycle.     ... View more