Hi @mburgess,   I was looking for a processor that would let me pull a query out of a field of the incoming flow file, instead of turning the entire flow file into a query. PutDatabaseRecord allowed to do that (which is when I discovered that the Hive connection does not support an explicit call of conn.commit()).   I want to keep as much of the flow file intact as possible, is there a way to do that?   Thank you. ... View more

InferAvroSchema requires you to enter the record name yourself, if you need access to that record name later, you could set a variable on the process group and use that in Expression Language for the Record Name, then you'd have access to that same variable everywhere in the process group and wouldn't have to extract it. If you don't have access to the value injected into the schema by InferAvroSchema, and the schema is in an attribute (let's say "inferred.avro.schema"), then you can use the jsonPath() function in NiFi Expression Language to extract the record name into a separate attribute. You'd need an UpdateAttribute to set "record.name" to the following: ${inferred.avro.schema:jsonPath("$.name")} If your schema is in the content of the flow file, then since it is a JSON object you can use EvaluateJSONPath to get the record name into an attribute, using the following JSONPath expression: $.name ExtractAvroMetadata is for Avro files, so if your flow file contained Avro (with an embedded schema), you could use ExtractAvroMetadata (adding "avro.schema" to the list of metadata to extract) in order to get the schema. But this processor doesn't work for CSV files. ... View more

As of NiFi 1.5.0 (via NIFI-4522), you can issue a SQL query in PutSQL while still retaining the incoming flow file contents. For your case, you could send the CSV file to PutSQL and execute a "CREATE TABLE IF NOT EXISTS" statement, which will create the table the first time but allow the CSV to proceed to the "real" destination processor, likely PutDatabaseRecord. ... View more

@Amira khalifa Use one of the way from the above shared link to take out only the header from the csv file then in replace text keep the then search for (&|\(|\)|\/_|\s) and in Replacement value keep as empty string, now we are searching for all the special characters in the header flowfile then replacing with empty string.Now add this header flowfile with the other non header flowfile. all the explanation and template.xml are shared in this link. ... View more

@Amira khalifa Please mark the answer as accepted if it helped to resolve your problem ... View more

I some problem. not solution problem. 😞 ... View more

The relevant part of the log is "Address already in use", looks like you've configured ListenHttp to listen on port 8081 but some other process is using that port already. ... View more

My file extension is avro i can't use def schema = ff.read().withReader("UTF-8"){newJsonSlurper().parse(it)} I have to use each and then parse schema. Do you an example plz @Matt Burgess ... View more