Member since
11-16-2015
905
Posts
665
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 431 | 09-30-2025 05:23 AM | |
| 767 | 06-26-2025 01:21 PM | |
| 660 | 06-19-2025 02:48 PM | |
| 849 | 05-30-2025 01:53 PM | |
| 11388 | 02-22-2024 12:38 PM |
02-11-2019
06:28 PM
In an upcoming release you'll be able to use Hive 1.1 processors, so in your case you'd want to keep what you have (Avro in HDFS) and use PutHive_1_1QL to issue a LOAD DATA or CREATE EXTERNAL TABLE statement so Hive can see your data.
... View more
02-11-2019
03:20 PM
1-3: The processors that use a DMC client use the DMC in a very specific manner, so they CRUD cache entries as it applies to their operations. There isn't currently a generic processor that lets you call arbitrary cache API methods, that's what the scripting components are for. 4: We don't have the concept of tables in DMC, only key/value pairs. A table can probably be implemented by namespacing the key, not sure if the processors you're using support custom keys though. 5: The DMC operation in a cluster is very similar to how it works on a local installation, except there is a DMC server created on each node in the cluster. However a DMC client still has to choose a single host:port to connect to, and the individual DMC servers are not coordinated at the cluster, meaning if you update one, the others don't get that update; they are fully separate at the moment. 6: AFAIK there is no best practice as far as choosing a DMC server to connect to, other than choosing one on a node that tends to be available most often. You basically get individual, isolated instances to choose from. We have other DMC server implementations that possibly support High Availability and/or Data Durability, such as HBase- or Redis- backed solutions. However neither of these are included with an Apache NiFi distribution, you'd have to bring your own.
... View more
01-10-2019
03:28 PM
What does your schema look like? I'm guessing that your "imu_vg_z" field is of some type (maybe "bytes"), rather than a nullable type (such as ["null", "bytes"])?
... View more
01-10-2019
02:22 PM
3 Kudos
The sql.args.N.type parameters are actually numeric codes indicating the type, not the string name of the type. So instead of setting sql.args.4.type to "CHAR", set it to "1" instead. The mapping of numbers to types is given here.
... View more
01-09-2019
06:55 PM
Looks like @Bryan Bende answered your question on SO.
... View more
12-17-2018
06:37 PM
1 Kudo
You can use ConvertRecord for this, your JsonRecordSetWriter would have its Output Grouping property set to "One Line Per Object". The individual objects won't be pretty printed in the output as you have above, but it will remove the array braces as well as the comma delimiter, and just output each element of the above array as a JSON object on its own line.
... View more
12-12-2018
04:03 PM
What do you mean by "window variable"? Do you mean a Process Group Variable? If so, you can change it via the UI or REST API (I believe the endpoint is /nifi-api/process-groups/<id>/variable-registry/update-requests). Note that it will stop any affected processors or controller services, then change the variable value, then restart/re-enable the affected components. If you mean a Flow File attribute, you can add/change/delete those using the UpdateAttribute processor.
... View more
12-05-2018
06:30 PM
You shouldn't need SplitAvro in there, PutDatabaseRecord was designed to handle multiple records. I'll try to reproduce the issue and will keep you posted.
... View more
11-30-2018
06:23 PM
We don't publish the javadocs for NiFi (yet), but you can get at them via javadoc.io (for example, the 1.8.0 nifi-api Javadocs are here). I've been thinking about adding a Part 4 to my cookbook, collecting examples from HCC articles and mailing list questions, to provide more coverage of the available APIs. Also check my blog, those articles are usually left-of-center usages of the API, perhaps to do things for which it wasn't necessarily intended, or to give external tools to help develop scripts (like my script tester, e.g.)
... View more
11-28-2018
06:52 PM
A couple of notes: 1) There is no close() method on the session, I think you want remove(). 2) In your try block you are creating and transferring updated_flowFile, but you don't transfer or remove flowFile. If you replace close() with remove(), it should fix the "transfer relationship not specified" issue. 3) In your except block, you might need to check for updated_flowFile != None before you remove it. If the exception occurs before the updated_flowFile is created, the remove() might fail. 4) Consider using EvaluateJsonPath to extract the "left" and "top" attributes, then ReplaceText to clear the content. Then your provenance/lineage chain will remain intact. Jython is fairly slow so you may see a performance improvement by using EvaluateJsonPath -> ReplaceText
... View more