Created on 08-02-201612:08 AM - edited 08-17-201911:05 AM
Tips
Read each processor
directions carefully, inputs and outputs vary greatly from attributes, JSON, JSONPath statements and other
entries. JSONPath supports a radically different syntax than NIFI Expression Language.
Know your Expression
language, there are a lot of useful
functions like UUID() and nextint().
When you are doing a lot of streaming and testing data, you will find that logs grow huge on your Centos sandbox. I built up gigabytes of logs in
Hive, Hadoop and NiFi in just a few days of operations. Check your /var/log every few days and watch the Ambari screens for data usage. When your data gets too large, things can start failing or slowing down due to a lack of temporary space, swap space and space for logs.
Delete Test Files
For Ever!
hdfs dfs -rm -f -skipTrash /twitter/*.json
Clean up junk files.
cd /
du -hsx * | sort -rh | head -10 )
PutHiveQL requires
hiveql.args.1.value and hiveql.args.1.type for all fields. If you are using EvaluateJSONPath, you cannot
set your types there. Do those in an
UpdateAttribute processor after that.
Then you need to turn your code in SQL for Hive to execute.