How different is it from using livy to do the same? ... View more

A small correction. It's introduced in Ranger 0.7 and policies should look like this: //HDFS resource: path=/home/{USER} user: {USER} //Hive resource: database=db_{USER}; table=*; column=* user: {USER} where {USER} would substitute the user id of the currently logged in user. ... View more

Apart form JVM limitation, which you can increase, there are no definite limitation on size or number of flowfile records as such. I would say design your flow and if you feel that you're throttled, you can follow some good designing practices to tweak your flow. Take a look at this: https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html ... View more