It's actual title is "Hadoop and Kerberos: The Madness Beyond the Gate" —there's an HP Lovecraft theme of "forbidden knowledge which will drive you insane" which is less a joke and more commentary. it's actually rendered on gitbook If you are working with Kerberos, get a copy of the O'Reilly Hadoop Security book too. My little e-book was written to cover the bits that was left out: to extend rather than replace. Finally, being open source: contributions are welcome ... View more

This is old post but came up on my search. for others that may be looking now https://community.hortonworks.com/articles/85103/nifi-ports.html ... View more

Thank you @Ali Bajwa for good tutoral. I am trying this example with a difference, My nifi is local and I try to put tweets in a remote Solr. Solr is in a VM that contains Hortonworks sandbox. Unfortunately I am getting this error on PutSolrContentStream processor: PutSolrContentStream[id=f6327477-fb7d-4af0-ec32-afcdb184e545] Failed to send StandardFlowFileRecord[uuid=9bc39142-c02c-4fa2-a911-9a9572e885d0,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1487148463852-14, container=default, section=14], offset=696096, length=2589],offset=0,name=103056151325300.json,size=2589] to Solr due to org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; routing to connection_failure: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://172.17.0.2:8983/solr/tweets_shard1_replica1; Could you help me? thanks, Shanghoosh ... View more

2181 is the Zookeeper port, make sure your Zookeeper ensemble is running and accessible. You might want to create a new question and describe your setup a bit more, it is probably easier to solve this problem then 🙂 ... View more

e.g Create a file with the JSON data to create the key with tee -a ./body.json << EOF { "name" : "test_key_curl", "length" : 128, "material" : "lksvIq3yy9Xxk4EZTfLv6g", "description" : "test_key_curl" } EOF Then run the curl command, in my case, my KMS host is hannibal-1.openstacklocal and post is 9292 curl -i --negotiate -u : -H "Content-Type: application/json" -X POST -d @body.json http://hannibal-1.openstacklocal:9292/kms/v1/keys NOTE: You would need ticket for keyadmin before you can run the curl command Thanks, Pulkit ... View more

Thanks Joe. As I understand, in this scenario we could leave provenance and flowfile repos on the local disks (regular application server sizing), but for content could mount a big fat SAN/NAS/you-name-it and configure HDF to point to that. Are expiration policies configurable per-repository in that case? ... View more

@Andrew Grande pinging to close this 🙂 ... View more

Andrew, to put this in a few bullets for a slide I'd go with Automatically Load Balanced Fault Tolerant Supports Back Pressure When we setup the HA NCM, I think there will be no need to change anything on client side necessarily. Because NiFi provides the interactive command-and-control / immediate feedback, we can supply just a single URL. If NiFi cannot contact that URL it could ask for another. However, if it is able to contact the URL (the vast majority of the time it should be able to), it can auto-discover other nodes from that one. ... View more

I would highly recommend against re-using another ZK quorum for this purpose. The risk of the network partitioning is too high and the benefits aren't clear. As David mentions above, NN doesn't put high load on ZK for leader election. Have each NN HA pair (cluster for that matter) talk to their own ZK quorum within the same network segment. ... View more