Hi Last week I wrote a version of my XML2CSV but for AVRO It is still under improvement but you can check it out here Git Repo xml2avro Article XML2Avro NiFi processor ... View more

In case anyone still need this and get the same problem as me, when using kerberos HDP. You need to follow these steps to setup HTTP authentication before spnego would work for the YARN REST API Once these are added in ambari and corresponding services restarted, I was able to get new application ID. However any previous anonymous access to Resource Manager now needs authentication! [user@master hdfs]$ curl --negotiate -u : -v -X POST http://<fqdn>:8088/ws/v1/cluster/apps/new-application * About to connect() to <fqdn> port 8088 (#0) * Trying 192.168.33.11... connected * Connected to <fqdn> (192.168.33.11) port 8088 (#0) > POST /ws/v1/cluster/apps/new-application HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: <fqdn>:8088 > Accept: */* > < HTTP/1.1 401 Authentication required < Cache-Control: must-revalidate,no-cache,no-store < Date: Thu, 18 May 2017 13:42:56 GMT < Pragma: no-cache < Date: Thu, 18 May 2017 13:42:56 GMT < Pragma: no-cache < Content-Type: text/html; charset=iso-8859-1 < WWW-Authenticate: Negotiate < Set-Cookie: hadoop.auth=; Path=/; Domain=<my domain>; HttpOnly < Content-Length: 1427 < Server: Jetty(6.1.26.hwx) < * Ignoring the response-body * Connection #0 to host <fqdn> left intact * Issue another request to this URL: 'http://<fqdn>:8088/ws/v1/cluster/apps/new-application' * Re-using existing connection! (#0) with host <fqdn> * Connected to <fqdn> (192.168.33.11) port 8088 (#0) * Server auth using GSS-Negotiate with user '' > POST /ws/v1/cluster/apps/new-application HTTP/1.1 > Authorization: Negotiate YIICaQYJKoZIhvcSAQICAQBuggJYMIICVKADAgEFoQMCAQ6iBwMFAAAAAACjggFtYYIBaTCCAWWgAwIBBaEJGwdMQkcuQ09NoikwJ6ADAgEDoSAwHhsESFRUUBsWd29ya2VyLnNhbmRib3gubGJnLmNvbaOCASYwggEioAMCARKhAwIBAaKCARQEggEQYSIFHaKqdpjxjhuBgc+K8swimOG+UeIxhgNYhOEQXgfFcyQoNRcPwFRS0nbXeLN8HT8S8QEib5/KXJoPj0On7r7gWNDlcYEI9ycAJ8xe11FE5WTMgSL2BDeiOtA6OLLYGj5rHFCwsWByLBwBu8jI5Bmmnx93jN+XkjPWxvrS3dBwU3qDiwbWfqze34JDfLBAWJBjke0KcFCrzA9an4fw7Evvflu9NtT/XixW7edfF0+anV/tcrBSPqj1UFKqqNr2bYOdes3pApixmohe9xAvCd4Wg6T5JLUwRlbfdt/beqwMwkY0a1WpnnFOeuOoB6ReUIcsufmRZGMkrIh63mIz/O13lbQlzXOhjBfwKyiMo/Kkgc0wgcqgAwIBEqKBwgSBvwZfOYFelpjopPr89JOyFtKzPC6xxCyLjNGAZHMFF/VKHKtdytbf7Dy5YNtcoCK1nu2D8Ihkum1hYaxH1ugK4i5sKU8xaAp0qNanc6Lu+Y7sUH/s5XKCqwVQM96mYC0ejpWIq8WDrB3CX5+MshSOnbeIEcMyG8puQ/5nHfUlNsOC7vhq4Qbs8yTTqG+9W7+79sl9fbhmVqIOx5UUfHXtq3qkKAtgmSoQhpDi4ERC/bYBIMYyubtPiXKC/k0JxSyn > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: <fqdn>:8088 > Accept: */* > < HTTP/1.1 200 OK < Cache-Control: no-cache < Expires: Thu, 18 May 2017 13:42:56 GMT < Date: Thu, 18 May 2017 13:42:56 GMT < Pragma: no-cache < Expires: Thu, 18 May 2017 13:42:56 GMT < Date: Thu, 18 May 2017 13:42:56 GMT < Pragma: no-cache < Content-Type: application/json < Set-Cookie: hadoop.auth="u=user&p=user@<my domain>&t=kerberos&e=1495150976195&s=NxiE0Svo7+3QTPXC8L9aUlPN54c="; Path=/; Domain=<my domain>; HttpOnly < X-Frame-Options: SAMEORIGIN < Transfer-Encoding: chunked < Server: Jetty(6.1.26.hwx) < * Connection #0 to host <fqdn> left intact * Closing connection #0 {"application-id":"application_1495114416899_0001","maximum-resource-capability":{"memory":4096,"vCores":6}}[user@master hdfs]$ ... View more