Getting a weird issue : "ambari-server setup" fails instantly when running it for the first time. Getting this : "line 84: line: unbound variable" context : Trying to install a new HDP 2.6 cluster onto physical hardware (for a POC). Running CentOS 6.9 - fresh OS install. Following this doc : https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html Added the Ambari repo, installed the ambari package(ambari-server-2.6.1.0-143.x86_64) then proceeded to follow the rest of the documentation but can't find the issue here. It's not the first time I spool up such cluster but it is the first time with this specific version - any ideas what could be the issue? Line 84 is a simple version calculation : "numversion=$(( 10 * $majversion + $minversion))" not sure where this is coming from. ... View more

Hi, First a couple of things that work : -Nifi cluster running on 3 nodes (running Apache upstream V1.1.2) on Centos 6 -Login identity provider is kerberos and works A1 using username & password fields in the UI or though the API. -Once I get an API token using username & password, I can query the API without any issues. -KDC is Active Directory -Service is using nifi.kerberos.service.principal as "serviceaccount@DOMAIN.ORG" with corresponding keytab. Now I'm trying a little POC where I want a script to use the API to interact with some of my flows. Because I don't want to store a username & password for the script to use, I wanted to setup SPNEGO and just use a plain "kinit" with a keytab and fetch the API access token using something like : "curl --negotiate -X POST -v -u : https://myhost.mydomain.org:8989/nifi-api/access/kerberos" Steps I've taken: 1-Create 3 new accounts in AD with the logon name (principal) named "HTTP/myhost.mydomain.org". One for each of of my machine. 2-Created a keytab for each of those machine by using ktutil. Tested this with a "kinit HTTP/myhost.mydomain.org@DOMAIN.ORG" and they work. 3-Setup the 3 SPNEGO properties in nifi.properties : -nifi.kerberos.spnego.principal=HTTP/myhost.mydomain.org@DOMAIN.ORG -nifi.kerberos.spnego.keytab.location=[the_location_of_the_key_tab] -nifi.kerberos.spnego.authentication.expiration=12 hours After a service restart, I try to run the curl command mentioned above and I get the following error : curl --negotiate -X POST -v -u : https://myhost.mydomain.org:8989/nifi-api/access/kerberos * About to connect() to myhost.mydomain.org port 8989 (#0) * Trying [îp address]... connected * Connected to myhost.mydomain.org ([îp address]) port 8989 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: [redacted] CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using [redacted] * Server certificate: * [redacted] > POST /nifi-api/access/kerberos HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: myhost.mydomain.org:8989 > Accept: */* > < HTTP/1.1 401 Unauthorized < Date: Thu, 10 Aug 2017 17:10:41 GMT * gss_init_sec_context() failed: : Server not found in Kerberos databaseWWW-Authenticate: Negotiate < Content-Type: text/plain < Content-Length: 0 < Server: Jetty(9.3.9.v20160517) < * Connection #0 to host myhost.mydomain.org left intact * Closing connection #0 I was wondering what's missing. DNS and Reverse DNS is setup properly and everything else is working fine (like HUE, also using SPNEGO with the same method). Any idea? ... View more

Hello, First time posting here so sorry if this is in the wrong section / wrong format. First, some background : We started a POC using NiFi 1.0.0. We're using a 3 node cluster with limited ressources (this is a POC...). Each of the node has 16 cores, 32gb of ram and 2 volumes : a raid 1 volume for the OS and a Raid 10 volume on 2.5in splindles. I know this is not a recommended setup but the content repo, the provenance repo, the flow files, everything basically, is on the same raid 10 array. The disks are heavily used right now. Content Repo archiving is disabled. Now here's the thing : every 2-3 days, the disk fills up. Right now, the UI reports that we have, in queue : 450 000 (3.21gb). I would expect to have roughly the same amount of data in the nifi/content_repository folder but it's not the case : On one of the node, the content_repo folder is 73gb. I can't tell how big the 2 others nodes are since the "du -h" operation is still running after 10minutes but using "df", I can estimate around 700-800gb on each. When we restart one of the node, it can take hours while the process cleans the content_repo and spams the log with a bunch of "unknown files" Any ideas / Suggestions? This is running on CentOS 6. Thanks Here's the relevant config section : nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.repository.directory.default=./content_repository nifi.content.repository.archive.max.retention.period=1 hours nifi.content.repository.archive.max.usage.percentage=1% nifi.content.repository.archive.enabled=false nifi.content.repository.always.sync=false ... View more