@c pat Have you done a repolist on your Ambari server node and verified the installed version of the ambari-server package? The agents and server versions must match. If you have the wrong server version installed, remove it, update your repo, and do a 'yum clean all' before reinstalling the ambari-server package. ... View more

Hive tables should be secured with the following steps: Disable "doAs" for Hive. This will make everything run as the hive user on the backend. This is important because then you can control access to the HDFS files to just the hive user and this will allow you to more effectively use column level security. Create a Ranger HDFS policy to give the hive user access to the HDFS files that underly the hive table. Create a Ranger Hive policy to grant users accesses to the tables/columns. This allows the most granular security control of the Hive tables. ... View more

@nejm hadj You can parse a JSON with the EvaluateJsonPath processor. You will need to make sure that the information that your getHTTP processor outputs is strictly JSON, then feed it into EvaluateJsonPath. From there, you can reference the fields in the JSON and forward them on to other processors. Twitter_Dashboard.xml is a Twitter processing template that has a good example of evaluating JSON files in it. Import the template into your NiFi dashboard then instantiate it to check out the EvaluateJsonPath processor to see how it's done: ... View more

@Balaji M It is possible to use a single certificate for the entire cluster. You will need to make sure that the certificate is valid for a range of hosts and not host specific to make this work. The preference for 3rd party vs. self-signed certs will really depend on your security policies within your organization. Third party Certificate Authorities (or CA, e.g. Verisign) are used for purposes of trusting the public key contained in the certificate. They are a "clearing house" that tells you that the public key has been validated and verified to belong to the owner. The difference between 3rd party certs is the level of trust with the CA. For internal uses, where you trust the CA signing the cert and don't need to ask other users to trust the certificate, then there really is no difference between the certificates. Likewise, if you are using the certificate for strictly internal purposes, then there is no difference between a 3rd party signed cert and a self-signed cert. Many organizations maintain an internal CA to generate their own certificates so they don't have to pay a 3rd party to sign the certificates. When installing a self-signed cert or a non-famous CA signed cert, you will need to set up trust for the certificate and/or CA when installing the certificate on the system. Because the process for verifying and trusting the certificate does not vary for 3rd party signed certs vs. self-signed, there is really no performance difference for the type of certificate used. There will be a fairly significant performance penalty for using SSL and certificates for all of the inter-process communication due to the overhead of verifying the certificate, decrypting/encrypting the traffic, etc. This performance penalty will depend on your workload, but I've seen reports of up to 15-20% performance penalty when enabling wire encryption. Because of the complexities of certificate management, performance impacts, etc., you should design the security of your system with all aspects in mind. For example, at-rest data encryption for sensitive data keeps the data encrypted on the wire until it is encrypted/decrypted by the client. Security is a complex topic, so be aware that there may be more than one way to meet your needs. ... View more

@pvyas It looks like you may be running into HIVE-4625. This bug presents when Hiveserver2 is running with doAs enabled (hive.server2.enable.doAs=true). Best practices for securing the cluster call for running Hiveserver2 with doAs disabled because of the ability for a user to skirt Hive authorization policies in Ranger by going directly to HDFS to read files. Here is a link to the article: Best Practices for Hive Authorization. Setting doAs to false should also solve your issue of delegation token errors. This bug is corrected in Hive 1.2 which is included with HDP 2.3 and Hiveserver2 will not ask for delegation tokens from metastore any more. ... View more

@Geoffrey Shelton Okot During the installation, you can (and should) specify where the data for HDFS resides by editing the HDFS configuration parameters before deploying the cluster. By default, Ambari picks up any filesystems besides / and puts them in the list. If you don't edit the namenode and datanode directories, you will be using /tmp, /var, /usr, etc., to store data and metadata if those are separate filesystems on your system. The bits get installed under /usr/hdp. That can not be modified. The packages (RPMs) are built to put things in this standard location, and any 3rd party applications that expect the binaries and configs to be in the standard locations will not be able to function otherwise. Likewise, during the installation, you can specify the service user accounts if you don't wish to use the default usernames. Since, in an unsecured cluster, any user can access the data stored in HDFS, you don't need to consolidate the service accounts in order to be productive right off the bat. If you are securing your cluster, then you won't want to run these services as the same username anyway because it can cause a security hole if you want to separate the users who can access certain functions on the cluster. There is a python script that can be used to clean up a failed installation. If Ambari detects users that already exist or a few other conditions that suggest a failed install, it will recommend that you run this script to clean up the systems before proceeding with the installation. Here is where the script lives and the help information for running it: [root@sandbox ~]# python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --help Usage: HostCleanup.py [options] Options: -h, --help show this help message and exit -v, --verbose output verbosity. -f FILE, --file=FILE host check result file to read. -o FILE, --out=FILE log file to store results. -k SKIP, --skip=SKIP (packages|users|directories|repositories|processes|alt ernatives). Use , as separator. -s, --silent Silently accepts default prompt values ... View more