Yes you can run ephemeral spark jobs via NiFi please refer https://community.hortonworks.com/repos/64179/launching-spark-jobs-from-nifi.html and https://github.com/diegobaez/PUBLIC/tree/master/NiFi-SnapSpark Note: Please upvote or mark this answer accepted if it resolves you found it useful ... View more

Please refer below answer from https://community.hortonworks.com/questions/140030/druid-installation.html First, yes you can co-locate all those service together. Second in order to get high availability you need to have at least 2 different physical nodes running all the services. Thus you will get HA with a replication of 2. Or you can choose an other combination of collocation where each service is run at least over 2 different nodes. Although ideally you want to have something like this. Node1 Broker Node2 Broker Node3 Router/Overlord/Coordinator/Superser Node4 Router/Overloard/Coordinator/Superset The reason what you need broker to be alone is the fact that broker usually needs way more memory than all the other together therefore you might have special hardware for that. But to keep it simple you can start with collocate all the services X 2 and make sure that broker is not running with another service that needs RAM as well. ... View more

It is generally set to 3 ... View more

Please manually install that missing package libtirpc-devel on the nodes first and then try to proceed with the installation yum install libtirpc-devel Let me know how this goes ... View more

Please make sure that you are using server hostname to access strom UI You will need a valid kerberos ticket on your communication node and after that you will able to access the strom UI fine Note: Please mark this answer as accepted if you found it useful ... View more

Was there is any JDK changes made on the server like upgrade of java or java security changes ? Can you please paste the value of variable jdk.tls.disabledAlgorithms from file $JRE_HOME/lib/security/java.security I believe changes in above parameter may have caused this issue ... View more

have you tried on connecting the port 6667 from outside you can configure additional listners according to your need and can use those ports to connect kafka from outside ... View more

No it will not require those details I have upgraded kerberized HDP cluster from version HDP-2.6.4.0 to HDP-2.6.5.0 using express upgrade There was no need of admin principal password at any point. Upgrade went smoothly. Note: Please upvote and accept this answer if you found it useful ... View more

ranger plugin jar seems to be missing from the server you can copy that and then start the namenode ... View more

Authorization params impacted: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/wasb-authorization.html Authentication params impacted: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/authentication-wasb.html Auditing param impacted: set these values in the ranger-hdfs-audit.xml , for Audit to HDFS on WASB <property> <name>xasecure.audit.destination.hdfs</name> <value>enabled</value> </property> <property> <name>xasecure.audit.destination.hdfs.dir</name> <value>wasb://ranger-audit1@youraccount.blob.core.windows.net</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.account.key.youraccount.blob.core.windows.net</name> <value>YOUR ENCRYPTED ACCESS KEY</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.account.keyprovider.youraccount.blob.core.windows.net</name> <value>org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider</value> </property> <property> <name>xasecure.audit.destination.hdfs.config.fs.azure.shellkeyprovider.script</name> <value>/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh</value> </property> Note: Please upvote or accept the answer if you found it useful ... View more

In the installation process there is step of configure services you can go back to that step then correct the paths namenode directory and datanode directory save that config and then proceed with the installation process further Note: Please upvote and accept the answer if you find it useful ... View more

Yes you can do this by using regex expression Below regex expression should suffice your need /homepath/customer_*/inbox/ If there are only specific number of customers then you can user something like /homepath/customer_[0-9]/inbox/ /homepath/customer_[A-Z]/inbox/ Note: Please upvote and accept this answer if you found it useful ... View more

This is expected behavior if your cluster is stop for extended period of time. When you start the cluster back the last checkpoint becomes very old respective to current time on the cluster server hence it throws that error alert Note: Please upvote and accept this answer if you found it useful ... View more

is your cluster also having ranger enabled ? You will need to grant access in ranger policies too. If you have enabled kerberos then please do kinit after login as hbase user ... View more

@Sanchit Arora As ambari server is now installed you can follow tutorial https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-installation/content/ch_Deploy_and_Configure_a_HDP_Cluster.html to further create cluster using the ambari server above document contains step by step instruction to create cluster using apache ambari server UI Note: Please accept and upvote this answer if it helps you ... View more

@sganeshkumar databases are kept in folder /apps/hive/warehouse If you want to check it in HDFS to see the files associated with the database then use below command hdfs dfs -ls /apps/hive/warehouse If you want to access hive command line from the sandbox shell then first change the user to hive using su - hive then run command hive on hive command line hive> SHOW DATABASES; Note: Please mark this answer accepted and upvote if you find it useful ... View more

can you please give us more information like exactly at what task upgrade is stuck Click in upgrade in progress link and provide screenshot of the task it have executed and task it is stuck at so we can troubleshoot this further accordingly. ... View more