Prerequisites  Base cluster with Hive on Tez, Zookeeper, Spark services installed  DDL files copied to the base cluster Steps Using beeline Kinit with hive user to base cluster host where Zookeeper is installed. We can get Zookeeper using CM client services. Example: Execute beeline command on Zookeeper host. Sample beeline URL: beeline -f {data.sql} -u 'jdbc:hive2://{zookeeper_server}:2181/default;" \ f"serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore={crt.jks};trustStorePassword={password} Using Impala shell   Kinit with impala user to base cluster host where Zookeeper/impala-server is installed. We can get Zookeeper, impala server using CM client services  Execute impala-shell command on Zookeeper/Impala host. Sample impala-shell command:  impala-shell -i {impala_server}:25003 -d default -k --ssl -f {data}.sql ... View more

In this article, we will learn how to build Twitter NiFi template in a flow management datahub. Pre-Requisite  CDP environment is created and flow management datahub is installed. In that, the required users are added. Steps Login to NiFi by clicking on the NiFi log as shown below: Add GetTwitter process group and put a file from the process group as follows: Configure Twitter process group by right-clicking on GetTwitter process group, and then select Configure; add required parameters: Consumer Key, Consumer Secret, Access Token, Access Token Secret as shown as follows: Configure the PUT file process to store the Twitter JSON files in one of the NiFi hosts. To verify the data received, log in to one of the NiFi hosts, and check the directory mentioned in the above step.  ... View more