Prerequisites: Launch Sandbox on Azure VM Size: Minimum of A4 or A5 A Twitter App You'll use the API credentials The "Application Details" don't matter Prepare the Sandbox Connect to SSH & Ambari Connect to the Sandbox using SSH or web console: http://<<ip>>:4200/ Become root: sudo su - Reset the Ambari password: ambari-admin-password-reset Login to Ambari: http://<<ip>>:8080 User: admin Before moving to the next steps, ensure all services on the left are started (green) or in maintenance mode (black). Install NiFi In Ambari, Click "Actions" (bottom left) -> Add Service Choose NiFi and continue through the dialogs. You shouldn't need to change anything NiFi should now be accessible at http:<<ip>>:9090/nifi/ Tune Sandbox The Sandbox is tuned to run on minimal hardware. We need to update the Hive, Tez & YARN configuration for our use case. This could take up to 15 minutes to complete: bash <(curl -sSL https://git.io/vVRPs) Solr & Banana Solr enables the ability to search across large corpuses of information through specialized indexing techniques. Banana is a dashboard visualization tool for Solr. Download the Banana Dashboard curl -L https://git.io/vVRP3 -o /opt/hostname-hdpsearch/solr/server/solr-webapp/webapp/banana/app/dashboards/default.json Update Solr to support Twitter's timestamp format curl -L https://git.io/vVRPz -o /opt/hostname-hdpsearch/solr/server/solr/configsets/data_driven_schema_configs/conf/solrconfig.xml Start Solr JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64 /opt/hostname-hdpsearch/solr/bin/solr start -c -z localhost:2181 Create Solr collection for tweets /opt/hostname-hdpsearch/solr/bin/solr create -c tweets -d data_driven_schema_configs -s 1 -rf 1 ... View more