Created on 11-26-2016 09:46 PM - edited 08-19-2019 03:18 AM
create table IF NOT EXISTS tweets_sentiment stored as orc as select tweet_id,
case when sum( polarity ) > 0 then 'positive' when sum( polarity ) < 0 then 'negative' else 'neutral' end as sentiment from l3 group by tweet_id;
Tihs query gives following error :-
Created 11-27-2016 12:13 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 11-26-2016 09:48 PM
plz help @Mushtaq Rizvi
Created 11-27-2016 12:13 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 11-27-2016 12:17 AM
ok i am trying to delete all docs from solr to restart fresh downloads but can not find a working solution for that. @Mushtaq Rizvi
Created 11-27-2016 12:19 AM
you don't have to delete anything in Solr. You are accessing data from HDFS, not Solr. Delete your HDFS folder /tmp/tweets_staging and then run Nifi workflow.
Created 11-27-2016 12:25 AM
I deleted all files from tweets_staging but solr still showing num docs the same number as before. @Mushtaq Rizvi
Created 11-27-2016 12:33 AM
Solr is not dependent on HDFS directory /tmp/tweets-staging, its getting data from Nifi and storing it in /etc/solr/data_dir, not HDFS directory.
You are getting confused between the usage of all the tools here.
Nifi is fetching data from Twitter API, sending it to Solr to view the streamed data in real-time to gather information. Nifi is also storing the data in HDFS in JSON format, we are creating tables in Hive referencing this new HDFS data to analyze the social sentiment. Lastly, we are using Zeppelin to visualize our dataset
Created 11-27-2016 12:54 AM
Use of solr is not mandatory here right? as its only for search and perform queries on tweets in json format? @Mushtaq Rizvi
Created 11-27-2016 12:55 AM
yes, you are right.
Created on 11-27-2016 01:05 AM - edited 08-19-2019 03:18 AM
When i reload ambari web page then i have to run this query always before running any query -
ADDJAR/usr/hdp/2.5.0.0-1245/hive2/lib/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar;
Or i get the following error for query
select * from tweets_clean;
what is the cause for that ?