Okay, after "res" insert this: res1 = foreach (group res BY word) { tweets = foreach res generate id, user_id, created_time, created_date, text; generate group as pattern, tweets; } The inner foreach is to get rid of the "word" associated which each output recored. Try "res2 = group res BY word" to see the difference. And please accept & up-vote the answer. ... View more

Can you insert "describe twitter; describe c;" after the CROSS statement, and find the output. If you loaded "twitter" like in your post, twitter::text should be there... ... View more

For command line input, as Michael said, check your LANG settings, it should end in "UTF-8" but it doesn't have to be "en_US", for example for Japanese it's "ja_JP.UTF-8". As for the Hive view, CVJK languages don't work and I'm afraid Europian umlauts don't work either. The fix is coming in Ambari-2.4. ... View more

You omitted $0: aa =foreach avg_rate generate $0 as var1,... and then try to store using HCatalog per your original question. ... View more

It seems something is wrong with hdfs, can you try "hdfs dfs -ls /"? If it doesn't work can you go to HDFS->Configs and check "Name node directories" and "Data node directories". Remove any "unwelcome" members from there, like "/tmp" (Ambari will suggest volumes on all mounting points). Then make sure the remaining directories are owned by user "hdfs". [NN dirs must exist on NN and Secondary NN node, DN dirs must exist on all data nodes.] Finally, restart hdfs, and check NN and DN logs in /var/log/hadoop/hdfs. ... View more

The built-in JsonLoader has a somewhat limited functionality and expects all entries (tweets) to have the same order of elements as given in the Pig schema. So, first make sure this condition is satisfied. For example, you have in your schema "id:int" but in the record returned by warnings you don't have an integer element at that position. Also, element names are not preserved, Pig takes them one by one as given in the input, so you can as well name them a, b, c, ... You may also wish to try Elephant Bird JsonLoader which has more advanced features. ... View more

You have a 404, not found error by yum. Login to one node where ambari.repo is available and try "yum clean all", followed by "yum repolist" and "yum install ambari-agent". If either of these commands produce error, check and adjust if needed your base URL in ambari.repo, and also make sure that the node (and all other nodes) can access your web server running on lexbz1187. ... View more