I need to pull the data which is exposed by web service. Am planning to make use of Flume to get the data from web service and store it in HDFS. From where data will be loaded into Hive tables.Need to know 1) Is it a better option to make use of Flume? 2) Any alternative approach available which is better than Flume? Data from web service and another source (oracle) needs to be merged and loaded into Hive. Your views and ideas are welcome. ... View more

Am using oozie 4.2 with kerberos. While submitting jobs, its throwing the error as specified below: Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 1. Exception = Could not authenticate, Authentication failed, status: 500, message: Internal Server Error Oozie server has started and its showing proper status in Ambari dashboard. Am able to run the hive jobs individually. Only while submitting the jobs am getting the authentication error. Can you please help on this.. ... View more

Am trying to setup the web service using HandleHTTPRequest ->InvokeHTTP->FetchHbaseRow->HandleHTTPResponse processor. While starting the nifi process group getting the error: [Timer-Driven Process Thread-6] o.a.n.p.standard.HandleHttpRequest org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server Any ideas on the cause. Please let me know if you need any more info.. ... View more

I have a Hbase table which needs to be exposed as a web service using Nifi. My actual path of usage is HandleHTTPRequest -- ExtractText -- (GetHbase/FetchHbase) -- HandleHTTPResponse. But am unable to connect the processors from request -> hbase-> response at all. Is there anything am missing here? Is it possible to query hbase table with request url param and send the output data in json format to external system using rest api? Is there any limitation (content/size) in using Nifi in sending the content via rest? ... View more

Scenario: Two hive tables with data from entirely different systems created and loaded with data. Need to merge the 2 tables into a single table by performing full outer join which obviously have null values. This merged table has been planned to be created with a partition column. Now with null coming into picture, thinking of removing partition column from here. Now the real question is, how to integrate this merged hive table with Hbase. It has nulls and Hbase doesn't understand null and its not possible to retrieve null data (since its full join, even the primary key column also contains null). How to integrate this hive table with hbase table. All columns and rows must be preserved and null value rows should not be filtered out. Is there any way to achieve this? Or should we be thinking of some other alternative? Any other out of box alternatives are most welcome. ... View more