About hemant_dindi

hemant_dindi · ‎03-18-2016

Hi, I tried upgrading non kerberized HDP 2.3.2 cluster to HDP 2.4. During the upgrade process the Oozie service check failed. Did anyone encounter this? Thanks.

hemant_dindi · ‎01-05-2016

Thanks a lot @Robert Metzger

hemant_dindi · ‎12-22-2015

@Robert Metzger, I have prepared the Custom HTTPJSONStream reader and here is the attached code. loganalytics.txt-> This is the main Class which fetches the data from the custom source. In our case its JSON via HTTP url. httpjsonstream.txt -> This class implements the SourceFunction and provides a SourceContext of custom-type FlinkJSONObject flinkjsonobject.txt -> This class uses java.net.* and javax.json.* to connect to provided URL and get the content in JSON Object format. The logic can be applied on the JSONObject to get the desired results. Please suggest if any changes can be made.

hemant_dindi · ‎12-22-2015

I agree @Jonas Straub. When we have no other choice, we have make changes in the metastore.

hemant_dindi · ‎12-22-2015

@Blair Vanderlugt Can you please let us know the error that you are getting while restarting the service. If you want to forcefully delete the terminated node, I guess you can delete the host entry in the table "hosts" from the ambari database. I have never tried this. 🙂

hemant_dindi · ‎12-22-2015

@Darpan Patel Pig view doesn’t seem to support NN HA. We encountered issues with PIG during our recent upgrade. In order to fix this, We created 2 Pig views, one for each NN. +@Predrag Minovic

hemant_dindi · ‎12-21-2015

Thanks @Ali Bajwa @Robert Metzger @Stephan Ewen I was trying to read a Web log which is accessible through a HTTP URL. I used a InputStreamReader to read the content from the HttpConnection. As InputStreamReader is not serializable, I got the above exception. Should we prepare a custom SourceFunction to handle HTTP Streams or does Flink provide any methods/objects in order to enable users to read HTTP streams?

hemant_dindi · ‎12-21-2015

Thanks @Neeraj Sabharwal For HTTP, I am in a state where I have to write a custom SourceFunction. Before I try it, I am exploring if there is a possibility to Handle HTTP Requests by using any of the existing StreamExecutionEnvironment functions/objects. I am also trying to handle HTTP Stream for Flink using NiFi Connector as mentioned here I hope either of these will help me in achieving desired results. 🙂

hemant_dindi · ‎12-21-2015

Which methods/objects does Flink provide in order to enable users to read HTTP streams?

hemant_dindi · ‎12-20-2015

Below is the expeption - Caused by: java.io.IOException: Failed to deserialize an element from the source. If you are using user-defined serialization (Value and Writable types), check the serialization functions. Serializer is org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer@599fcdda at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:55) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:218) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584) at java.lang.Thread.run(Thread.java:745)

Online	Offline
Last Visited	‎10-08-2018 06:51 PM

Member Since	‎12-10-2015 02:48 AM
Last Visited	‎10-08-2018 06:51 PM
Posts	32
Kudos received	11

Cloudera Community

Re: Ambari:Problem accessing /webhdfs/v1/ats/done ...

Re: How to : HTTP Stream in flink

Re: YARN Capacity Scheduler Queue utilisation grap...

Upgrade from HDP 2.3.2 to HDP 2.4 - Oozie service ...

Re: How to : HTTP Stream in flink

Re: How to : HTTP Stream in flink

Re: Force delete or move services from terminated ...

Re: Force delete or move services from terminated ...

Re: Configuring ambari views on Kerberized Cluster

Re: Error while creating a DataStream using fromEl...

Re: How to : HTTP Stream in flink

How to : HTTP Stream in flink

Error while creating a DataStream using fromElemen...