About bigspark

bigspark · ‎07-06-2016

@Simon Elliston Ball thanks for the reply. Iam trying to filter the kafka json events on eventname and based on the eventaname need to insert the events into cassandra. So i believe i need EvaluateJsonPath -> RouteOnAttribute -> UpdateAttribute for filtering events with event Name? thanks

bigspark · ‎07-06-2016

@Mukesh Kumar i think you are right but just wondering for only filtering the events dont u think its a overkill to use storm as we have to implement lot of code and maintain a big cluster for storm whereas we can do it quickly in Nifi. What do u say?

bigspark · ‎07-05-2016

@Bryan Bende i was looking to use Nifi as well but not sure if it would work. However i have created the PutCassandraQL processor and in terms of insert query not sure how to pass it as input to the putcassandra processor? My Kafka json event is complex and nested. Currently iam filtering on eventName using EvaluateJsonPath and RouteOnAttribute processors but i need to pass the input to putcassandra processor in a cql insert query format i guess and iam not aware of it. Any suggestions to pass the insert query to putcassandra. thank u

bigspark · ‎07-05-2016

Consuming kafka json events and filtering on event type & event name and then finally save them to cassandra. Which is better kafka->Storm->Cassandra or kafka->Nifi->Cassandra?

bigspark · ‎07-05-2016

hi @chris grant would you like to explain the process how you passed query to putcassandraQL processor. I have managed to use QueryCassandra processor and succesfully retrieve data from cassandra. Plz let me know the steps you followed thank you 🙂

bigspark · ‎07-05-2016

Looking to extract tables from sql server using nifi and export it to s3 in either csv or .txt format. Iam using ExecuteSQL processor and it outputs avro format but we are looking for csv or txt (dont know if json is any good). ANy out of the box processors you suggest or do we have to create a processor? plz let me know thank u. For Avrotojson there is a processor but looking for AvrotoCSV.

bigspark · ‎07-05-2016

Just wondering if anyone has any suggestions on Unit Testing & Integration Testing Nifi. Want to make these tests automated as well. Is there any framework or examples currently to acheive these. Thank you.

bigspark · ‎06-27-2016

hi @kkawamura i have used the same parameters zookeeper url & port and also the topic name. I thought it was about kafka versions but my issue was network connection issues with zookeeper. But its resolved now. Thank you 🙂 However as you looked at the other question just wondering any suggestions on connecting to S3 from as iam getting permission issues as described in the other question. Thank you

bigspark · ‎06-21-2016

@jwitt iam using kafka 0.10.0.0. Just wondering the GetKafka processor doest have kafka broker config property how does it connect to kafka. It has zookeper property and topic to connect to. Any more properties i need to add. With just the default properties iam unable to connect to kafka running locally. thanks.

bigspark · ‎06-20-2016

ERROR 3491c89e-35c2-4443-a6b8-03303193629a GetKafka[id=3491c89e-35c2-4443-a6b8-03303193629a] GetKafka[id=3491c89e-35c2-4443-a6b8-03303193629a] failed to process session due to java.lang.IllegalStateException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve partitions for topic test: java.lang.IllegalStateException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve partitions for topic test Please let me know if i need to more properties rather than just the default properties in getKafka processor. Please let me know. thanks

Online	Offline
Last Visited	‎10-19-2020 08:16 AM

Member Since	‎09-25-2015 09:05 AM
Last Visited	‎10-19-2020 08:16 AM
Posts	112
Kudos received	37

Cloudera Community

Re: Spark Testing base 1.6.1_0.3.3 for spark versi...

Re: Kafka json events filter on event name & event...

Re: Kafka json events filter on event name & event...

Re: Kafka json events filter on event name & event...

Kafka json events filter on event name & event typ...

Re: Can I use the PutCassandraQL NiFi processor to...

In NiFi is there Any processor for converting Avro...

Testing Nifi Unit & Integration Testing & automati...

Re: nifi 0.6.1 getkafka processor error: cannot co...

Re: nifi 0.6.1 getkafka processor error: cannot co...

nifi 0.6.1 getkafka processor error: cannot connec...