About regie_canada

regie_canada · ‎09-11-2017

Hi, Im trying the basic concat using QueryRecord Processor in NIFI. sample firstname +' '+ lastname, but im having an error. Please see the attached image. How can i concat in this processor? Thank you.

regie_canada · ‎08-31-2017

Hi @Wynner, sorry for the late response. I tried it and its working. Thank you so much! But i need to delete the file so it will just fetch 1 time.

regie_canada · ‎08-23-2017

Hi, Im throwing multiple csv files on my hdfs every minute using logstash. I need to get the files from the past minute from the current time. Im using nifi in this process. For example right now is 11:30 AM, I need to get ONLY all the files that are saved 1 minute ago or 11:29AM. What is the best approach here using nifi? Thank you.

regie_canada · ‎08-17-2017

It worked! thank you so much. can this query record make aggregation in real time manner? like ill accept data every second then go save it in hbase, but i always should update the data in hbase.

regie_canada · ‎08-16-2017

Hi, How can i aggregate my csv data in nifi? Can you please give me options. I have csv coming every hour. For example my csv data is: Name, Subject stud1, math stud2, english stud3, math stud4, math stud5, english stud6, science My needed output is: math, 3 english, 2 science, 1 Can i make this without using any execute scripts? Thank you.

regie_canada · ‎07-06-2017

Hi, Im running a usecase where i throw events per second(1-20k) in solr. Now im making a dashboard in banana ui, where i tracked the events coming for span of 5 minutes, and im tracking it per second. You can view the attached image. My problem now is, it is possible to get the peak or the max number of events in the span of 5 mins, group by timestamp. Then also get the average number of events for the last 5 mins. I hope you understand my question. Thank you.

regie_canada · ‎06-22-2017

@Pierre Villard Thanks for the response sir, hmm. sorry about the screenshot cause i already tried the listensyslog but they have the same error. But trying to stop the logstash which is also listening to this port made it. Thank you so much!

regie_canada · ‎06-22-2017

Hi Guys, I have a log simulator, which is sending 10k events per second on my port 5043. I used the ListenTCP Processor to fetch the events but im having an error. 1. 2. So, i tried to lowered the Socket Buffer Size. The first image error was gone, but the second error is still showing. Am i correct that i can already listen to 5043 port and i'm still getting this error(second pic), because my nifi cannot handle the amount of events? I'm sorry, newbie here. Thank you. Regie

regie_canada · ‎05-31-2017

Hi, How can i off the multivalued in solr? I tried to edit the manage-schema file inside /data_driven_schema_configs/conf/ folder and set all multivalued="true" to false. Then restart solr and then create new collection, but nothing changed. I need this because i need to sort my data in the table in banana ui. Thank you.

regie_canada · ‎05-24-2017

@Matt Clarke Thanks sir. Anyway for addition question, is there a processor that will change this Line1 : value Line2 : value Line3 : valu Line4: value to JSON format? Thank again.

regie_canada · ‎05-24-2017

@Matt Clarke @Eyad Garelnabi Thank you so much!!

regie_canada · ‎05-23-2017

@Eyad Garelnabi

regie_canada · ‎05-23-2017

Hi sir, thanks for the reply. I need all the first 4 lines in every ####################################################################### START of Request ####################################################################### sir. it has a multiple request in every file.

regie_canada · ‎05-19-2017

Hi guys, i would like to ask, if this is possible in nifi alone without using any script execution. test.txt <--- so this file of mine need to be extracted. Please see the attached file. I just need to get the VALUE of line1 to line4 and save it to hbase. The problem here is it has multiple request in 1 file. I need to get all the line1 to line4 per request. PS. the count of lines per requests are different. My file is just an example of what the file looks like. Thank you.

regie_canada · ‎03-08-2017

Thanks sir @Artem Ervits i'll try this one.

regie_canada · ‎03-07-2017

Hi, Im using NiFi here and the data flow is running every minute. Im fetching csvs from a web service and put it in hdfs. Now every file inserted in HDFS, i need to save it to HBASE. But I need to delete rows that is over 2 hours ago. For example the time now is 11:03AM and i have a records/rows that is inserted at 9:03 AM, when the time became 11:04AM i need to delete the records that was insert at 9:03AM. This process of deleting records should also run every minute. And this deleted records also need to be deleted in SOLR+Banana UI. Is this possible? Thanks.

regie_canada · ‎02-22-2017

Can i use SolR + Banana Ui as Big Data reporting? Im looking for realtime update reporting for our project and i saw this Solr. Now im trying it and its fine. But the problem is, when im running my nifi job going to Solr, it will fucntion well but after 20 secs the solr and banana started to crashed and and the service stops automatically. I saw that the JVM Memory is full. By the way im using sandbox. Any tips how can i fixed this? Thank you! Regie

regie_canada · ‎02-08-2017

@vadivel sambandam hi sir, have you answer this? i really need this too. thank you.

regie_canada · ‎02-08-2017

No sir, its just a sample. any param name causes error.

regie_canada · ‎02-08-2017

For example sir is, select * from table where date = @date; Hive is not reading that style of parameter. I've searched that they are making somewhat like ${hiveconf:param}, but if i do that, there is an error in ssrs.

regie_canada · ‎02-08-2017

Hi guys, Have any of you tried this? Im using SSRS for my reporting and im using hive as source. Thank you.

regie_canada · ‎01-24-2017

Is there any NiFi processor i can use in executing SQOOP? If none, any data flow i can use in getting my table then save it to HDFS? Thanks.

regie_canada · ‎01-17-2017

My table in hive have more than 80 million records. Now we have to make a back-up of it. Do you have a best practice to do this? i tried to use my data flow in nifi but it is hanging. Now im trying PUTTY (hive -e 'select * from your_Table' | sed 's/[\t]/,/g' > /home/yourfile.csv), nothing happened when i run it. Im using sandbox hdp 2.5. Thanks!

regie_canada · ‎01-10-2017

Hi Guys, Im trying to make a data flow that will save my data to HBASE. Recently my error is like this http://ec2-52-89-10-2.us-west-2.compute.amazonaws.com/questions/63933/cant-connect-to-hbase-through-nifi-100.html Then trying the answers they have given i have another error.

regie_canada · ‎01-05-2017

@Timothy Spann thanks for the response sir. yes i used that version of soap processor and still get the error. I think the problem is in my soap security?

regie_canada · ‎01-04-2017

Hi guys, Im using the GetSoap processor in NiFi. I need to get my data in web service. But it always have an error, GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] failed to process session due to java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;: java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog; Im using https, is this the one that causes the error? Thank you.

regie_canada · ‎01-04-2017

@Timothy Spann Hi sir, i used this nifi-soap processor (GETSOAP), but i always get an error. GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] failed to process session due to java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;: java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog; can you help me about this?

regie_canada · ‎01-04-2017

@Rahul Reddy Hi sir, may i know where did you put the wsdl url in the http processor? thank you.

regie_canada · ‎12-15-2016

@Karthik Narayanan Thank you so much for answering. Do I need to install anything to run a python script?

regie_canada · ‎12-15-2016

@Karthik Narayanan Hi sir, yes it is XSL. We made a program that will turn our XML to XSL. This is the current sample output: 1000002746|CRl Campbell|905-899-3495|905-899-3495|lrobert@crlcampbell.com|11675 Burnaby Rd ||Wainfleet|province-ONTA|L0S1V0|Active|HostedIPV_1000003883_SA We use PIPE for delimiter. and now I need to make it csv so i can convert it to json. Thank.

Online	Offline
Last Visited	‎12-07-2017 10:39 AM

Date Registered	‎11-18-2016 05:58 AM
Last Visited	‎12-07-2017 10:39 AM
Total Messages Posted	38
Total Kudos Received	7

How to Concat string in QueryRecord in Nifi?

Re: Fetch file last one minute ago from the curren...

Fetch file last one minute ago from the current ti...

Re: How to aggregate csv or json data in Nifi?

How to aggregate csv or json data in Nifi?

Needs help in SOLR Query

Re: Nifi ListenTCP Processor Error

Nifi ListenTCP Processor Error

Solr MultiValued off

Re: Nifi Extraction

Re: Nifi Extraction

Re: Nifi Extraction

Re: Nifi Extraction

Nifi Extraction

Re: How can I delete Records in Hbase and SolR tha...

How can I delete Records in Hbase and SolR that is...

Is it fine to use SolR + Banana UI for Big Data Re...

Re: How to pass parameter From SSRS to Hive ?

Re: Passing Parameters from SSRS to Hive

Re: Passing Parameters from SSRS to Hive

Passing Parameters from SSRS to Hive

Execute sqoop on NiFi

Export Hive Tables Using Putty

Connecting in HBASE Error using NiFi

Re: GETSOAP Processor in NiFi error.

GETSOAP Processor in NiFi error.

Re: Get SOAP data from SOAP webservice using Apach...

Re: Get SOAP data from SOAP webservice using Apach...

Re: XSL to CSV using NiFi

Re: XSL to CSV using NiFi