Member since
11-18-2016
38
Posts
7
Kudos Received
0
Solutions
09-11-2017
07:02 AM
Hi, Im trying the basic concat using QueryRecord Processor in NIFI. sample firstname +' '+ lastname, but im having an error. Please see the attached image. How can i concat in this processor? Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
08-31-2017
03:26 AM
Hi @Wynner, sorry for the late response. I tried it and its working. Thank you so much! But i need to delete the file so it will just fetch 1 time.
... View more
08-23-2017
05:19 AM
Hi, Im throwing multiple csv files on my hdfs every minute using logstash. I need to get the files from the past minute from the current time. Im using nifi in this process. For example right now is 11:30 AM, I need to get ONLY all the files that are saved 1 minute ago or 11:29AM. What is the best approach here using nifi? Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
08-17-2017
03:09 AM
It worked! thank you so much. can this query record make aggregation in real time manner? like ill accept data every second then go save it in hbase, but i always should update the data in hbase.
... View more
08-16-2017
09:53 AM
Hi, How can i aggregate my csv data in nifi? Can you please give me options. I have csv coming every hour. For example my csv data is: Name, Subject stud1, math stud2, english stud3, math stud4, math stud5, english stud6, science My needed output is: math, 3 english, 2 science, 1 Can i make this without using any execute scripts? Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
06-22-2017
10:34 AM
@Pierre Villard Thanks for the response sir, hmm. sorry about the screenshot cause i already tried the listensyslog but they have the same error. But trying to stop the logstash which is also listening to this port made it. Thank you so much!
... View more
06-22-2017
09:57 AM
Hi Guys, I have a log simulator, which is sending 10k events per second on my port 5043. I used the ListenTCP Processor to fetch the events but im having an error. 1. 2. So, i tried to lowered the Socket Buffer Size. The first image error was gone, but the second error is still showing. Am i correct that i can already listen to 5043 port and i'm still getting this error(second pic), because my nifi cannot handle the amount of events? I'm sorry, newbie here. Thank you. Regie
... View more
Labels:
- Labels:
-
Apache NiFi
05-24-2017
07:13 AM
@Matt Clarke Thanks sir. Anyway for addition question, is there a processor that will change this
Line1 : value
Line2 : value
Line3 : valu
Line4: value to JSON format? Thank again.
... View more
05-23-2017
03:20 AM
Hi sir, thanks for the reply. I need all the first 4 lines in every ####################################################################### START of Request ####################################################################### sir. it has a multiple request in every file.
... View more
05-19-2017
06:27 AM
1 Kudo
Hi guys, i would like to ask, if this is possible in nifi alone without using any script execution. test.txt <--- so this file of mine need to be extracted. Please see the attached file. I just need to get the VALUE of line1 to line4 and save it to hbase. The problem here is it has multiple request in 1 file. I need to get all the line1 to line4 per request. PS. the count of lines per requests are different. My file is just an example of what the file looks like. Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
03-08-2017
03:02 AM
Thanks sir @Artem Ervits i'll try this one.
... View more
03-07-2017
07:00 AM
Hi, Im using NiFi here and the data flow is running every minute. Im fetching csvs from a web service and put it in hdfs. Now every file inserted in HDFS, i need to save it to HBASE. But I need to delete rows that is over 2 hours ago. For example the time now is 11:03AM and i have a records/rows that is inserted at 9:03 AM, when the time became 11:04AM i need to delete the records that was insert at 9:03AM. This process of deleting records should also run every minute. And this deleted records also need to be deleted in SOLR+Banana UI. Is this possible? Thanks.
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache NiFi
-
Apache Solr
02-22-2017
07:16 AM
1 Kudo
Can i use SolR + Banana Ui as Big Data reporting? Im looking for realtime update reporting for our project and i saw this Solr. Now im trying it and its fine. But the problem is, when im running my nifi job going to Solr, it will fucntion well but after 20 secs the solr and banana started to crashed and and the service stops automatically. I saw that the JVM Memory is full. By the way im using sandbox. Any tips how can i fixed this? Thank you! Regie
... View more
Labels:
- Labels:
-
Apache Solr
02-08-2017
03:55 AM
For example sir is, select * from table where date = @date; Hive is not reading that style of parameter. I've searched that they are making somewhat like ${hiveconf:param}, but if i do that, there is an error in ssrs.
... View more
02-08-2017
03:16 AM
1 Kudo
Hi guys, Have any of you tried this? Im using SSRS for my reporting and im using hive as source. Thank you.
... View more
Labels:
- Labels:
-
Apache Hive
01-24-2017
06:20 AM
2 Kudos
Is there any NiFi processor i can use in executing SQOOP? If none, any data flow i can use in getting my table then save it to HDFS? Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Sqoop
01-17-2017
09:47 AM
My table in hive have more than 80 million records. Now we have to make a back-up of it. Do you have a best practice to do this? i tried to use my data flow in nifi but it is hanging. Now im trying PUTTY (hive -e 'select * from your_Table' | sed 's/[\t]/,/g' > /home/yourfile.csv), nothing happened when i run it. Im using sandbox hdp 2.5. Thanks!
... View more
Labels:
- Labels:
-
Apache Hive
01-05-2017
03:12 AM
@Timothy Spann thanks for the response sir. yes i used that version of soap processor and still get the error. I think the problem is in my soap security?
... View more
01-04-2017
09:55 AM
1 Kudo
Hi guys, Im using the GetSoap processor in NiFi. I need to get my data in web service. But it always have an error, GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] failed to process session due to java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;: java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog; Im using https, is this the one that causes the error? Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
01-04-2017
09:51 AM
@Timothy Spann Hi sir, i used this nifi-soap processor (GETSOAP), but i always get an error. GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] GetSOAP[id=63baafa9-0159-1000-e53d-5195bbfe3819] failed to process session due to java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;: java.lang.NoSuchMethodError: org.apache.nifi.processors.soap.GetSOAP.getLogger()Lorg/apache/nifi/logging/ProcessorLog;
can you help me about this?
... View more
01-04-2017
06:52 AM
@Rahul Reddy Hi sir, may i know where did you put the wsdl url in the http processor? thank you.
... View more
12-15-2016
06:56 AM
@Karthik Narayanan Thank you so much for answering. Do I need to install anything to run a python script?
... View more
12-15-2016
06:11 AM
@Karthik Narayanan Hi sir, yes it is XSL. We made a program that will turn our XML to XSL. This is the current sample output: 1000002746|CRl Campbell|905-899-3495|905-899-3495|lrobert@crlcampbell.com|11675 Burnaby Rd ||Wainfleet|province-ONTA|L0S1V0|Active|HostedIPV_1000003883_SA We use PIPE for delimiter. and now I need to make it csv so i can convert it to json. Thank.
... View more
12-15-2016
05:41 AM
Hi guys, Is there any process where i can make my XSL File to CSV File? Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi