Member since
07-14-2017
99
Posts
5
Kudos Received
4
Solutions
02-07-2019
04:04 PM
@Shu can you please help me
... View more
02-06-2019
10:29 AM
Hi All, I have a string String:
some text with an ip 111.111.111.111 and a decimal 11.2323232 and some text here and then an int 1 and then some HTTP/1.1 with a 503 request and then another ip 222.222.222.222 and some imaginary 999.999.999.999 I want to output all the ip addresses in comma saperated. I tried the below select regexp_replace(regexp_replace(String,'[^(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})]',' '),'\\s+',',');
+------------------------------------------------------------------------+--+
| _c0 |
+------------------------------------------------------------------------+--+
| ,111.111.111.111,11.2323232,1,1.1,503,222.222.222.222,999.999.999.999 |
+------------------------------------------------------------------------+--+
Expected output is : 111.111.111.111,222.222.222.222,999.999.999 Could you please help me
... View more
Labels:
- Labels:
-
Apache Hive
08-31-2017
12:02 PM
@Bryan Bende Thanks for the explanation. I have a usecase where syslog listener has to receive 10M messages/sec. I am worried if this can be achieved? because the processing of those messages takes quite good amount of time. Since I have to extract data out of each messages and store it to hdfs in csv, I use extracttext->replacetext->routeoncontent->puthdfs. Can you suggest me if 10Million msg/sec will be achieved.
... View more
08-02-2017
11:24 AM
@Matt Clarke Also, I need some help, thankful if you could guide me. I have a file in hdfs, which have a lot of fields, which I want to put in to hive. e.g: --------------------------------------------------------------------------------- text in hdfs "These are the attributes to save in hive _source="/a/b/c" _destination="/a/b/d" - - _ip="a.b.c.d" text="hive should save these attributes in different columns"". I made an external table in hive with columnns |source |
destination |
ip |
text | I want to get the key value pairs from above text in hdfs and place in hive in respective columns. --------------------------------------------------------------------------------- In hdfs file, a series of such lines are present, they are unordered and not exactly in the same order of source, destination etc. Any suggestion Thankyou
... View more
08-02-2017
09:24 AM
@Matt Clarke Hi Matt, I have followd your suggestion, I got the expected text. As I am new to Nifi, need more learning. And your suggestions helped me.Thank you.
... View more
08-01-2017
03:28 PM
@Matt Clarke I have used your suggestion, but result is same, it fetches the complete line instead of [hdfs....... .log"] for clarification I will let you know the steps which I am following 1. GetHDFS 2. Splittext: count-1. 3. Extract text:
(\[hdfs.*log"\]) 4. Update Attribute 5. PutHDFS not sure why it is pulling complete line? Thanks
... View more
08-01-2017
09:56 AM
Hi ,
I have stream data (GetHDFS will be running continuosly ) which contains number of lines.
e.g:
<start>this is 123_@":text coming from [hdfs file="/a/b/c" and' the; '''', "", file is streamed. The location=["/location"] and log is some.log"] linedelimited.
A stream of above lines of data will be in file
I have to extract text from above message
[hdfs file="/a/b/c" and' the; '''', "", file is streamed. The location=["/location"] and log is some.log"]
I tried using a extract text processor and used custom property
extract: ([hdfs.*log"]).
I tried the above in java regex evaluator, it shows correct text extracted. but when I run the flow, output gets the complete text.
expected: [hdfs file="/a/b/c" and' the; '''', "", file is streamed. The location=["/location"] and log is some.log"]
actual : <start>this is 123_@":text coming from [hdfs file="/a/b/c" and' the; '''', "", file is streamed. The location=["/location"] and log is some.log"] linedelimited.
Please help me to correct the regex to extract correct text.
... View more
Labels:
- Labels:
-
Apache NiFi