Support Questions
Find answers, ask questions, and share your expertise

NIFI-PROCESSOR : ListenRELP - RELP Frames

NIFI-PROCESSOR : ListenRELP - RELP Frames

Explorer

Hi all,

I will replace listensyslog by listenTCP because i don't want lost message if nifi node crash.

What's format RELP need to be send to listenTCP ?

2016-11-02 17:13:29,455 ERROR [pool-75-thread-3] o.a.nifi.processors.standard.ListenRELP ListenRELP[id=25582216-0158-1000-ffff-ffffee224d94] Error reading RELP frames due to Error decoding RELP frame: For input string: "nifi011": org.apache.nifi.processors.standard.relp.frame.RELPFrameException: Error decoding RELP frame: For input string: "nifi011"
2016-11-02 17:13:29,457 ERROR [pool-75-thread-3] o.a.nifi.processors.standard.ListenRELP
org.apache.nifi.processors.standard.relp.frame.RELPFrameException: Error decoding RELP frame: For input string: "nifi011"
        at org.apache.nifi.processors.standard.relp.frame.RELPDecoder.process(RELPDecoder.java:99) ~[nifi-standard-processors-1.0.0.jar:1.0.0]
        at org.apache.nifi.processors.standard.relp.handler.RELPSocketChannelHandler.processBuffer(RELPSocketChannelHandler.java:70) ~[nifi-standard-processors-1.0.0.jar:1.0.0]
        at org.apache.nifi.processor.util.listen.handler.socket.StandardSocketChannelHandler.run(StandardSocketChannelHandler.java:76) [nifi-processor-utils-1.0.0.jar:1.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.lang.NumberFormatException: For input string: "nifi011"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[na:1.8.0_101]
        at java.lang.Long.parseLong(Long.java:589) ~[na:1.8.0_101]
        at java.lang.Long.parseLong(Long.java:631) ~[na:1.8.0_101]
        at org.apache.nifi.processors.standard.relp.frame.RELPDecoder.processTXNR(RELPDecoder.java:127) ~[nifi-standard-processors-1.0.0.jar:1.0.0]
        at org.apache.nifi.processors.standard.relp.frame.RELPDecoder.process(RELPDecoder.java:76) ~[nifi-standard-processors-1.0.0.jar:1.0.0]

Thanks

5 REPLIES 5

Re: NIFI-PROCESSOR : ListenRELP - RELP Frames

Do you mean replace ListenSyslog with ListenRELP?

RELP is a specific protocol mostly developed by rsyslog, you can find a description here:

http://www.rsyslog.com/doc/relp.html

You can also try to see what is going on by looking at the unit test for ListenRELP:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-proce...

Re: NIFI-PROCESSOR : ListenRELP - RELP Frames

Explorer

@Bryan Bende

Like I know ListenSyslog don't to ACK when receive syslog messages so I would use ListenRELP to my dataflow to ingest syslog messages.

Before, I have : ListenSyslog ==> MergeContent ==> PutHDFS

After : ListenRelp ==> ParseSyslog ==> MergeContent ==> PutHDFS

For using ListenRelp, I need to install librelp, rsyslogrelp on the nifi node and the client ?

Re: NIFI-PROCESSOR : ListenRELP - RELP Frames

You don't need to install anything on the NiFi node, it is not dependent on any external libraries. ListenRELP just receives data over TCP, and uses Java networking libraries to read and write RELP data.

Re: NIFI-PROCESSOR : ListenRELP - RELP Frames

Explorer

@Bryan Bende : I'm trying to send data use 'nc' command but I've this error.

[client01 ~]$ LOG="INFO [Process Cluster Protocol Request-2] o.a.n.c.p.im"
[client01 ~]$ echo $LOG | nc nifi011 10003

9163-relp-error.jpg


					
				
			
			
				
			
			
			
			
			
			
			
		

Re: NIFI-PROCESSOR : ListenRELP - RELP Frames

The data you echoed is not a RELP frame. There is a detailed description of what a RELP Frame is here:

http://www.rsyslog.com/doc/relp.html

RELP-FRAME = HEADER DATA TRAILER
DATA = [SP 1*OCTET] ; command-defined data, if DATALEN is 0, no data is present
HEADER = TXNR SP COMMAND SP DATALEN
TXNR = NUMBER ; relp transaction number, monotonically increases, starts at 1
DATALEN = NUMBER
#old:COMMAND = "open" / "syslog" / "close" / "rsp" / "abort" ; max length = 32
COMMAND = 1*32ALPHA
TRAILER = LF ; to detect framing errors and enhance human readibility
ALPHA = letter ; ('a'..'z', 'A'..'Z')
NUMBER = 1*9DIGIT
DIGIT = %d48-57
LF = %d10
SP = %d32

RSP DATA CONTENT:
RSP-HEADER = TXNR SP RSP-CODE [SP HUMANMSG] LF [CMDDATA]
RSP-CODE = 200 / 500 ; 200 is ok, all the rest currently erros
HUAMANMSG = *OCTET ; a human-readble message without LF in it
CMDDATA = *OCTET ; semantics depend on original command
TXNR is as in the relp frame, it is the TXNR of the frame being responded to.

Here is a code in NiFi that creates a RELP frame:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-proce...