Member since
02-15-2023
1
Post
0
Kudos Received
0
Solutions
03-01-2023
09:19 AM
I have a pretty simple setup. We have two NiFi servers on different networks, and they need to transfer data between them. I have site to site set up and working using HTTPS, but for some reason it seems like every single file transfer takes a minimum of 15 seconds. A 1 byte file will take barely over 15000ms, at a transfer rate of a couple hundred bytes per second, or a much larger file will transfer in just under 30 seconds, at a transfer rate of about 85MB/s. I'm guessing the actual transfer of the file is happening quickly, but the transfer speed takes into account the 15 second handshake....or whetever the process is that is taking 15 seconds each time. I looked around and couldn't find anyone else experiencing quite the same thing. I have tried using RAW instead of HTTPS, but I haven't found a working setup for that yet--every time I try I end up with the "Magic Header not found" message. Here are some snippets of my config files, let me know if there is anything obviously wrong. A suggestion for how exactly to set up a RAW connector would be appreciated as well, since I've heard those have better performance anyway. Server A:
# Site to Site properties
nifi.remote.input.host=$localhostname (not a variable, my superiors are just paranoid about sharing too much information)
nifi.remote.input.secure=true
nifi.remote.input.socket.port=443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
nifi.web.https.host=$localhost IP
nifi.web.https.port=8443
nifi.web.https.network.interface.default=
nifi.web.https.application.protocols=http/1.1
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000
nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=60 secs
nifi.web.request.ip.whitelist=$server b IP
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"
Server B:
# Site to Site properties
nifi.remote.input.host=
nifi.remote.input.secure=true
nifi.remote.input.socket.port=443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
nifi.web.https.host=
nifi.web.https.port=8443
nifi.web.https.network.interface.default=
nifi.web.https.application.protocols=http/1.1
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000
nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=600 secs
nifi.web.request.ip.whitelist=$Server A IP
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"
... View more
Labels:
- Labels:
-
Apache NiFi