Fetch Provenance data using SiteToSiteProvenanceReportingTask

[#1]

I am running NiFi 1.24 in my local as a docker image in WSL as a standalone mode, trying to implement SiteToSiteProvenanceReportingTask.
I was referring to Extracting NiFi Provenance Data using SiteToSitePr... - Cloudera Community - 248469 to set it up. docker has volume mounted the conf directory so that i can change properties.
Every time I set the nifi.remote.input.secure to false and rerun the docker, it reset the property file to true. I have tried changing the other properties, they don't reset back to default.I have tried setting the env variable using -e SITE_TO_SITE_SECURE=false and -e NIFI_REMOTE_INPUT_SECURE=false, both have unfortunately not taken effect.docker command

docker run -d --name nifi24 -p 8443:8443 -e SITE_TO_SITE_SECURE=false -v ~/tools/nifi24_conf/conf:/opt/nifi/nifi-current/conf -v ~/tools/nifi24_conf/lib:/opt/nifi/nifi-current/lib -v ~/tools/nifi24_conf/nar_extensions:/opt/nifi/nifi-current/extensions apache/nifi:1.24.0

site to site properties

# Site to Site properties
nifi.remote.input.host=c30abd07b4ba
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10000
nifi.remote.input.http.enabled=false
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
nifi.web.http.host=
nifi.web.http.port=
nifi.web.http.network.interface.default=
############################################# 
nifi.web.https.host=c30abd07b4ba
nifi.web.https.port=8443
nifi.web.https.network.interface.default=
nifi.web.https.application.protocols=http/1.1
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000 nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=60 secs

nifi.web.request.ip.whitelist=
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"

I have tried to configure the Site To Site Provenance Reporting task 

 

Unfortunately, I am not able to get it working as i understand that it is not possible to configure Site to Site with security disabled while also running NiFi with HTTPS. Those settings go together. 

Please advise on how to get this working. Many Thanks

[#2]

Hi @MattWho , 

I tried a similar attempt with SiteToSiteBulletinReportingTask

it complains that there is no port. 

and in the site to site component state it has

 
 o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [CN=nifi-api-admin] 10.255.14.191 GET https://nifi-0.nifi-headless.doc-norc.svc.cluster.local:9443/nifi-api/site-to-site

o.a.n.r.SiteToSiteBulletinReportingTask SiteToSiteBulletinReportingTask[id=0adf334f-0190-1000-0000-00000230b61a] Error running task SiteToSiteBulletinReportingTask[id=0adf334f-0190-1000-0000-00000230b61a] due to org.apache.nifi.processor.exception.ProcessException: Failed to send Bulletins to destination due to IOException:Could not find Port with name 'prov' for remote NiFi instance"

o.a.n.remote.StandardRemoteProcessGroup Unable to connect to RemoteProcessGroup[https://nifi-0.nifi-headless.doc-norc.svc.cluster.local:9443/nifi] due to org.apache.http.NoHttpResponseException: nifi-0.nifi-headless.doc-norc.svc.cluster.local:9443 failed to respond

 please let me know if you can help here.

thank you 

[#3 · Accepted solution]

Hi @MattWho  , 

I have figured it out, 
I set the access policy recieve data via site-to-site and its has now started to work.

i used an api call to set the value referring to this.

Access Policies | CDP Private Cloud (cloudera.com)

thank you so much for your help.

TO Summarize, 

nifi.properties 

bash-4.4$ cat conf/nifi.properties | grep remote
nifi.remote.input.host=nifi-0.nifi-headless.namespace.svc.cluster.local
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs

in another pod
nifi.remote.input.host=nifi-1.nifi-headless.namespace.svc.cluster.local

nifi.web.https.host=nifi-0.nifi-headless.namespace.svc.cluster.local 
nifi.web.https.port=9443 

and respectively on another pod 

nifi.web.https.host=nifi-1.nifi-headless.namespace.svc.cluster.local 
nifi.web.https.port=9443

set access policies 

created reporting task 

url set is podname.svc/https port 

eg 

https://nifi-0.nifi-headless.doc-norc.svc.cluster.local:9443/nifi

set management controller service

created an input port and remote group to send data