Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

MiNiFi: not connecting to Nifi with certificate

Hi,

I want to connect my pi with nifi as described in https://de.hortonworks.com/tutorial/analyze-iot-weather-station-data-via-connected-data-architecture.... I secured nifi with certificate authentification and generated two truststores and keystores as @Matt Clarke suggested:

"I would execute the 'hostname -f" command on both the server where NiFi is installed and the server wheer MiNiFi is/will be installed.

Then use those full hostnames in the tls-tookit.sh script to create two keystores for use on these two instances

./tls-toolkit.sh standalone -n '<nifi.hostname>,<minifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> 

You will end up with a directory for each server being created that contains the needed TLS configuration info/files.

If you want to include SAN to your new certificates, you will need to create each one at a time:

./tls-toolkit.sh standalone -n '<nifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<nifi-ip-address>,<etc>'
./tls-toolkit.sh standalone -n '<minifi.hostname>' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<minifi-ip-address>,<etc>'

You should not need to any SAN entry if you generate your keystores using the actual hostnames assigned to your servers.

Also make sure that the following properties have been set to the appropariate hostnames as well in the nifi.properties file:

nifi.remote.input.host=nifi.web.https.host=

Once your NiFi server is using the new keystore and truststore files, you can use the nifi server hostname in the RPG.

Thanks,

Matt"

I copied the files in the conf folders of the NiFi and MiNiFi servers. The yml file of MiNiFi looks like:

MiNiFi Config Version: 3
Flow Controller:
  name: MiNiFi
  comment: ''
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
  implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository:
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: './conf/keystore.jks'
  keystore type: 'jks'
  keystore password: 'xxxxxx'
  key password: 'xxxxxx'
  truststore: './conf/truststore.jks'
  truststore type: 'jks'
  truststore password: 'xxxxx'
  ssl protocol: TLS
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: ab316f87-2c97-3fe4-0000-000000000000
  name: ExecuteProcess
  class: org.apache.nifi.processors.standard.ExecuteProcess
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    Argument Delimiter: ' '
    Batch Duration: 5 sec
    Command: python
    Command Arguments: /home/pi/Documents/sensehat.py
    Redirect Error Stream: 'false'
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: 7aba8512-3f8c-3dd9-0000-000000000000
  name: ExecuteProcess/success/1e4831a5-0161-1000-53c4-30f108c20272
  source id: ab316f87-2c97-3fe4-0000-000000000000
  source relationship names:
  - success
  destination id: 1e4831a5-0161-1000-53c4-30f108c20272
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups:
- id: db36c66b-4e7b-316b-0000-000000000000
  name: ''
  url: https://<nifi hostname>:8011/nifi
  comment: ''
  timeout: 30 sec
  yield period: 10 sec
  transport protocol: RAW
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: 1e4831a5-0161-1000-53c4-30f108c20272
    name: MiNiFi
    comment: ''
    max concurrent tasks: 1
    use compression: false
  Output Ports: []
NiFi Properties Overrides: {}

But I get the following error:

2018-01-23 18:30:46,323 ERROR [Timer-Driven Process Thread-3] o.a.n.c.t.ContinuallyRunConnectableTask RemoteGroupPort[name=MiNiFi,targets=https://gcvhdp01.ad.german-mgmt.de:8011/nifi] failed to process session due to java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
2018-01-23 18:30:46,325 ERROR [Timer-Driven Process Thread-3] o.a.n.c.t.ContinuallyRunConnectableTask 
java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
at org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:257)
at org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:81)
at org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:40)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
at org.apache.nifi.remote.client.SiteInfoProvider.refreshRemoteInfo(SiteInfoProvider.java:69)
at org.apache.nifi.remote.client.SiteInfoProvider.getActiveClusterUrl(SiteInfoProvider.java:247)
at org.apache.nifi.remote.client.socket.EndpointConnectionPool.getEndpointConnection(EndpointConnectionPool.java:163)
at org.apache.nifi.remote.client.socket.SocketClient.createTransaction(SocketClient.java:127)
at org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:238)
at org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:250)
... 10 common frames omitted

Where do I need to change something?

Kind regards

Jan

6 REPLIES 6

Master Guru
@Jan

- The configure "transport protocol" is set to RAW. Was this method enabled on the target NiFi?

- Is port 8011 the configured port used for the NiFi URL on target NiFi?

- Are the permissions and ownership on the keystore and truststore files on your MiNiFi set correctly?

- Do you see any authentication or authorization log lines coming from this MiNiFi host in the nifi-user.log on the target NiFi?

- Does the content of both truststore.jks files (MiNiFi and NiFi) contain identical entries?

Thanks,

New Contributor

Hello Matt,

I have same issue connecting minifi to the Nifi using ssl, Checked all your recommendation, but couldn't identify the issue.

- The configure "transport protocol" is set to RAW. Was this method enabled on the target NiFi? : it is HTTP

- Is port 8011 the configured port used for the NiFi URL on target NiFi? : yes ( port 9091)

- Are the permissions and ownership on the keystore and truststore files on your MiNiFi set correctly? ( Yes )

- Do you see any authentication or authorization log lines coming from this MiNiFi host in the nifi-user.log on the target NiFi? ( No )

- Does the content of both truststore.jks files (MiNiFi and NiFi) contain identical entries? ( Yes )

Error message : ( Minifi Log)

2018-02-21 11:23:23,306 ERROR [Remote Process Group 097540fc-19d1-355c-b129-71ce900782e7 Thread-1] org.apache.nifi.engine.FlowEngine A flow controller task executi on stopped abnormally java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/a pache/http/impl/client/HttpClientBuilder; at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpCli entBuilder; at org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361) at org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:346) at org.apache.nifi.remote.StandardRemoteProcessGroup$InitializationTask.run(StandardRemoteProcessGroup.java:1177) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ... 2 common frames omitted

Thanks,

Saravana

Master Guru
@Jan @Saravanakumar Nagarajan

Did you modify or add any non standard jars or nars to your NiFi installation?

You may have something unexpected in your class path

Thanks,

Matt

Explorer

@Jan @Saravanakumar Nagarajan

Did you solved this issue?

New Contributor

Hi Matt / Jan,

It was failing with in minifi 0.4, so I tried Minifi 0.3 with minifi toolkit 0.3.0 and it worked for me.

Thanks,

Saravana

New Contributor

@Jan @Saravanakumar Nagarajan

I did many tests and can confirm that ( on Raspberry Pi - running raspbian ) minifi version 0.3 works but minifi version 0.4 did not. I also updated my java version from "1.8.0_65" to "1.8.0_171" without success. Hope this helps someone else.

Regards,

Ricardo

,

@Jan @Saravanakumar Nagarajan

I did many tests and can confirm that ( on Raspberry Pi - running raspbian ) minifi version 0.3 works but minifi version 0.4 did not. I also updated my java version from "1.8.0_65" to "1.8.0_171" without success. Hope this helps someone else.

Regards,

Ricardo