Member since
11-14-2019
11
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5968 | 12-19-2019 01:40 AM |
12-19-2019
01:40 AM
Solution: docker run --name nifi-ssl \
-v /home/ec2-user/project/nifi-standalone/certs:/opt/certs \
-v /home/ec2-user/project/nifi-standalone/conf:/opt/conf \
-p 8080:8443 \
-e NIFI_WEB_PROXY_HOST=11.111.111.111:8080 \
-e AUTH=tls \
-e KEYSTORE_PATH=/opt/certs/keystore.jks \
-e KEYSTORE_TYPE=JKS \
-e KEYSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_PATH=/opt/certs/truststore.jks \
-e TRUSTSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_TYPE=JKS \
-e INITIAL_ADMIN_IDENTITY='CN=admin, OU=nifi' \
-d \
apache/nifi:latest
... View more
12-18-2019
12:42 PM
I have an EC2 instance running with the IPv4 Public IP 11.111.111.111.
The first thing I tried was to get NiFi running.
docker run --name nifi-standalone\
-p 8080:8080 \
-d \
apache/nifi:latest
At 11.111.111.111.111:8080/nifi/ I can reach NiFi. Great.
Now I wanted to setup a secure NiFi instance.
Step 1: Create certificates with the toolkit.
./bin/tls-toolkit.sh standalone -n '11.111.111.111' -C 'CN=admin,OU=nifi' -B SuperSecretPassword -o './standalone'
Step 2: Move keystore.jks and truststore.jks into a specific folder (here /home/ec2-user/project/nifi-standalone/certs).
Scenario 1: set NIFI_WEB_HTTP_HOST
I run the following docker command.
docker run --name nifi-ssl \
-v /home/ec2-user/project/nifi-standalone/certs:/opt/certs \
-v /home/ec2-user/project/nifi-standalone/conf:/opt/conf \
-p 8443:8443 \
-e NIFI_WEB_HTTPS_HOST=11.111.111.111 \
-e AUTH=tls \
-e KEYSTORE_PATH=/opt/certs/keystore.jks \
-e KEYSTORE_TYPE=JKS \
-e KEYSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_PATH=/opt/certs/truststore.jks \
-e TRUSTSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_TYPE=JKS \
-e INITIAL_ADMIN_IDENTITY='CN=admin,OU=nifi' \
-d \
apache/nifi:latest
Note: I read the passwords for the keystore and truststore directly from the nifi.properties file that the toolkit creates.
Unfortunately the container shuts down because of
2019-12-18 20:18:29,400 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. java.io.IOException: Failed to bind to.../11.111.111.111:8443
Scenario 2: unset NIFI_WEB_HTTP_HOST
I run the following docker command.
docker run --name nifi-ssl \
-v /home/ec2-user/project/nifi-standalone/certs:/opt/certs \
-v /home/ec2-user/project/nifi-standalone/conf:/opt/conf \
-p 8443:8443 \
-e AUTH=tls \
-e KEYSTORE_PATH=/opt/certs/keystore.jks \
-e KEYSTORE_TYPE=JKS \
-e KEYSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_PATH=/opt/certs/truststore.jks \
-e TRUSTSTORE_PASSWORD=passwordFROMnifi.properties \
-e TRUSTSTORE_TYPE=JKS \
-e INITIAL_ADMIN_IDENTITY='CN=admin,OU=nifi' \
-d \
apache/nifi:latest
Now the jetty server starts:
2019-12-18 20:34:23,104 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2019-12-18 20:34:23,104 INFO [main] org.apache.nifi.web.server.JettyServer https://d592fc9f7974:8443/nifi
As I saw the host d592fc9f7974 looks unexpected. https://d592fc9f7974:8443/nifi is unavailable. Next attempt https://11.111.111.111:8443/nifi.
Result:
System Error
The request contained an invalid host header [11.111.111.111:8443] in the request [/nifi]. Check for request manipulation or third-party intercept.
Valid host headers are [empty] or:
127.0.0.1
127.0.0.1:8443
localhost
localhost:8443
[::1]
[::1]:8443
d592fc9f7974
d592fc9f7974:8443
172.17.0.2
172.17.0.2:8443
... View more
Labels:
- Labels:
-
Apache NiFi
-
Docker
12-05-2019
05:34 AM
You have to use the PyRFC and SAP NW RFC SDK. Let us assume I like to call the function module STFC_CONNECTION: from pyrfc import Connection
conn = Connection(ashost='123', sysnr='00', client='100', user='user', passwd='secret')
conn.call('STFC_CONNECTION', REQUTEXT=u'Hello SAP!') I have created an Anaconda environment to have the libraries I need. Currently I am still waiting for the SAP NW RFC SDK. Would it work like this? Flow: Flow ExecuteStreamCommand
... View more
12-02-2019
08:00 AM
I am faced with the following task. Call a remote-enabled module (ABAP) via RFC using Nifi, pass the required parameters and write the results away. My idea is to do the whole thing via pyrfc https://sap.github.io/PyRFC/intro.html . How would you do that? Any experience?
... View more
Labels:
- Labels:
-
Apache NiFi
11-28-2019
04:41 AM
Hi wcdata, I took your advice and redesigned my workflow but even with 10 records the PutDatabaseRecord loads and loads. I suspect that the Translate Field Names setting is to blame. Because my source columns are capitalized and my target columns are small. I don't want to define a schema here.
... View more
11-27-2019
08:46 AM
The following scenario: I extract (as a one-time load) all data from a very large table (40 Mio records) in 100k partitions. These are then converted to JSON (ConvertRecord) and finally converted with ConvertJSONtoSQL into one row SQL statement each. Strangely enough, the penultimate step (PutSQL) runs very slowly. I would like to insert the data into my Redshift cluster. What could be the reason? And how can I speed it up? Details to the PutSQL processor:
... View more
Labels:
- Labels:
-
Apache NiFi
11-14-2019
08:31 AM
Where can I find the exact differences between the Hortonworks and the raw Nifi version? Would like to understand why I should choose HDD instead of Nifi in its pure form.
... View more
11-14-2019
06:51 AM
Sorry but I don't get your point "8 versions behind". I still use the latest Nifi version 1.10. I can't remember that didn't work in the previous version 1.9.2.
... View more
11-14-2019
02:13 AM
I'm trying to change an existing variable. If the variable was defined in a process group and this process group is then copied, I get the following message when I try to change the variable.
"Unable to complete variable update request: Failed to update Variable Registry because failed while performing step: Applying updates to Variable Registry"
NiFi 1.10
... View more
Labels:
- Labels:
-
Apache NiFi