Member since
01-18-2017
58
Posts
5
Kudos Received
0
Solutions
07-19-2024
01:33 PM
@Bern As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.
... View more
02-09-2021
10:41 AM
Here is a cool NiFi websocket app https://www.datainmotion.dev/2020/12/ingesting-websocket-data-for-live-stock.html Hosting web apps in NiFi https://www.datainmotion.dev/2020/11/flank-smart-weather-websocket.html https://www.datainmotion.dev/2020/12/ingesting-websocket-data-for-live-stock.html
... View more
07-17-2018
10:49 PM
@Andy Liang ExecuteSql processor will result output flowfile with embedded avro schema in it. So in your PutParquet processor configure the Record Reader as Avro Reader and use the embedded avro schema as Schema access strategy By using Embedded avro schema you don't have to set up any avro schema registry and this processor will work dynamically based on the embedded avro schema.
... View more
07-23-2018
12:49 PM
1 Kudo
You can use a JOIN clause in the select statement, but it will only work for a single RDBMS. You may find you can join two tables from two databases/schemas in the same RDBMS (if that system lets you), but you can't currently join two tables from totally separate database systems. You could investigate Presto, it allows for joining of tables across multiple systems, and you could have a single connection to it from NiFi in ExecuteSQL. That way it will look like a single RDBMS to NiFi, but Presto can be configured to do the cross-DB join.
... View more
05-15-2017
05:48 PM
@Wynner Thanks for your help, I had updated it to java.arg.2=-Xms512m java.arg.3=-Xmx8g
... View more
03-06-2017
06:57 PM
2 Kudos
The TLS/SSL properties in nifi.properties are only used for setting up authentication to NiFi. When talking to other systems, NiFi lets you specify a SSLContextService because a lot of times the certificate and truststore you want to use for communicating with other systems is going to be different. If they are the same then you can definitely create a StandardSSLContextService and point it at the same keystore and truststore that nifi.properties is pointing at.
... View more
02-24-2017
10:32 PM
update: I finally figured out how to setup IBM MQ for Nifi Here is my steps to setup IBM MQ for the Nifi connection 1) Create queue manager myMQManager using MQ Explorer 2) create Channel and queue using the following commands runmqsc myMQManager def chl(myMQChannel) chltype(svrconn) replace def chl(myMQChannel) chltype(clntconn) conname('localhost(1420)') QMNAME('myMQManager') replace def ql(myMQqueue) 2) Grant necessary authorization for a user using MQ Explorer 2.1 Queue Manager Authorities Setting right click your queue manager-->Object Authorities-->Add Role Based Authorities -->give the user access permission 2.2 Queue Authorities Setting right click your queue-->Object Authorities-->Manage Authority Records-->Specific Profiles--> your queue--> Accumulated authorities-->setup your user Access Permission for the queue 3) After the configuration and setup your Nifi ConsumeJMS processor, you should be able connect to IBM MQ
... View more
02-24-2017
10:29 PM
@ozhurakousky Here is the steps to setup IBM MQ for the Nifi connection 1) Create queue manager myMQManager using MQ Explorer 2) create Channel and queue using the following commands
runmqsc myMQManager def chl(myMQChannel) chltype(svrconn) replace def chl(myMQChannel) chltype(clntconn) conname('localhost(1420)') QMNAME('myMQManager') replace def ql(myMQqueue) 2) Grant necessary authorization for a user using MQ Explorer 2.1 Queue Manager Authorities Setting right click your queue manager-->Object Authorities-->Add Role Based Authorities -->give the user access permission 2.2 Queue Authorities Setting right click your queue-->Object Authorities-->Manage Authority Records-->Specific Profiles--> your queue--> Accumulated authorities-->setup your user Access Permission for the queue 3) After the configuration and setup your Nifi ConsumeJMS processor, you should be able connect to IBM MQ
... View more
01-02-2018
05:57 PM
Thanks @Manchun Kumar , but i am unable to find post for this in the link you provided. could you please share the particualr link for ssl/TLS in Nifi-1.1.1 or above
... View more
01-15-2019
11:36 PM
@Pierre Villard @Andy Liang could you tell how to do the second approach you've suggest? "As a final side note: if you need to perform a login/password authentication like Basic Authentication (with HTTP headers), then you don't need all of this, and you just need to check the attributes sent by the user in the request with a RouteOnAttribute processor and using the expression language."
... View more