Member since
07-30-2018
25
Posts
3
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
172 | 10-14-2020 03:58 AM | |
108 | 10-12-2020 10:29 AM | |
165 | 05-30-2020 05:52 AM | |
719 | 05-27-2020 09:18 AM |
01-19-2021
02:23 AM
@Delio It would be bit complicated flow design as direct support for delete events is not available for GetHDFSEvents processor yet. But you can refer this solution, which can help you to move further. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash
... View more
01-18-2021
02:40 AM
@Law serveraddress is hostname of host where nifi service is installed. E.g you have installed it on samplehost.com then it would be "http://samplehost.com:8080/nifi" .If you enable SSL for nifi later then the URL will change to "https://samplehost.com:<secure_port>/nifi". Also you should check that host and port are reachable from your browser to access NiFi UI. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash
... View more
01-13-2021
03:42 AM
1 Kudo
@dzbeda You can use one of below approach. 1. Install minifi on windows machine and send logs from minifi to NiFi [Push] 2. You can use ConsumeWindowsEventLog to get event logs in NiFi [Pull]
... View more
01-13-2021
03:36 AM
@Delio What do you mean by "But what about file delete?" Do you want to delete the data at source HDFS as soon as you fetch it in NiFi?
... View more
12-08-2020
09:45 AM
@sarithe It is expected behaviour as Ranger has to pull resources from NiFi and as NiFi is secured it will look for authentication token from Ranger but as Ranger is un-secure it can not present the token and get 403 as response. You should enable SSL [or kerberise the cluster] for Ranger so Ranger can authenticate using SSL [or tgt]certificate. Meanwhile you can use https://community.cloudera.com/t5/Community-Articles/NiFi-Ranger-based-policy-descriptions/ta-p/246586 as reference to setup policies manually. -Akash
... View more
12-08-2020
09:27 AM
Jaro, nipiapi Windows automated testing not attempted. Are there any particular things you want to achieve using nipiapi+Windows combination? -Akash
... View more
12-08-2020
09:17 AM
@kumar79 I am not very sure with your complete use case, if you explain more would be better. Why you want to take date from shell/cron? you can use UpdateAttribute processor with custom property and use NiFi Regular Expression language to set date for you. E.g https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates The attribute later you can use in your flow to satisfy QL condition. Hope it helps. -Akash
... View more
12-08-2020
09:05 AM
@DarkStar Is the join query works fine on PostgreSQL CLI? If yes, then what is the jdbc driver you are using? is that jdbc version supports joins with duplicate fields? It looks like you have T1.track_id and T3.track_id which are conflicting. May be can you try to change the field name, test it on psql CLI and then test again in NiFi ? -Akash
... View more
12-08-2020
08:49 AM
@scoe74 If it is just token you have to take from SiteA and then using that token you can download data from SiteB then you can use 1st InvokeHTTP to take the token, add the token as flowfile attribute and then using that token you can use 2nd InvokeHTTP to download the data. Hope it helps you. -Akash
... View more
11-23-2020
12:21 AM
1 Kudo
@dzbeda You should extend the Content repo as it will store the content of your flowfiles. Please go through the best practices for setup https://community.cloudera.com/t5/Community-Articles/HDF-CFM-NIFI-Best-practices-for-setting-up-a-high/ta-p/244999 Also note that NiFi should not be used as backup for any failover mechanism as it is meant for processing and not for storage. I would recommend to store the data to may be some centralised location from where you can store and retrieve data as per use case and in case of failure as well you can have backup on that location. NiFi can be used to hold data for a day or 2 but if it is not processed then it will hold data at content repository and can fill the content repo storage.
... View more
11-20-2020
02:30 AM
Do you see any Warns, Errors for "CN=admin, OU=ApacheNiFi" user in nifi-user.log file when you login to the ui?
... View more
11-17-2020
08:16 AM
@Vapper Looks like there are issues with authorization, can you please check your authorizations.xml file and check if the user you have mentioned in INITIAL_ADMIN_IDENTITY is assigned to all expected policies?
... View more
11-17-2020
05:28 AM
@fgarcia Cloudera NiFi Kafka processors should work with all Kafka distributions be it HDP, CDH, CDP, Confluent or Apache Kafka. We generally recommend to use *Kafka_0_10 with Broker version as 0.10 and likewise. Hope it helps 🙂
... View more
10-14-2020
03:58 AM
@justenji The timestamp details are not exposed by NiFi on Flow but NiFi does maintain it in flow history. [Only if NiFi is secured with SSL] So you can try https://${hostname}:${NIFI_PORT}/ nifi-api/ flow/history/components/{componentId} to get the details. But it will provide all history of that component you have to parse it as per your need. Note that this info gets purged as per flow history parameters in nifi.properties so you will have these details not from beginning but beginning of the date [days/hours] mentioned in properties. If you are looking for flow timestamp when it is exported to NiFi-Registry then checkout for https://${hostname}:${NR_PORT}//nifi-registry-api/buckets/{bucketId}/flows/{flowId} Br, Akash
... View more
10-12-2020
10:29 AM
@sunile_manjee I am not very familiar with AWS ELB but you can try to use HandleHttpRequest and HandleHttpResponse processors and check if it serves your use case
... View more
10-12-2020
10:18 AM
@rstewart Counter is in-memory content for NiFi service. There is no direct way to get value of particular counter. But you can try below call to get a json response from NiFi about all counters. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/counter" -H "Authorization: Bearer $TOKEN" -H "Connection: close" -k Once you get the Json response you can parse it as per your use case
... View more
10-12-2020
09:57 AM
@justenji It is not possible yet to get all info in one NiFi API call as you desired but you can try below call to get the versioning info about particular process group curl -v -X GET "https://$HOSTNAME:9091/nifi-api/versions/process-groups/{id}" -H "Authorization: Bearer $TOKEN" -H "Connection: close" -k Br, Akash
... View more
10-12-2020
09:33 AM
@varun_rathinam You can use MergeContent processor if it fits for your use case. It is better way to handle small file issue in NiFi. Please refer below link for more details. https://community.cloudera.com/t5/Support-Questions/Merge-Content-for-small-content-issue/td-p/167660 BR, Akash
... View more
08-24-2020
08:56 AM
@Koffi Can you try to lower the initial memory and restart the service please?
... View more
08-03-2020
06:51 AM
@abhinav_joshi NiFi supports " /access/kerberos" for kerberos/spnego negotiation and "/access/token" for username/password to provide token to use with " Authorization: Bearer" in api request. So you can try to use below calls 1. TOKEN=$( curl -k "https://$HOSTNAME:9091/nifi-api/access/token" -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data 'username=xxx@REALM&password=secret' --compressed ) 2. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/flow/status" -H "Authorization: Bearer $TOKEN" -H "Connection: close" and so on.
... View more
08-03-2020
06:40 AM
@bsivalingam83 This error generally occurs when you copied the flow.xml.gz from another NiFi cluster and tries to load it in new NiFi setup. This happens because the "nifi.sensitive.props.key" from first NiFi setup is different from "nifi.sensitive.props.key" value from second NiFi setup. If flow.xml.gz is not copied and CFM is 1.0.x version then you may need to check https://docs.cloudera.com/cfm/1.0.1/upgrading-cfm/content/upgrading-cfm.html to resolve the issue
... View more
05-30-2020
05:52 AM
@samarsimha You can use configure MongoDBControllerService > SSL Context Service and provide certificate details.So NiFi can use these details to communicate with MongoDB database. [Note: Make sure that the certificate details like keystore and truststore path should be same on all NiFI nodes] SSL Context Service you can use StandardSSLContextService which will allow you to communicate with both legacy and modern systems Thanks, Akash
... View more
05-27-2020
09:18 AM
1 Kudo
@anand27krishn CDF packages are not supported on Windows OS but yes you can use single machine like you do for HDP 2.6.5 on virtualbox or vmware. Note that you have to install the VM and respective parcels manually there is no sandbox setup yet for CDF parcels. For more details for each service you can visit https://docs.cloudera.com/ and select for services like CFM, CSA,CSP and download the trail version parcels. Thanks, Akash
... View more
09-12-2018
07:00 AM
Hi Inam, Please find below link for reference : https://community.hortonworks.com/articles/178747/mqtt-with-apache-nifi.html (I have tested it and worked fine for me) Also as you are using psk-identity please check for "use_identity_as_username" value in your mosquitto.conf file. I am not very much sure about psk-identity but you can use mosquitto_passwd to set password for particular user and redirect the password to a file which file can be used for authentication to publish MQTT messages.
... View more