Member since
07-30-2018
25
Posts
3
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
59 | 12-08-2020 09:27 AM | |
208 | 10-14-2020 03:58 AM | |
130 | 10-12-2020 10:29 AM | |
181 | 05-30-2020 05:52 AM | |
924 | 05-27-2020 09:18 AM |
01-19-2021
02:23 AM
@Delio It would be bit complicated flow design as direct support for delete events is not available for GetHDFSEvents processor yet. But you can refer this solution, which can help you to move further. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash
... View more
01-18-2021
02:40 AM
@Law serveraddress is hostname of host where nifi service is installed. E.g you have installed it on samplehost.com then it would be "http://samplehost.com:8080/nifi" .If you enable SSL for nifi later then the URL will change to "https://samplehost.com:<secure_port>/nifi". Also you should check that host and port are reachable from your browser to access NiFi UI. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash
... View more
01-15-2021
07:11 AM
MiNiFi offers CPP version that is well suited for Windows event log ingestion.
... View more
12-10-2020
02:56 PM
Hello Akash, Thanks for your feedback. Actually, its work fine if I change my query as below: SELECT
DISTINCT T1.block_id AS block_id,
T1.block_name AS block_name,
T1.block_kp_begin AS block_kp_begin,
T1.block_kp_end AS block_kp_end,
T2.ap_track_id AS ap_track_id,
T2.ap_kp_in_track AS ap_kp_in_track,
T2.ap_physical_name AS ap_physical_name,
T3.track_name AS track_name,
T3.track_type AS track_type
FROM amsterdam.amd_block_info T1
LEFT JOIN amsterdam.amd_ap_info T2 ON T1.track_id = T2.ap_track_id
LEFT JOIN amsterdam.amd_track_info T3 ON T1.track_id = T3.track_id; Here all I needed to define my variable as Alias again then the problem fixed.
... View more
12-10-2020
06:46 AM
@ashinde What would I use to get the token passed to the next InvokeHTTP processor? Is the token just an attribute from the first InvokeHTTP processor?
... View more
12-10-2020
12:57 AM
Hi Akash, Sorry for the delay and thanks for the response. I want to avoid updating the flow incase if we have to run the flow for any old dates. If any such request comes I just want to run the flow through shell by just passing old date. Hope the requirement is clear to you. Thanks,
... View more
12-08-2020
09:45 AM
@sarithe It is expected behaviour as Ranger has to pull resources from NiFi and as NiFi is secured it will look for authentication token from Ranger but as Ranger is un-secure it can not present the token and get 403 as response. You should enable SSL [or kerberise the cluster] for Ranger so Ranger can authenticate using SSL [or tgt]certificate. Meanwhile you can use https://community.cloudera.com/t5/Community-Articles/NiFi-Ranger-based-policy-descriptions/ta-p/246586 as reference to setup policies manually. -Akash
... View more
12-08-2020
09:27 AM
Jaro, nipiapi Windows automated testing not attempted. Are there any particular things you want to achieve using nipiapi+Windows combination? -Akash
... View more
12-07-2020
11:28 AM
I recommend not using nifi and working with the console. Using nifi is not recommended because additional logs are generated. It is recommended to divide and compress the files that you want to move from the console into appropriate sizes and send them. For HDFS, you can use the distcp command. https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html
... View more
11-22-2020
04:55 AM
also just small clarification, I regenerated cert and tried 'CN=root, OU=ApacheNiFi' instead 'CN=admin, OU=ApacheNiFi' but still an issue, same errors in logs, please help
... View more
11-17-2020
05:28 AM
@fgarcia Cloudera NiFi Kafka processors should work with all Kafka distributions be it HDP, CDH, CDP, Confluent or Apache Kafka. We generally recommend to use *Kafka_0_10 with Broker version as 0.10 and likewise. Hope it helps 🙂
... View more
10-15-2020
04:20 AM
Thanks for your help @ashinde ! I see I was looking at the wrong section of rest-api. OK I have to go on nifi rest-api section flow/history. I will have a look on the nifi.properties but because my report is running every night I suppose the purging doesn't matter. Second option you mentioned concerning when a flow was brought to registry doesn't interest... at the moment.
... View more
10-12-2020
10:29 AM
@sunile_manjee I am not very familiar with AWS ELB but you can try to use HandleHttpRequest and HandleHttpResponse processors and check if it serves your use case
... View more
10-12-2020
10:18 AM
@rstewart Counter is in-memory content for NiFi service. There is no direct way to get value of particular counter. But you can try below call to get a json response from NiFi about all counters. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/counter" -H "Authorization: Bearer $TOKEN" -H "Connection: close" -k Once you get the Json response you can parse it as per your use case
... View more
10-12-2020
09:33 AM
@varun_rathinam You can use MergeContent processor if it fits for your use case. It is better way to handle small file issue in NiFi. Please refer below link for more details. https://community.cloudera.com/t5/Support-Questions/Merge-Content-for-small-content-issue/td-p/167660 BR, Akash
... View more
08-24-2020
09:28 AM
@Koffi If you have a nifi flow created and tuned at a very large spec, and you downgrade that spec, you are going to have all kinds of problems like you are experiencing. You are going to need to go into the flow and reduce concurrency and min/max thread pool settings and completely re-tune the flow for the new environment since you reduce the ram and per core of the nodes. Another suggestion is that nifi 1.7 is very dated. You should consider an upgrade to nifi 1.12 and use at least 3 nodes. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven @ DFHZ
... View more
08-03-2020
06:51 AM
@abhinav_joshi NiFi supports " /access/kerberos" for kerberos/spnego negotiation and "/access/token" for username/password to provide token to use with " Authorization: Bearer" in api request. So you can try to use below calls 1. TOKEN=$( curl -k "https://$HOSTNAME:9091/nifi-api/access/token" -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data 'username=xxx@REALM&password=secret' --compressed ) 2. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/flow/status" -H "Authorization: Bearer $TOKEN" -H "Connection: close" and so on.
... View more
08-03-2020
06:40 AM
@bsivalingam83 This error generally occurs when you copied the flow.xml.gz from another NiFi cluster and tries to load it in new NiFi setup. This happens because the "nifi.sensitive.props.key" from first NiFi setup is different from "nifi.sensitive.props.key" value from second NiFi setup. If flow.xml.gz is not copied and CFM is 1.0.x version then you may need to check https://docs.cloudera.com/cfm/1.0.1/upgrading-cfm/content/upgrading-cfm.html to resolve the issue
... View more
05-30-2020
07:08 PM
1 Kudo
Hello, @Bender Yes, I have got the link [2] from your reply Thank you very much! Paul
... View more
05-30-2020
05:52 AM
@samarsimha You can use configure MongoDBControllerService > SSL Context Service and provide certificate details.So NiFi can use these details to communicate with MongoDB database. [Note: Make sure that the certificate details like keystore and truststore path should be same on all NiFI nodes] SSL Context Service you can use StandardSSLContextService which will allow you to communicate with both legacy and modern systems Thanks, Akash
... View more
09-12-2018
07:00 AM
Hi Inam, Please find below link for reference : https://community.hortonworks.com/articles/178747/mqtt-with-apache-nifi.html (I have tested it and worked fine for me) Also as you are using psk-identity please check for "use_identity_as_username" value in your mosquitto.conf file. I am not very much sure about psk-identity but you can use mosquitto_passwd to set password for particular user and redirect the password to a file which file can be used for authentication to publish MQTT messages.
... View more