About ashinde

ashinde · ‎01-19-2021

@Delio It would be bit complicated flow design as direct support for delete events is not available for GetHDFSEvents processor yet. But you can refer this solution, which can help you to move further. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash

ashinde · ‎01-18-2021

@Law serveraddress is hostname of host where nifi service is installed. E.g you have installed it on samplehost.com then it would be "http://samplehost.com:8080/nifi" .If you enable SSL for nifi later then the URL will change to "https://samplehost.com:<secure_port>/nifi". Also you should check that host and port are reachable from your browser to access NiFi UI. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here. -Akash

MattWho · ‎01-15-2021

MiNiFi offers CPP version that is well suited for Windows event log ingestion.

DarkStar · ‎12-10-2020

Hello Akash, Thanks for your feedback. Actually, its work fine if I change my query as below: SELECT DISTINCT T1.block_id AS block_id, T1.block_name AS block_name, T1.block_kp_begin AS block_kp_begin, T1.block_kp_end AS block_kp_end, T2.ap_track_id AS ap_track_id, T2.ap_kp_in_track AS ap_kp_in_track, T2.ap_physical_name AS ap_physical_name, T3.track_name AS track_name, T3.track_type AS track_type FROM amsterdam.amd_block_info T1 LEFT JOIN amsterdam.amd_ap_info T2 ON T1.track_id = T2.ap_track_id LEFT JOIN amsterdam.amd_track_info T3 ON T1.track_id = T3.track_id; Here all I needed to define my variable as Alias again then the problem fixed.

scoe74 · ‎12-10-2020

@ashinde What would I use to get the token passed to the next InvokeHTTP processor? Is the token just an attribute from the first InvokeHTTP processor?

kumar79 · ‎12-10-2020

Hi Akash, Sorry for the delay and thanks for the response. I want to avoid updating the flow incase if we have to run the flow for any old dates. If any such request comes I just want to run the flow through shell by just passing old date. Hope the requirement is clear to you. Thanks,

ashinde · ‎12-08-2020

@sarithe It is expected behaviour as Ranger has to pull resources from NiFi and as NiFi is secured it will look for authentication token from Ranger but as Ranger is un-secure it can not present the token and get 403 as response. You should enable SSL [or kerberise the cluster] for Ranger so Ranger can authenticate using SSL [or tgt]certificate. Meanwhile you can use https://community.cloudera.com/t5/Community-Articles/NiFi-Ranger-based-policy-descriptions/ta-p/246586 as reference to setup policies manually. -Akash

ashinde · ‎12-08-2020

Jaro, nipiapi Windows automated testing not attempted. Are there any particular things you want to achieve using nipiapi+Windows combination? -Akash

youngick · ‎12-07-2020

I recommend not using nifi and working with the console. Using nifi is not recommended because additional logs are generated. It is recommended to divide and compress the files that you want to move from the console into appropriate sizes and send them. For HDFS, you can use the distcp command. https://hadoop.apache.org/docs/current/hadoop-distcp/DistCp.html

Vapper · ‎11-22-2020

also just small clarification, I regenerated cert and tried 'CN=root, OU=ApacheNiFi' instead 'CN=admin, OU=ApacheNiFi' but still an issue, same errors in logs, please help

ashinde · ‎11-17-2020

@fgarcia Cloudera NiFi Kafka processors should work with all Kafka distributions be it HDP, CDH, CDP, Confluent or Apache Kafka. We generally recommend to use *Kafka_0_10 with Broker version as 0.10 and likewise. Hope it helps 🙂

justenji · ‎10-15-2020

Thanks for your help @ashinde ! I see I was looking at the wrong section of rest-api. OK I have to go on nifi rest-api section flow/history. I will have a look on the nifi.properties but because my report is running every night I suppose the purging doesn't matter. Second option you mentioned concerning when a flow was brought to registry doesn't interest... at the moment.

ashinde · ‎10-12-2020

@sunile_manjee I am not very familiar with AWS ELB but you can try to use HandleHttpRequest and HandleHttpResponse processors and check if it serves your use case

ashinde · ‎10-12-2020

@rstewart Counter is in-memory content for NiFi service. There is no direct way to get value of particular counter. But you can try below call to get a json response from NiFi about all counters. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/counter" -H "Authorization: Bearer $TOKEN" -H "Connection: close" -k Once you get the Json response you can parse it as per your use case

ashinde · ‎10-12-2020

@varun_rathinam You can use MergeContent processor if it fits for your use case. It is better way to handle small file issue in NiFi. Please refer below link for more details. https://community.cloudera.com/t5/Support-Questions/Merge-Content-for-small-content-issue/td-p/167660 BR, Akash

stevenmatison · ‎08-24-2020

@Koffi If you have a nifi flow created and tuned at a very large spec, and you downgrade that spec, you are going to have all kinds of problems like you are experiencing. You are going to need to go into the flow and reduce concurrency and min/max thread pool settings and completely re-tune the flow for the new environment since you reduce the ram and per core of the nodes. Another suggestion is that nifi 1.7 is very dated. You should consider an upgrade to nifi 1.12 and use at least 3 nodes. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven @ DFHZ

ashinde · ‎08-03-2020

@abhinav_joshi NiFi supports " /access/kerberos" for kerberos/spnego negotiation and "/access/token" for username/password to provide token to use with " Authorization: Bearer" in api request. So you can try to use below calls 1. TOKEN=$( curl -k "https://$HOSTNAME:9091/nifi-api/access/token" -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data 'username=xxx@REALM&password=secret' --compressed ) 2. curl -v -X GET "https://$HOSTNAME:9091/nifi-api/flow/status" -H "Authorization: Bearer $TOKEN" -H "Connection: close" and so on.

ashinde · ‎08-03-2020

@bsivalingam83 This error generally occurs when you copied the flow.xml.gz from another NiFi cluster and tries to load it in new NiFi setup. This happens because the "nifi.sensitive.props.key" from first NiFi setup is different from "nifi.sensitive.props.key" value from second NiFi setup. If flow.xml.gz is not copied and CFM is 1.0.x version then you may need to check https://docs.cloudera.com/cfm/1.0.1/upgrading-cfm/content/upgrading-cfm.html to resolve the issue

Paul Yang · ‎05-30-2020

Hello, @Bender Yes, I have got the link [2] from your reply Thank you very much! Paul

ashinde · ‎05-30-2020

@samarsimha You can use configure MongoDBControllerService > SSL Context Service and provide certificate details.So NiFi can use these details to communicate with MongoDB database. [Note: Make sure that the certificate details like keystore and truststore path should be same on all NiFI nodes] SSL Context Service you can use StandardSSLContextService which will allow you to communicate with both legacy and modern systems Thanks, Akash

ashinde · ‎09-12-2018

Hi Inam, Please find below link for reference : https://community.hortonworks.com/articles/178747/mqtt-with-apache-nifi.html (I have tested it and worked fine for me) Also as you are using psk-identity please check for "use_identity_as_username" value in your mosquitto.conf file. I am not very much sure about psk-identity but you can use mosquitto_passwd to set password for particular user and redirect the password to a file which file can be used for authentication to publish MQTT messages.

Online	Offline
Last Visited	‎03-03-2021 07:51 AM

Date Registered	‎07-30-2018 12:32 PM
Last Visited	‎03-03-2021 07:51 AM
Total Messages Posted	25
Total Kudos Received	3

Re: nipyapi on windows 10

Re: NiFi - REST-API - searching for information ab...

Re: using AWS ELB with ListenHTTP on nifi

Re: nifi mongo x509 connection

Re: Is cloudera not free to use

Re: CDP Public Cloud : Data Replication

Re: HDF on HDI - NiFi

Re: How to collect windows event log using Nifi

Re: Joining multiple tables in PostgreSQL using Ex...

Re: NiFi InvokeHTTP Problem

Re: NiFi: Passind date from command line

Re: Secured nifi with unsecured ranger issue

Re: nipyapi on windows 10

Re: Can NIFI storage be extended?

Re: NIFI docker ssl issue

Re: Cloudera DataFlow and Confluent Kafka

Re: NiFi - REST-API - searching for information ab...

Re: using AWS ELB with ListenHTTP on nifi

Re: Retrieve Value of Signal Counter in Wait/Notif...

Re: How to handle small file issue

Re: Nifi There is insufficient memory for the java...

Re: Starting process group from Linux for Secured ...

Re: Could not decrypt sensitive value - NiFi

Re: Is cloudera not free to use

Re: nifi mongo x509 connection

Re: Apache nifi : MQTT Bokerr consumer need help