About DianaTorres

Heeya8876 · ‎01-13-2024

Oh, I successfully managed to integrate and run NiFi 2.0 with Python on Windows using the method you suggested. Thank you so much!

achemeleu · ‎01-12-2024

Hello, sorry but due to the urgency I abandoned the migration of the components and finally reinstalled everything. I now have a hbase table export problem and submitted a new topic. If anyone here would take the time to help me find a solution, that would be very kind.

cardozogp · ‎01-09-2024

As I was already using the Hadoop Credential Provider, I found a solution that does not require decrypting the password as follows: PySpark code: # Spark session spark = SparkSession.builder \ .config("spark.yarn.keytab=/etc/security/keytabs/<APPLICATION_USER>.keytab") \ .appName('SPARK_TEST') \ .master("yarn") \ .getOrCreate() credential_provider_path = 'jceks://hdfs/<PATH>/<CREDENTIAL_FILE>.jceks' credential_name = 'PASSWORD.ALIAS' # Hadoop credential conf = spark.sparkContext._jsc.hadoopConfiguration() conf.set('hadoop.security.credential.provider.path',credential_provider_path) credential_raw = conf.getPassword(credential_name) for i in range(credential_raw.__len__()): password = password + str(credential_raw.__getitem__(i)) The important point above is the .config() line in SparkSession. You must enter the keytab to access the password. Otherwise you will get the encrypted value. I can't say that I'm very happy with being able to directly manipulate the password value in the code. I would like to delegate this to some component in a way that the programmer does not have direct access to the password value. Maybe what I'm looking for is some kind of authentication provider, but for now the solution above works for me.

DianaTorres · ‎01-08-2024

@elemenop Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

MattWho · ‎01-02-2024

@benimaru It is important to understand that NiFi does not replicate active FlowFiles (objects queued in connection between NiFi processor components) across multiple nodes. So in a five node NiFi cluster where you are load balancing FlowFiles across all nodes, each node has a unique subset of the full data received. This if node 1 goes down, the FlowFiles on node 1 will not be processed until node 1 is back up. 100% agree with @joseomjr that placing an external load balancer in front of the ListenUDP endpoint is the correct solution to ensure high availability of that endpoint across all your NiFi nodes. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt

RangaReddy · ‎01-02-2024

Thanks @Chandler641 Your issue is resolved after building the spark code properly. Note: We will not support Upstream Spark installation in our cloudera cluster because we are done lot of customisation in cloudera spark to support multiple integration components. Please let me know if you have further concerns on this issue.

Sadhana21 · ‎01-01-2024

Your input means a lot, and I appreciate your assistance @DianaTorres, @SAMSAL and @joseomjr. Thank you all for sharing your knowledge and insights!

joseomjr · ‎12-28-2023

Agree with @SAMSAL's approach and if you can provide a parameter or something in the header or request so your API returns a JSON response each time it'll make things a lot easier for you to parse and build the request for the next step in your flow.

DianaTorres · ‎12-27-2023

Here are some highlights from the month of November 120 new support questions 13 new community articles 474 new members Rank Community Article Author Components/ Labels #1 Metadata in Cloudera data warehouses Anshul Gupta @Anshul_Gupta Apache Atlas Apache Hive Apache Impala #2 How to optimize IMPALA/KUDU queries Shantanu Gope @ShantanuGope Apache Impala Apache Kudu Cloudera Data Platform (CDP) #3 How to connect to CDW (Impala) to return actively running query progress Ryan Cicak @RyanCicak Cloudera Data Warehouse (CDW) Cloudera Machine Learning (CML) #4 CML Runtime with Nvidia Libs and VSCode Editor Paul De Fusco @pauldefusco Cloudera Machine Learning (CML) #5 How to find Meta RegionServer detail via command line Akash Singh @aps Apache HBase We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions. See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page. @MattWho @steven-matison @vaishaakb @RangaReddy @Yuexin Zhang @SAMSAL @cotopaul @joseomjr @KarthiK439 @hegdemahendra Share your expertise and answer some of the below open questions. Also, be sure to bookmark the unanswered question page to find additional open questions. Unanswered Community Post Components/ Labels Data Visualization - Users & Groups Cloudera Data Platform (CDP) Cloudera Machine Learning (CML) Data Visualization Password secure way to use Spark JDBC Apache Spark PublishKafkaProcessor , request is not going to failure relation when kafka broker value is wrong Apache NiFi Nifi ConsumeKafka error while receiving the data Apache Kafka Apache NiFi What is the best approach for retrying a flow file once an hour for a 24 hour period? Apache NiFi

cjervis · ‎12-22-2023

Welcome to the community @skylarblu4650. As this post is a year old, I would suggest starting a new thread. That way you can provide details specific to your setup, version, how you encountered the issue and any steps you have tried already. It will provide others more details to assist you.

Online	Offline
Last Visited	‎06-08-2026 08:31 AM

Member Since	‎11-17-2021 08:08 AM
Last Visited	‎06-08-2026 08:31 AM
Posts	1,158
Kudos received	240

Cloudera Community

Re: How to change the company in the profile

Re: Keycloak SSO Hive REST Catalog

Re: Error connecting to NiFi Registry from NiFi UI...

Re: How to change my Account Email Address?

Re: Cannot erase old /opt/cloudera/parcels

Re: Nifi 2.0.0 M1 Installation error with python

Re: Ambari : Move master wizard : stucked in the ...

Re: Password secure way to use Spark JDBC

Re: Revert local changes problem in NiFi

Re: ListenUDP Fault tolerance

Re: Spark version is empty in CDH6.3.2

Re: Preparing nested JSON using SQL in NiFi

Re: Request for Support with Passing Request Body ...

December 2023 Community Highlights

Re: Generate missing credential failed with constr...