Member since
02-01-2022
285
Posts
103
Kudos Received
60
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1178 | 05-15-2025 05:45 AM | |
| 5109 | 06-12-2024 06:43 AM | |
| 8112 | 04-12-2024 06:05 AM | |
| 5995 | 12-07-2023 04:50 AM | |
| 3294 | 12-05-2023 06:22 AM |
06-10-2022
12:17 AM
2 Kudos
Follow the steps to install, lack _psycopg psycopg2 package. Py files The solution 1:wget https://github.com/psycopg/psycopg2/archive/refs/tags/2_8_5.tar.gz 2:tar -xf 2_8_5.tar.gz 3:python setup.py build python setup.py install 4:cp psycopg2-2.8.5-py3.7-linux-x86_64.egg/psycopg2 /usr/share/python3
... View more
06-01-2022
10:59 AM
1 Kudo
@steven-matison thank you so much.
... View more
05-24-2022
06:02 AM
1 Kudo
@FediMannoubi Below is a basic approach to solve. Assuming both postgres tables are populated with rows per your example, your nifi flow would need to get the CSV (various ways to do that), once the contents of the csv are in a flowfile (i use GenerateFlowFile processor), you can use a RecordReader based processor to read the csv. This will allow you to write SQL against the flowfile with QueryRecord to get a single value. For example: SELECT city_name FROM FLOWFILE Next, in your flow you will need to get the city_name value into an attribute, i use EvaluateJsonPath. After that a ExecuteSQL processor and associated DBCP Connection pool to postgres. Then in ExecuteSQL your query is SELECT city_id FROM CITY WHERE city_name=${city_name} At the end of this flow you will have the city_name from csv, and city_id from postgres. You can now combine or use the further downstream to suit your needs. INSERT is done similarly, once you have the data in flowfiles, or attributes, using the same ExecuteSQL you write an insert instead. My test flow looks like this, but forgive the end, as I did not actually have a postgres database setup. You can find this sample flow [here]. I hope this gets you pointed in the right direction for reading csv and querying data from database.
... View more
05-17-2022
02:43 PM
@joshtheflame
I just wanted to provide a bit more context. The partial page shot you've included above appears to show Cloudera Manager running against a CDH 6.1.0 cluster. CDH 6.1.0 was released in December of 2018. As you no doubt are aware, that was quite a while ago, especially in terms of "internet time". Hopefully you are aware that CDH 6.1.x has reached its End of Support (EoS). You can find the most recent official reminder of the previous announcement that Cloudera Enterprise 6.x reached End of Support (EoS) in 2021 here:
March 2021 Customer Advisory - 2: End of Support for Cloudera Products (CDH/CM 6.x & HDP 3.x).
Cloudera's lifecycle support policies are documented here:
https://www.cloudera.com/legal/policies/support-lifecycle-policy.html
My understanding is that organizations with a valid Cloudera subscription for legacy products such as CDH would have been sent this announcement directly.
If that screenshot represents what your bank is running in production, I would recommend that you reach out to your Cloudera Account team and discuss your upgrade options as soon as possible. You are going to have to upgrade in order to take advantage of any of the offerings mentioned in @steven-matison 's reply earlier.
... View more
05-17-2022
12:35 PM
Nice one sir!
... View more
04-26-2022
06:06 AM
@Tokolosk I would recommend that you provide java's cacerts as the keystore/truststore with the default password (changeit). This will work for most public certs. If you have a custom cert at the SFTP end you will need to import that cert and create your own keystore/truststore. I always try cacerts first.
... View more
04-08-2022
08:10 AM
1 Kudo
Hi All, I was able to get my script tested on my 10 nodes DEV cluster. Below are the results: 1. All HDP core services started / stopped okay 2. None of Hive Service Interactive service started and hence, Hive service was not marked as STARTED though HMS and HS2 were started okay 3. None of the Spark2_THRIFTSERVER was started Any one can share some thoughts on points 2 and 3? Thanks snm1523
... View more
03-24-2022
03:05 AM
1 Kudo
@Nifi_Al , Here's a way to do that: The RouteOnAttribute processor checks the file name and if it's a ZIP file it sends it for unpacking. Otherwise sends the non-zip file downstream. Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-16-2022
12:58 PM
@steven-matison Thanks for the response i have setup CDP is working fine ,since the image which i have installed is 6 year old image and it is missing KUDU/Kafka/Nifi and so...on i thought of using the HDP sandbox since it is having the latest necessary applications compared to the once i have tested the image https://hub.docker.com/r/cloudera/quickstart/ which is a 6 years old image and i have installed successfully ,however during the installation i did noticed encountered a warning but the installation went fine and i have tested some examples working fine as expected in MAC M1 , So my confusion over here is how come Horton works sandbox images is not working in MAC m1 in docker ,what i also noticed is the script which i mentioned is some thing different when compared agains the 6 year old images which i have tested. anyways things are getting complicated ,to keep it short i will stick to the 6 year old images ,sicne it is open source (free) , i don't want to touch anything which is limited trail/ asking me to provide card details (cloud(aws/gcp/cloudera cloud version).....
... View more
03-11-2022
05:24 AM
1 Kudo
@kumsath Yes, when i saw your post I started digging around too. The issue here is number manipulation and type casting a "number" versus a "string". That easy gets complicated in attributes and with nifi expression language functions. The solution i referenced would take the "number" and make a number in which the visual representation should then be correct. Its worth a try since that execute script should be very easy to test. The alternative is to do your "math" upstream or in an execute script processor not in updateattribute.
... View more
- « Previous
- Next »