Member since
01-10-2024
28
Posts
11
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
507 | 01-18-2024 11:10 PM |
01-25-2024
06:26 PM
1 Kudo
Dear @VidyaSargur, does anyone can help this issue? Thank you so much, much appreciated!!
... View more
01-24-2024
10:46 PM
Dear all, A bit background about this: I receive fresh data from a data vendor on a weekly basis and aim to either insert or update my existing database. If the lookup identifies matching values (such as company ID, fiscal year, etc.) in a record, we perform an update on specific fields (e.g., announcement date); otherwise, we insert a new record. My chain of lookups: My lookup processor setup: My data: Before lookup: After first lookup (Company ID) (It return me fs_ID): After second lookup (Fiscal_Year) (It return me another fs_ID): Note: fs_ID is retrived value from lookup tables However, I check my lookup table: It does not match the "Company_ID", it only match the "Fiscal_year", It seems like those lookup processor worked independently. I expected this output: myincomingdata.Company_ID = ent_fundamentals.Company_ID AND myincomingdata.Fiscal_year = ent_fundamentals.Fiscal_year..etc., (I can update this record base on those columns if they all match) Can anyone please help me this issue or is there any other way in doing lookup inNIFI?
... View more
Labels:
- Labels:
-
Apache NiFi
01-24-2024
10:33 PM
Thanks Samsal
... View more
01-24-2024
10:29 PM
Dear all, A bit background about this: I have newly incoming data from data vendor every week, and I would like to do either insert or update to my current database. If lookup matched some value (such as companyid, fiscal_year, ...etc,. )in a record then we update some value(annoucement date), otherwise insert. My lookup service setup: (Company ID) (Fiscal Year): My data is like this: Before lookup: After lookup company ID (it return fs_ID): After lookup fiscal year (it return fs_ID): And I go to Mysql database to check fs_ID = 351225894: IT DOES NOT MATCH THE COMPANY_ID (but it matched Fiscal_year)!!!!! It seems like each lookup processor working independently. My expected output it should does this: ent_fundamental.Company_ID = myincomingdata.company_ID AND ent_fundamental.Fiscal_year = myincomingdata.Fiscal_year etc., (then it will return me a fs_ID so that I can update this record) Or Does anyone have any good idea to do a multiple columns lookup in NIFI?
... View more
Labels:
- Labels:
-
Apache NiFi
01-21-2024
06:46 PM
Hi @SAMSAL Input from data provenance: Output:
... View more
01-18-2024
11:10 PM
1 Kudo
Problem solve after reading: https://community.cloudera.com/t5/Support-Questions/NiFi-Trigger-a-Processor-once-after-the-Queue-gets-empty-for/td-p/381801
... View more
01-18-2024
09:41 PM
1 Kudo
Dear all, I am using a Lookup processor, I cant find out why this is happening even there are not null!! 2024-01-19 13:33:29,195 ERROR [Timer-Driven Process Thread-148] o.a.n.processors.standard.LookupRecord LookupRecord[id=80fc149c-3740-1c93-1495-5d58f7e6e197] Failed to process StandardFlowFileRecord[uuid=902a8756-04e8-4837-adc7-d7ab711fc37e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1705642408394-2309, container=default, section=261], offset=0, length=185545],offset=0,name=e138b963-3dd9-4a3a-8f7d-5db9ca3a6758,size=185545] org.apache.avro.file.DataFileWriter$AppendWriteException: java.lang.NullPointerException: null value for (non-nullable) long at nifiRecord.fs_ID Please refer to my attachment: I cant find out any problem in here!!Can anyone help please, Thank you so much
... View more
Labels:
- Labels:
-
Apache NiFi
01-18-2024
09:29 PM
1 Kudo
Can anyone help this?
... View more
01-18-2024
08:20 PM
Anyone can help about this issue?
... View more
- « Previous
-
- 1
- 2
- Next »