Member since
07-28-2022
25
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1981 | 02-03-2023 01:57 AM |
01-04-2023
02:49 AM
Hi Team, Am getting error on JoinEnrichment and don't know why it is coming, Please suggest. My query in JoinEnrichment : SELECT original.*, enrichment.* FROM original FULL JOIN enrichment ON original.name = enrichment.name Error receiving JoinEnrichment[id=a17b1f06-f80c-3641-945f-c2ff331f8028] Failed to join 'original' FlowFile FlowFile[filename=cmdbci-mtaas] and 'enrichment' FlowFile FlowFile[filename=cmdbci-mtaas]; routing to failure: java.sql.SQLException: Error while preparing statement [SELECT original., enrichment. FROM original FULL JOIN enrichment ON original.name = enrichment.name] Caused by: org.apache.calcite.runtime.CalciteContextException: From line 4, column 4 to line 4, column 34: Cannot apply '=' to arguments of type '<JAVATYPE(CLASS JAVA.LANG.OBJECT)> = <JAVATYPE(CLASS JAVA.LANG.STRING)>'. Supported form(s): '<COMPARABLE_TYPE> = <COMPARABLE_TYPE>' Caused by: : Corg.apache.calcite.sql.validate.SqlValidatorExceptionannot apply '=' to arguments of type '<JAVATYPE(CLASS JAVA.LANG.OBJECT)> = <JAVATYPE(CLASS JAVA.LANG.STRING)>'. Supported form(s): '<COMPARABLE_TYPE> = <COMPARABLE_TYPE>' Sample input from Original Location,Environment,ip_address,category,name,dv_sys_updated_on "",,,Hardware,Ndiggan,2022-12-17 22:37:28 "",,,Hardware,class,2022-12-31 22:37:38 "",,,,Vlan2,2022-12-27 02:17:13 Sample input from Enrichment Location,Environment,ip_address,category,name,dv_sys_updated_on "",,,Hardware,vpna,2022-12-17 22:36:02 "",,,Hardware,dlcccno,2022-12-17 22:37:04 "",,,Hardware,Ndiggan,2022-12-17 22:37:28
... View more
Labels:
- Labels:
-
Apache NiFi
12-21-2022
07:53 AM
@MattWho Thanks for the response, Below is the inline comments and my scenario. 1. Is this a one time execution flow? -Not 2. if not, how do you differentiate between different complete batches (when does new one merge bundle end and another begin?)? - I'm calling the same rest-api with the record limit of 1000, and let suppose my attribute "X-Total-Count" is 5600 (record), then am going to call the rest-api 6 times. (1000*6) to capture the complete record. so till reach to 5600 records i want "Wait" processor before merge it all. 3. Are all 1000 records from each rest-api call going into a single NiFi FlowFile or 1 FlowFile per record? -One FlowFile per record, as after "invokeHTTP" am using "splitjson". 4. Is there some correlation identifier as a rest of rest-api call that identifies all 1000 Record batch pulls as part of same complete bundle? -i'm using offset for this. What I have : rest-api with limit set to 1000, total number of records is not always same, capturing from "X-Total-Count". What am looking for : Before merge the whole data, i want "Wait" processor for waiting the completion of API (api could be run 2 times or 15times) after getting the whole data that "FlowFile" (fragment.count) is equals to "X-Total-Count" i.e ${fragment.count:equals(${X-Total-Count})}. then it notify to pass all data "MergerRecord" and a single FlowFile would generates.
... View more
12-21-2022
06:07 AM
I have a scenario where am triggering API in batches (1000 records in each). from the first API I get the value of total records in attribute list X-Total-Count, i wants to merge them all. Please let me know how i can leverage the wait processor in it to wait until we get the whole data till value of X-Total-Count. Wait Processor properties, i have configured.
... View more
Labels:
- Labels:
-
Apache NiFi
11-13-2022
11:50 PM
Hi, I have a use, where i need to upload the processor group (.json) file from the curl / API. Please suggest. OR can we do this operation in NiFi CLI without using NiFi registry.
... View more
Labels:
- Labels:
-
Apache NiFi
10-11-2022
02:22 AM
Hi Team, Am using self signed certificates, for API calling but while import the configuration passwords are not stick with the properties, can someone please suggest a method, How we can add truststore and keystore password using NiFi CLi or via API call to SSL ContextService.
... View more
Labels:
- Labels:
-
Apache NiFi
09-05-2022
05:41 AM
@araujo No, I'm getting the same current time in "timeOneMinu", Subtraction is not performing.
... View more
09-05-2022
03:07 AM
Hi Everyone, I have a use case, where i need to generate timestamp in Epoch UTC format till seconds 1662371646, and perform Substract operation for generate one minute ago time in same format. below is my Jolt Spec : [
{
"operation": "default",
"spec": {
"currenttime": "${now():toNumber()}"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"timeOneMinu": "=Subtract(@(1,currenttime),60)"
}
}
] Output : {
"currenttime": "1662372281",
"timeOneMinu": "1662372221"
} Please suggest, How we can do this.
... View more
Labels:
- Labels:
-
Apache NiFi
09-02-2022
03:09 AM
I have a use case where I have couple of key values and perform if-else operation on it. If condition is not matched then whole content will drop, else pass the content as a result. Input json: {
"id": 30006,
"SourceName": "network",
"Number": 1,
"SourceNameCopy": "network",
"currenttime": "Thu Aug 30 21:19:27 IST 2022"
} My Jolt Spec: [
{
"operation": "shift",
"spec": {
"SourceNameCopy": {
"network": {
"@1": "&2",
"#id": "id",
"#SourceName": "SourceName",
"#Number": "Number",
"#currenttime": "currenttime"
},
"hardware": {
"@1": "&2",
"#id": "id",
"#SourceName": "SourceName",
"#Number": "Number",
"#currenttime": "currenttime"
}
}
}
}
] Expected output if condition matched : {
"id": 30006,
"SourceName": "network",
"Number": 1,
"SourceNameCopy": "network",
"currenttime": "Thu Aug 30 21:19:27 IST 2022"
} Else (condition not matched) Drop the whole event as null. Problem Statement : The Key values is getting as a string, it should contain actual value in output as a result.
... View more
Labels:
- Labels:
-
Apache NiFi
08-31-2022
06:13 AM
Hi @hegdemahendra @SirV , I have a same scenario, and your point 1 resolution seems straight forward, but i didn't get in "MergeContent" processor. Your first dataEvaluateJsonPath is going to add in MergeContent with comman attribute "FILLA_ID" and detail_file content also going to MergeContent processor. but its not giving result with mapped field. sharing in my flow. Please suggest.
... View more
08-03-2022
10:46 PM
Hi @araujo , Please find the command output.
... View more
- « Previous
-
- 1
- 2
- Next »