Member since
07-29-2020
574
Posts
320
Kudos Received
175
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
248 | 12-20-2024 05:49 AM | |
282 | 12-19-2024 08:33 PM | |
294 | 12-19-2024 06:48 AM | |
245 | 12-17-2024 12:56 PM | |
238 | 12-16-2024 04:38 AM |
07-24-2024
04:46 AM
1 Kudo
Hi @anasb , I honestly never done it using .NAR extension. I have always placed my processor under the $NIFI_HOME/python/extensions directory and let Nifi handle the dependency download (https://nifi.apache.org/documentation/nifi-2.0.0-M3/html/python-developer-guide.html#deploying) For .NAR packaging please make sure to follow the steps here: https://nifi.apache.org/documentation/nifi-2.0.0-M3/html/python-developer-guide.html#dependency-isolation Also make sure you create the same folder structure specified in the instruction. From the error message you provided it seems like you are missing\misplaced the "NAR-INF/bundled-dependencies" where all dependencies are stored. Hope that helps.
... View more
07-23-2024
02:52 PM
I guess Im having hard time understanding what are you trying to do . In the first post you said you are just wanting to copy the value from Currency Condition into Calculate Pricee, so I assumed there is always going to be one value in the Currency Condition Array, but now you are saying can be dynamic and it can change . So what are the rules here in case you have multiple values? how do you decide? Can you specify the different scenarios that you would have in the input and what is the expected output for each ?
... View more
07-23-2024
02:44 AM
1 Kudo
Hi, I think you can do this part of the first shift spec unless Im missing something: [
{
"operation": "shift",
"spec": {
"tiers": {
"*": {
"priceRuleAttributes": {
"*": {
"id": {
"PR_CURR_CONDITION": {
"@(2,values[0])": "paths.[&5].currencyCondition[#2].currency"
}
}
}
},
"rewards": {
"*": {
"changeType": "paths.[&3].calculatePrice[&1].calculationType",
"changeAmount": "paths.[&3].calculatePrice[&1].amount",
"changePercent": "paths.[&3].calculatePrice[&1].percentage",
"@(2,priceRuleAttributes[0].values[0])": "paths.[&3].calculatePrice[&1].currency"
}
}
}
}
}
}
]
... View more
07-23-2024
02:34 AM
1 Kudo
Hi @SushmaNaya , I believe this was asked before here but I dont think there is a way to do it in one shot. you probably have to do it in custom script like executeScript processor where you can have part of the data stored as an attributes (depending how big the part is ) and then the other part is your flowfile content or have both stored as attributes (not recommended). Once you get the flowfile the execute script will get the content of the flowfile and the other aprt form the attribute construct your multi part body and send it that way. By the way if you are using Nifi 2.0 you probably can utilize python extensions to create your own custom processor using pure python when you can use different packages to help you with that. You can find example here: https://community.cloudera.com/t5/Support-Questions/Nifi-1-18-multipart-form-data-with-binary-part-and-json-part/m-p/386351#M246014 https://www.w3schools.com/python/ref_requests_post.asp Hope that helps
... View more
07-20-2024
05:51 PM
3 Kudos
Hi, Another option is to use FreeFormTextRecrodSetWriter for that. The documentation is lacking about this unfortunately but you can find some examples like this if you google it. All you need is a ConvertRecord processor to get the desired result. here is an example : - GenerateFlowFile is to simulate generating the csv input: - ConvertRecord: which takes the CSV input using CSVReader and Record Writer using the FreeFormTextRecordSetWriter: - CSVReader Service Configuration: You can use default configuration. -FreeFormTextRecordSetWriter: The Text used in the above Service to provide desired output: username = ${username}
first name = ${"first name"}
middle name = ${"middle name"}
last name = ${"last name"} Output: username = test_user
first name = test_FN
middle name = test_MN
last name = test_LN
username = test_user2
first name = test2_FN
middle name = test2_MN
last name = test2_LN Hope that helps.
... View more
07-15-2024
07:29 AM
1 Kudo
Hi, Can you elaborate more on the issue. For example are you getting an error? what is the status code of the response?I assume you are using InvokeHttp processor, Can you paste screenshot of the configuration you have ? What kind of API is it and are you sure this a GET API vs Others ? Have you tried to do the GET from pastman or curl ? was it successful? It seems using body with Get API is not good idea after all per below links , so if its something developed internally you probably should re consider. https://www.linkedin.com/pulse/using-body-http-get-method-still-bad-idea-danny-logsdon https://www.baeldung.com/cs/http-get-with-body
... View more
07-12-2024
10:18 AM
For jslt , like I said sometimes its easier to write and work with specially if you are trying to get\aggregate data from heavily nested structure. The other advantage jslt has its ability to traverse nested levels dynamically using recursive function calls for example lets say you are working with json structure but have no idea how many nested level it might have (see my json\jslt answer in the this post to get an idea what I mean by that ). On the other hand jolt appears to perform a little bit better in comparison but they are very close. In regards to your question you use the follow to extract single value for the path name: "@(2,values[0])": "paths.[&5].pathName"
... View more
07-12-2024
03:58 AM
1 Kudo
Hi, I dont think there is hostname property under ListenHttp and if there was at some point it probably got removed later on. Regarding the hostname under HandleHttpRequest, its basically to specify which node you would like to receive the request on in case of cluster. If you specify certain node it will only allow connection to that node only however if you leave it empty as the description says it will bind to all which mean you can send request using any node host name. Hope that helps.
... View more
07-12-2024
03:05 AM
2 Kudos
Hi, Actually I went against the main input since the second input (simplified version) is not valid json. Hopefully the following will work: [
{
"operation": "shift",
"spec": {
"description": "priceRuleName",
"startDatetime": "startDate",
"endDatetime": "enddate",
"locations": {
"0": {
"location": "storeLocation"
}
},
"customAttributes": {
"*": {
"id": {
"PR_PRICE_RULE_PROMO": {
"@(2,valueBoolean)": "priceRulePromo"
}
}
}
},
"tiers": {
"*": {
"priceRuleAttributes": {
"*": {
"id": {
"PR_BRANCH_NAME": {
"@(2,values)": "paths.[&5].pathName"
},
"PR_TARGET_SEGMENT": {
"@(2,values)": "paths.[&5].customerCondition[#2].targetSegments"
}
}
}
},
"rewards": "paths[&1].calculatePrice",
"conditions": {
"*": {
"conditionsMerch": {
"*": {
"item": {
"": {
"@(2,excludeInd)": {
//group categories under yes\no include ind
"0": {
"@(4,categoryId)": "paths[&9].catalogCondition.category.yes.categoryList[]"
},
"1": {
"@(4,categoryId)": "paths[&9].catalogCondition.category.no.categoryList[]"
}
}
}, //there is a valye in item -> products
"*": {
"@(2,excludeInd)": {
//group products under yes\no include ind
"0": {
"@(4,item)": "paths[&9].catalogCondition.products.yes.productList[]"
},
"1": {
"@(4,item)": "paths[&9].catalogCondition.products.no.productList[]"
}
}
}
}
}
}
}
}
}
}
}
}
,
{
"operation": "shift",
"spec": {
"*": "&",
"paths": {
"*": {
"*": "&2[&1].&",
"catalogCondition": {
"*": { //products or category
"*": { //bucket each yes\no group into its own array element
"*": "&5[&4].&3.&2[#2].&",
"$": "&5[&4].&3.&2[#2].include"
}
}
}
}
}
}
}
/**/
] Also something to consider is using JSLT. JSTL is another json transformation language similar to xquery in syntax ( check reference here) . There is jslt processor in nifi for that. In my openion jslt works better in these situation specially when you trying to query data from very nested structure using some complex logic to match on certain values . You probably have to write less lines in jslt and the logic is easier to follow than jolt (of course if you have get familiar with jslt syntax and how it works first to make sense of it) Here is how this transformation looks using jslt: import "http://jslt.schibsted.com/2018/experimental" as exp
let priceRulePromoArray = [for (.customAttributes) .valueBoolean if(.id=="PR_PRICE_RULE_PROMO")]
let mainInfo = {
"priceRuleName":.description,
"startDate":.startDatetime,
"enddate":.endDatetime,
"storeLocation":.locations[0].location,
"priceRulePromo":$priceRulePromoArray[0]
}
let paths= [for(.tiers)
{
"pathname":flatten([for(.priceRuleAttributes) .values
if(.id=="PR_BRANCH_NAME")]),
"customerCondition":[for(.priceRuleAttributes).values
if(.id=="PR_TARGET_SEGMENT")],
"calculatePrice": .rewards,
"catalogCondition":[for(.conditions)
let groupProductByInd =
exp:group-by([for (.conditionsMerch) . if(.item!="")]
,.excludeInd,
.item)
let groupCategoryByInd =
exp:group-by([for (.conditionsMerch) . if(.categoryId!="")]
,.excludeInd
, .categoryId)
let products = [for ($groupProductByInd )
{
"include": if(.key==0) "yes" else "no",
"productList":.values
}
]
let category = [for ($groupCategoryByInd )
{
"include": if(.key==0) "yes" else "no",
"categoryList":.values
}
]
if(any($products)) {"products":$products} else {} +
if(any($category )) {"category":$category} else {}
]
}
]
{"paths":$paths}+$mainInfo
... View more
07-11-2024
10:47 AM
Hi @Syed0000 , There is a lot going on in the json input and expected output that is hard to follow what you are trying to do . Can you shorten\simplify your json input by removing un necessary or or repeated info. It would be even better to come up with dummy json that focuses on the issue. That would help me in providing accurate resolution. Thanks
... View more