Member since
12-03-2017
147
Posts
24
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1149 | 11-03-2023 12:17 AM | |
2887 | 12-12-2022 09:16 PM | |
1094 | 07-14-2022 03:25 AM | |
1748 | 07-28-2021 04:42 AM | |
2008 | 06-23-2020 10:08 PM |
07-23-2018
09:20 AM
@Matt Burgess , @Shu- Your help would be appreciated. Thanks
... View more
07-23-2018
09:19 AM
Hello, I have a json input for which I need to add one key-value("correlationId": "123456789") under 'tripSummaryMetaData' section dynamically. I tried with below spec and its working fine, but the issues is when 'tripSummaryMetaData' is not available the it is creating new 'tripSummaryMetaData' and adding my attribute. When 'tripSummaryMetaData' is not available the it should not add my new attribute, is that possible in JOLT? Thanks in advance. Input: {
"tripSummaryMetaData": {
"orgId": "demo",
"message_type_id": "MB-CD-MSG0001"
},
"eventSummary": {
"tripId": "2018072204cc7851d3e84f7ea8a1155c95f65b06",
"referenceData": "050bfa62-430a-45a5-b7a6-78d688d45167",
"gpsData": [{
"gpsTimeReceived": 1532319157338,
"gpsTime": "2018-07-22T21:12:38-07:00"
}]
}
} Output: {
"tripSummaryMetaData" : {
"orgId" : "demo",
"message_type_id" : "MB-CD-MSG0001",
"correlationId" : "123456789"
},
"eventSummary" : {
"tripId" : "2018072204cc7851d3e84f7ea8a1155c95f65b06",
"referenceData" : "050bfa62-430a-45a5-b7a6-78d688d45167",
"gpsData" : [ {
"gpsTimeReceived" : 1532319157338,
"gpsTime" : "2018-07-22T21:12:38-07:00"
} ]
}
} Spec : [{
"operation": "default",
"spec": {
"tripSummaryMetaData": {
"correlationId": "123456789"
}
}
}] Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
05-03-2018
09:33 AM
Yes, its possible. Alter table <table_name> set tblproperties ('transactional'='true');
... View more
05-03-2018
09:31 AM
Thanks for the nice explanation Ned. When I have created the flow, NiFi processor is able to store data to the table but its very slow in the cluster environment. I am able to push only 2-3 flowfile/records (few kb in size) per second. I tried with more thread counts as well, but no luck. Any suggestion will be much appreciated.
... View more
04-14-2018
06:00 PM
Thanks for replies, Issue was due to concurrent thread configuration. When we increase the max allowed thread count in controller settings, issue solved. We stopped all flows, increased the count, restarted the node and then flows.
... View more
04-14-2018
05:56 PM
hello, Facing an issue with 'InvokeHTTP' NiFi processor. Have configured below URL for GET operation - https://******/**/users/${userId} When I run the processor some time it will get proper response from url. But if I restart the processor, suddenly it will start telling 'URL not found' error and route the flow file to 'No Retry' relationship. If I try to restart the processor 2-3 times then once it will start responding correctly. Please suggest me to resolve the issue. screen shots of config - Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
03-29-2018
04:26 PM
Hi, I have a kafka consumer, that is consuming messages after that flowfiles are not moving at all. Suddenly this issue started. many messages are stuck at diff connection/queue in btw processors even though processor running it is not getting processing dataflow-struck.png Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
03-19-2018
11:09 AM
I am enriching my request payload with the payload attribute 'http.headers.orgId' and its working fine. SPEC: [{
"operation": "default", "spec": { "*": {
"orgId": "${http.headers.orgId}"
} } }] INPUT : {
"drivewiseDeviceData": {
"imei": "sampleimei",
"tripId": "3d9df81a-1828-40eb-940e-d2434eb5f50f",
"sumVer": null,
"payloadType": "Binary",
"packetInfo": "",
"payloadData": "g1bjSWX",
"recordinfo": null,
"sequenceNum": null,
"timeStamp": "2017-05-26T12:07:29.968Z" }
} OUTPUT: { "drivewiseDeviceData": { "imei": "sampleimei", "tripId": "3d9df81a-1828-40eb-940e-d2434eb5f50f", "sumVer": null, "payloadType": "Binary", "packetInfo": "", "payloadData": "g1bjSWX", "recordinfo": null, "sequenceNum": null, "timeStamp": "2017-05-26T12:07:29.968Z", "orgId": "Sample Org Name Here" } } ......................................................................................................................................... Now scenarios is I may get header attribute as 'http.headers.OrgID' or 'http.headers.orgId', So is there any way to handle both scenarios in my JOLT spec? If 'http.headers.orgId' value is null/empty/attribute-not-present then I should use 'http.headers.OrgID' for enrich.
... View more
Labels:
- Labels:
-
Apache NiFi
02-23-2018
10:47 AM
Thanks for the quick reply @Matt Clarke 🙂 One more query, I may have N number of files, so how can I find out the number of files in directory and ensure that all files reached Mergecontent processor before I proceed further to create trigger file and FTP ?
... View more
02-13-2018
10:39 AM
I have scenario where I need to transfer 10+ files from directory A to another server directory B using FTP at 10pm every day. After transferring all the files to directory B I have to create one trigger file in directory B to trigger the downstream process. How can I do this using NiFi ? Problem is creating trigger file in directory B at the end of all files transfer. please suggest. Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
- « Previous
- Next »