Member since
07-08-2016
260
Posts
44
Kudos Received
10
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2263 | 05-02-2018 06:03 PM | |
4546 | 10-18-2017 04:02 PM | |
1449 | 08-25-2017 08:59 PM | |
2017 | 07-21-2017 08:13 PM | |
8036 | 04-06-2017 09:54 PM |
06-06-2018
04:45 PM
Hi,
i need to transform the input JSON below {
"TransactionId": "-1",
"Source": "ihrs",
"Name": "tag-data",
"Id": "126",
"Records": [
{
"Name": "tag-master",
"PayLoad": {
"file-name": "testfile.json",
"plant": "1086",
"collector": "testcollector",
"tag-name": "testtag",
"tag-description": "test desc"
}
},
{
"Name": "tag-detail",
"PayLoad": {
"tag-value": "98998.55",
"tag-timestamp": "2018-01-02T16:09:39.000000+00:00",
"tag-quality": "100"
}
},
{
"Name": "tag-detail",
"PayLoad": {
"tag-value": "91009.47",
"tag-timestamp": "2018-01-02T16:09:340000000+00:00",
"tag-quality": "100"
}
},
{
"Name": "tag-detail",
"PayLoad": {
"tag-value": "91021.80",
"tag-timestamp": "2018-01-02T16:09:41.000000+00:00",
"tag-quality": "100"
}
}
]
} to output like below. {
"TransactionId": "-1",
"Source": "ihrs",
"Name": "tag-data",
"Id": "100",
"Records": [
{
"Name": "tag-master",
"PayLoad": {
"file-name": "testfile.json",
"plant": "1000",
"collector": "testcollector",
"tag-name": "testtag",
"tag-description": "test desc",
"Children": [
{
"Name": "tag-detail",
"PayLoad": {
"tag-value": "98998.55",
"tag-timestamp": "2018-01-02T16:09:39.000000+00:00",
"tag-quality": "100"
}
},
{
"Name": "tag-detail",
"PayLoad": {
"tag-value": "98998.55",
"tag-timestamp": "2018-01-02T16:09:39.000000+00:00",
"tag-quality": "100"
}
}
]
}
}
]
} i need to move the details records under master as array..
basically first element of the record array (tag master)is parent to next 3 elements(tag details). i tried to use the convert record processor with JSON converters giving correct input and output schemas for files. it didnt convert. it looks like i may have to write JOLT which i need some help with.
Thanks in advance!!! attached files for reference.. input.jsonoutput.json Regards,
Sai
... View more
Labels:
- Labels:
-
Apache NiFi
05-29-2018
08:19 PM
Hi, On my windows NiFi server , I am able to send unauthenticated emails using PutEmail processor by supplying SMTP Host name, Port 25 and user credentials. But in the email subject starts with UNVERIFIED SENDER. but when I tried with port 587 which we are using for authenticated emails , I am getting the below error. I am able to send emails using port 587 using PowerShell command. so i know the user has permissions to send emails on that port. any ideas on how to solve this.??? 15:10:47 CDT
ERROR
015e112b-6c5c-180e-115e-cece012de960 PutEmail[id=015e112b-6c5c-180e-115e-cece012de960] Failed to send email for StandardFlowFileRecord[uuid=741a83b9-ee95-4bef-aea4-93ae30d28c17,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1527624082884-412, container=default, section=412], offset=361603, length=4],offset=0,name=1232268045691876,size=4]: 535 5.7.3 Authentication unsuccessful
; routing to failure: javax.mail.AuthenticationFailedException: 535 5.7.3 Authentication unsuccessful
... View more
Labels:
- Labels:
-
Apache NiFi
05-29-2018
04:18 PM
@Matt Burgess But ListHDFS will keep the state and only supposed to pull the changed files.right.?? @Paul Hernandez what are the properties of your ListHDFS.?
... View more
05-18-2018
08:54 PM
@Shu, i thought about this , but the only issue is my files are huge and to split them by lines may not be ideal.
... View more
05-18-2018
07:36 PM
Hi, I am trying to update header of my CSV file with a regular expression to remove special chars from header line only. how to do that.? i tried to do that by reading the file and on one route to RouteText,ReplaceText,ExtractText to get the firstline and storing it in headerline attribute. and another route to move the file without the header and tried to Merge it by using headerline from route1. But It only shows header in output when i first file arrives from Route 1 into MergeContent processor as it has the headerline property where as if it gets it from Route 2 , the output file doesnt have the headerline as it doent have that property. any idea how to solve this..?
... View more
Labels:
- Labels:
-
Apache NiFi
05-02-2018
06:03 PM
@vinayak krishnan , if you are using the unpackcontent processor , fragment.count should give you the count. Regards, Sai
... View more
04-16-2018
01:29 PM
@jwitt, @Shu, Thanks for your input , I am actually following the similar approach. But was trying to see if there is any thing I can do at a file level instead of splitting it . Regards, Sai
... View more
04-13-2018
03:56 PM
Hi, I have a JSON file coming in from google big query in the format below..(it looks like Google BQ only supports extracting as JSON with newline delimited ). {“person”:”person1 value”} {“person”:”person2 value”} so i am not able to use that file as regular JSON file and not able to use any JSON processors (Evaluate,Split etc) ..how can i change it to a valid readable file (look below) and use it with other processors. [{“person”:”person1 value”}, {“person”:”person2 value”}] i do not have a schema for incoming JSON file. Thanks, Sai
... View more
Labels:
- Labels:
-
Apache NiFi
04-06-2018
04:20 PM
Hi @Matt Burgess , i replaced the space with %20 after Bearer , but now getting this response.. { "error": { "errors": [ { "domain": "global", "reason": "authError", "message": "Invalid Credentials", "locationType": "header", "location": "Authorization" } ], "code": 401, "message": "Invalid Credentials" } } where as if i try the url directly using , its working..any help.?? https://www.googleapis.com/storage/v1/b/tarapareddy/o/test2?access_token=${access_token}
... View more
04-06-2018
03:57 PM
@Matt Burgess , I am getting this error when I tried to send the Authorization from Attributes To Send..any idea why.?? InvokeHTTP[id=e6af6693-2428-3261-5418-a9cd3fc4c76b] Routing to Failure due to exception: java.lang.IllegalArgumentException: Unexpected char 0xa0 at 14 in header value: Authorization: Bearer ya29.GlyVBYcjf5qiEVyoPHJzDjf3ZLiXacV5Oaoaj817Tl-FzChWs9TSRc3vVNO8aMEj_HGHQMStSSJOwzQ3-WOygapsx8n4qmM-IVWFstZFtMa5Ld-47ZmKSClWEKFW5A: java.lang.IllegalArgumentException: Unexpected char 0xa0 at 14 in header value: Authorization: Bearer ya29.GlyVBYcjf5qiEVyoPHJzDjf3ZLiXacV5Oaoaj817Tl-FzChWs9TSRc3vVNO8aMEj_HGHQMStSSJOwzQ3-WOygapsx8n4qmM-IVWFstZFtMa5Ld-47ZmKSClWEKFW5A
... View more