Member since
08-17-2016
9
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2912 | 08-18-2016 01:42 PM |
08-19-2016
08:49 AM
Hi We are experiencing trouble in flowFile = session.write(flowFile, ModJSON()) that line. javax.script.ScriptException: TypeError: write(): 1st arg can't be coerced to int
, byte[] in <script> at line number 64
Do you think what is the problem: import json
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback
class ModJSON(StreamCallback):
def __init__(self):
pass
def process(self, inputStream, outputStream):
text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
obj = json.loads(text)
endorsements = obj['endorsement'].split(',')
categoryIds = obj['categoryIds'].split(',')
orderItemIds = obj['orderItemIds'].split(',')
seq_num = 0
seq_num1 = 1
for endorsement in endorsements:
if seq_num == 0 and seq_num1 == len(endorsements):
newObj = '[{"endorsement":' + endorsement \
+ ',"eventId":"' + obj['eventId'] \
+ '", "categoryId":"' + categoryIds[seq_num] \
+ '", "createDate":"' + obj['createDate'] \
+ '", "buyerId":"' + obj['buyerId'] \
+ '", "channel":"' + obj['channel'] + '", "city":"' \
+ obj['city'] + '", "orderItemId": "'+orderItemIds[seq_num]+'"}]'
seq_num += 1
seq_num1 += 1
elif seq_num == 0:
newObj = '[{"endorsement":' + endorsement + ',"eventId":"' \
+ obj['eventId'] + '", "categoryId":"' \
+ categoryIds[seq_num] + '", "createDate":"' \
+ obj['createDate'] + '", "buyerId":"' \
+ obj['buyerId'] + '", "channel":"' + obj['channel'
] + '", "city":"' + obj['city'] + '", "orderItemId": "'+orderItemIds[seq_num]+'"},'
seq_num += 1
seq_num1 += 1
elif seq_num1 == len(endorsements) :
newObj += '{"endorsement":' + endorsement + ',"eventId":"' \
+ obj['eventId'] + '", "categoryId":"' \
+ categoryIds[seq_num] + '", "createDate":"' \
+ obj['createDate'] + '", "buyerId":"' \
+ obj['buyerId'] + '", "channel":"' + obj['channel'
] + '", "city":"' + obj['city'] + '", "orderItemId": "'+orderItemIds[seq_num]+'"}]'
seq_num += 1
seq_num1 += 1
else:
newObj += '{"endorsement":' + endorsement + ',"eventId":"' \
+ obj['eventId'] + '", "categoryId":"' \
+ categoryIds[seq_num] + '", "createDate":"' \
+ obj['createDate'] + '", "buyerId":"' \
+ obj['buyerId'] + '", "channel":"' + obj['channel'
] + '", "city":"' + obj['city'] + '", "orderItemId": "'+orderItemIds[seq_num]+'"},'
seq_num += 1
seq_num1 += 1
outputStream.write(bytearray(newObj.encode('utf-8')))
flowFile = session.get()
if flowFile != None:
flowFile = session.write(flowFile, ModJSON())
flowFile = session.putAttribute(flowFile, 'filename',
flowFile.getAttribute('filename'
).split('.')[0] + '_translated.json'
)
session.transfer(flowFile, REL_SUCCESS)
session.commit()
... View more
08-18-2016
01:42 PM
i found the problem. Reason was the hbase, i am sending same values as key so, it is impossible. After changing key values, everything working fine. Thanks
... View more
08-18-2016
07:34 AM
because of the attachment limit i continue with this message: two splitted json: two putsql queryy:
... View more
08-17-2016
06:01 PM
Splitting is done correctly. I am watching al the processes till Last process.putsql processes passes insert sql statements according to data provenance. But still last json part as a record is inserted all the time.
... View more
08-17-2016
02:13 PM
1 Kudo
Hi I'am using splitjson for splitting json. After that i am using convertJsonToSql and putsql but nifi only puts last json part of the splitted jsons. I want to insert all parts. Do you have any idea why i can't do that. Thanks. source json: {
"endorsement" : 59.9,
"eventId" : "c54902db-50de-4dbd-b908-4317981931fc",
"categoryId" : "1000230",
"createDate" : "2016-08-18T10:15:00.361Z",
"buyerId" : "4748764",
"channel" : "MOBILE_IOS",
"city" : "Bursa"
}, {
"endorsement" : 13.11,
"eventId" : "c54902db-50de-4dbd-b908-4317981931fc",
"categoryId" : "1000491",
"createDate" : "2016-08-18T10:15:00.361Z",
"buyerId" : "4748764",
"channel" : "MOBILE_IOS",
"city" : "Bursa"
} ]
after split json there are two jsons:
... View more
Labels:
- Labels:
-
Apache NiFi