Member since
06-26-2015
515
Posts
137
Kudos Received
114
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2025 | 09-20-2022 03:33 PM | |
5610 | 09-19-2022 04:47 PM | |
3042 | 09-11-2022 05:01 PM | |
3354 | 09-06-2022 02:23 PM | |
5301 | 09-06-2022 04:30 AM |
09-01-2022
10:15 PM
@ajignacio , What's the output of the command below if you run it from the same machine where NiFi is running? openssl s_client -connect ldap.dev.abcde:389 I know you are not using TLS, but the command above can still give us some useful information. Cheers, André
... View more
09-01-2022
10:09 PM
I actually wanted to have a look at the binary Avro data that is in Kafka, not the deserialized content. Something like this: kafka-console-consumer --from-beginning --bootstrap-server admin:9092 --topic pbs_jobs --max-messages 1 > message.avro Cheers, André
... View more
09-01-2022
02:25 AM
Would you be able to save one of these files in a file and share it with me?
... View more
08-31-2022
06:32 PM
@syntax_ , I believe you have a schema that you can use to parse your Avro data, right? Instead of using ConsumeKafka, use the ConsumeKafkaRecord processor. In that processor specify an Record Reader of type AvroReader and provide the correct schema so that the reader can properly deserialize your data. If you want to convert the data for JSON, you can then specify a JsonRecordSetWriter as the Record Writer for that processor, so that the output flowfiles will be in that format and you'll be able to inspect the content of the queues. Cheers, André
... View more
08-30-2022
03:47 PM
@Rgupta7 , Check your flows for flowfiles stuck in queues (.e.g. maybe you have connections to stopped processors or dead-end funnels and messages stay there indefinitely). Flowfiles are physically stored in bigger files that contain many flowfiles. If even one of those flowfiles are referenced by any queues, those files are never removed from disk. Cheers, André
... View more
08-30-2022
03:41 PM
@yagoaparecidoti , The opposite command doesn't exist, as far as I know. Querying the database is the only option that I'm aware. Cheers, André
... View more
08-30-2022
06:36 AM
Could you please run the kinit commands for both accounts and share a screenshot showing the command line and the output?
... View more
08-30-2022
06:32 AM
The names you listed are the servicePrincipalName. These are different from the userPrincipalName. Could you please check the latter and let me know what they are? Cheers, André
... View more
08-30-2022
05:24 AM
@sathish3389 , One way to do this is to use the ExecuteScript processor with a script like the one below. This script with set the attribute dv_sys_id with the content that you want and will also add that to the content of the flowfile. from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback
import json
class PyStreamCallback(StreamCallback):
def __init__(self):
self.dv_sys_id = None
def process(self, inputStream, outputStream):
text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
obj = json.loads(text)
self.dv_sys_id = 'dv_sys_id=%s' % (','.join(obj['dv_sys_id'],))
outputStream.write(bytearray(self.dv_sys_id.encode('utf-8','ignore')))
flow_file = session.get()
if flow_file != None:
callback = PyStreamCallback()
flow_file = session.write(flow_file, callback)
flow_file = session.putAttribute(flow_file, 'dv_sys_id', callback.dv_sys_id)
session.transfer(flow_file, REL_SUCCESS) Cheers, André
... View more