About P_Rat98

P_Rat98 · ‎05-18-2021

Greetings, I want to fetch one particular file from S3, only once. So I used listS3 and FetchS3object processors. But whenever I start the listS3 processor and stop it after 1 second, by that time it generates thousands of flowfiles and when these flowfiles are passed to FetchS3object processor, the same file is fetched thousands of time. I even changed the scheduling time in listS3 processor, but still, the same effect exists. Could someone tell me where I was going wrong with the configurations?

P_Rat98 · ‎11-02-2020

Greetings, I wanted to know how to set up a process where one will receive an email when ever a queue reaches its threshold limit , in apache nifi. Thanks in advance.

P_Rat98 · ‎10-19-2020

I have a csv file which has many lines of data and many fields, but whenever i have a file which has "9799678467\" under "boardingPointContactNo" field it throws error. I want to replace "9799678467\" with "9799678467". I want to remove the "/" . Which processor should i be used to do this. This is the snippet of my csv data. serviceProviderName,serviceId,routeId,busType,sourceLocation,sourceLocationId,boardingPointId,boardingPointContactNo {fields} "abc travels and Cargo","106123","15476562","A/C Sleeper (2+1)","Surat","473","21765854","9799678467\" {values of each field} As you can see the data is comma separated. Thanks in advance

P_Rat98 · ‎08-28-2020

I used a json array and split it into multiple json files and did some operations on it. Then converted each of these json files to parquet form and i wanted to upload each individual parquet file to S3. Uploading a single json file or a json array was not an issue, but if i had (for example) 3 parquet files to be uploaded then, only 1 got uploaded into S3, thus the other 2 files are lost. Thus i am assuming PutS3Object processor is allowing the uploading of only 1 file at any given time (correct me if i am wrong) That leaves me with two solutions: 1. Either somehow allow multiple files to be uploaded into S3 2. Or merge individual json files to one json array and then doing the conversion and then pushing it to S3 I would be grateful if anybody helps me with either of the solutions. Thanks in advance.

P_Rat98 · ‎08-27-2020

I want to host an API in Nifi using HandleHTTP processor. And i wanted to pass some dummy json data. I want to convert the JSON data to avro and then push it to S3 bucket. I am getting the following error when the json data goes from HandleHTTPResponse to ConvertRecord (Which is converting json -> Avro), thus the data goes to this failure pipeline. Can anyone help me out. Thanks in Advance

P_Rat98 · ‎08-27-2020

I want to host an API in Nifi using HandleHTTP processor. And i wanted to pass some dummy json data. I would like to get some information/resources about how to do it. I am new to Nifi. Thanks in Advance.

P_Rat98 · ‎08-25-2020

Thanks @SagarKanani i am now able to view the avro data. Sorry for the inconvenience, but i had another doubt. If i try to convert this Avro data to Parquet by using ConverAvroToParquet processor then i am getting an output like this: PAR1&&,6(prathikarunprathikarun,(<Hnifi.person% firstname%%age,&% firstnamenn&<6(prathikarun&v%agejj&v<(Ø,parquet.avro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}writer.model.nameavroJparquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a),ÃPAR1 If i try to put the "view as" option as formatted i get the following message: Unable to generate view of data: Not a data file. Thanks in Advance.

P_Rat98 · ‎08-25-2020

Hi @SagarKanani Thanks for your input. I have set the write strategy in AvroRecordSetWriter as Embedded Avro. And i have received the following output : Objavro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}avro.codecnull¢ÈFB¡-Ž`0„Ûv‚L@prathik*arun,¢ÈFB¡-Ž`0„Ûv‚L@ Again is red dot. One more Doubt, how can one view avro data if we can't view them on nifi. thanks.

P_Rat98 · ‎08-24-2020

Hi, I want to convert json data to avro. I have used GenerateFlowFile and put dummy json value [{"firstname":"prathik","age":21},{"firstname":"arun","age":22}]. I have then used ConvertRecord processor and set JsonTreeReader and AvroRecordSetWriter with AvroSchemaRegistry which has the following schema: { "name": "person", "namespace": "nifi", "type": "record", "fields": [ {"name": "firstname", "type": "string" }, { "name": "age", "type": "int" } ] } But i am getting this as my output : prathik*arun, where is a red dot I am new to Apache Nifi.

Online	Offline
Last Visited	‎05-19-2021 08:01 AM

Member Since	‎08-24-2020 11:25 PM
Last Visited	‎05-19-2021 08:01 AM
Posts	15

Cloudera Community

Cloudera Community

Fetch S3 file to Nifi

Sending an email whenever the Queue is Full- Nifi

How to modify text in csv file in Apache Nifi

pushing multiple objects in S3 bucket using PushS3...

Hosting an API and pushing Json data to S3 while c...

Hosting an API in Nifi

Re: Convert Json to Avro in Nifi

Re: Convert Json to Avro in Nifi

Convert Json to Avro in Nifi