Member since
08-24-2020
15
Posts
0
Kudos Received
0
Solutions
05-18-2021
09:05 PM
Greetings, I want to fetch one particular file from S3, only once. So I used listS3 and FetchS3object processors. But whenever I start the listS3 processor and stop it after 1 second, by that time it generates thousands of flowfiles and when these flowfiles are passed to FetchS3object processor, the same file is fetched thousands of time. I even changed the scheduling time in listS3 processor, but still, the same effect exists. Could someone tell me where I was going wrong with the configurations?
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
11-02-2020
03:25 AM
Greetings, I wanted to know how to set up a process where one will receive an email when ever a queue reaches its threshold limit , in apache nifi. Thanks in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
10-19-2020
05:00 AM
I have a csv file which has many lines of data and many fields, but whenever i have a file which has "9799678467\" under "boardingPointContactNo" field it throws error. I want to replace "9799678467\" with "9799678467". I want to remove the "/" . Which processor should i be used to do this. This is the snippet of my csv data. serviceProviderName,serviceId,routeId,busType,sourceLocation,sourceLocationId,boardingPointId,boardingPointContactNo {fields} "abc travels and Cargo","106123","15476562","A/C Sleeper (2+1)","Surat","473","21765854","9799678467\" {values of each field} As you can see the data is comma separated. Thanks in advance
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
08-28-2020
05:39 AM
I used a json array and split it into multiple json files and did some operations on it. Then converted each of these json files to parquet form and i wanted to upload each individual parquet file to S3. Uploading a single json file or a json array was not an issue, but if i had (for example) 3 parquet files to be uploaded then, only 1 got uploaded into S3, thus the other 2 files are lost. Thus i am assuming PutS3Object processor is allowing the uploading of only 1 file at any given time (correct me if i am wrong) That leaves me with two solutions: 1. Either somehow allow multiple files to be uploaded into S3 2. Or merge individual json files to one json array and then doing the conversion and then pushing it to S3 I would be grateful if anybody helps me with either of the solutions. Thanks in advance.
... View more
Labels:
- Labels:
-
Apache NiFi
08-27-2020
11:07 PM
I want to host an API in Nifi using HandleHTTP processor. And i wanted to pass some dummy json data. I want to convert the JSON data to avro and then push it to S3 bucket. I am getting the following error when the json data goes from HandleHTTPResponse to ConvertRecord (Which is converting json -> Avro), thus the data goes to this failure pipeline. Can anyone help me out. Thanks in Advance
... View more
Labels:
- Labels:
-
Apache NiFi
08-27-2020
01:55 AM
I want to host an API in Nifi using HandleHTTP processor. And i wanted to pass some dummy json data. I would like to get some information/resources about how to do it. I am new to Nifi. Thanks in Advance.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
08-25-2020
04:23 AM
Thanks @SagarKanani i am now able to view the avro data. Sorry for the inconvenience, but i had another doubt. If i try to convert this Avro data to Parquet by using ConverAvroToParquet processor then i am getting an output like this: PAR1&&,6(prathikarunprathikarun,(<Hnifi.person% firstname%%age,&% firstnamenn&<6(prathikarun&v%agejj&v<(Ø,parquet.avro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}writer.model.nameavroJparquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a),ÃPAR1 If i try to put the "view as" option as formatted i get the following message: Unable to generate view of data: Not a data file. Thanks in Advance.
... View more
08-25-2020
03:41 AM
Hi @SagarKanani Thanks for your input. I have set the write strategy in AvroRecordSetWriter as Embedded Avro. And i have received the following output : Objavro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}avro.codecnull¢ÈFB¡-Ž`0„Ûv‚L@prathik*arun,¢ÈFB¡-Ž`0„Ûv‚L@ Again is red dot. One more Doubt, how can one view avro data if we can't view them on nifi. thanks.
... View more
08-24-2020
11:27 PM
Hi, I want to convert json data to avro. I have used GenerateFlowFile and put dummy json value [{"firstname":"prathik","age":21},{"firstname":"arun","age":22}]. I have then used ConvertRecord processor and set JsonTreeReader and AvroRecordSetWriter with AvroSchemaRegistry which has the following schema: { "name": "person", "namespace": "nifi", "type": "record", "fields": [ {"name": "firstname", "type": "string" }, { "name": "age", "type": "int" } ] } But i am getting this as my output : prathik*arun, where is a red dot I am new to Apache Nifi.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry