Member since
06-08-2017
1049
Posts
514
Kudos Received
312
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
8274 | 04-15-2020 05:01 PM | |
4850 | 10-15-2019 08:12 PM | |
1839 | 10-12-2019 08:29 PM | |
8071 | 09-21-2019 10:04 AM | |
2779 | 09-19-2019 07:11 AM |
06-20-2018
10:21 PM
@Sonny Chee As a temporary work around to fix this issue please look into this link.
... View more
06-19-2018
02:20 PM
1 Kudo
@Vladislav Shcherbakov I think you are missing small configuration in InferAvroSchema processor Change Schema Output Destination
flowfile-attribute Instead of flowfile-content then the inferred.avro.schema attribute will be added to the flowfile. Writes Attributes: Name Description inferred.avro.schema If configured by "Schema output destination" to write to an attribute this will hold the resulting Avro schema from inferring the incoming FlowFile content. Once you make this changes then ConverCSVto Avro processor will be able to find out inferred.avro.schema attribute.
... View more
06-18-2018
07:34 PM
1 Kudo
@Suresh Dendukuri In Record Writer controller service of QueryRecord processor make sure you are having casecount name defined with the datatype associated with it.
... View more
06-18-2018
09:09 AM
2 Kudos
@Vladislav Shcherbakov When you are uploaded the template all the controller services will be in Disable state. We need to enable controller services associated with the PutDatabaseRecord processor In your case your need to enable CsvReader and DBCP connection pool controller service step1: Right Click on the processor and go to configure Step2: Enable CsvReader controller service Click on the --> after CSVReader controller service and click on Enable button Enable DBCP connection pool: Change the connection string(if needed) and make sure the jar path and driver class name,username and password is configured correct and click on Enable button DBCP connection pool
... View more
06-16-2018
09:37 PM
@Bal P For avro data file schema will be embedded and you want to write all the fields in CSV format so we don't need to setup the registry. if you are writing only specific columns not all and for other formats JSON.. then schema registry is required
... View more
06-16-2018
08:28 PM
1 Kudo
@Bal P Try with Schema Access Strategy as Embedded Avro Schema in AvroReader and Schema Access Strategy Inherit Record Schema in CsvrecordSetWriter controller services Avro Reader: CsvSetwriter: With the above configs i'm able to get the output flowfile in csv format as shown below. empno,name,deptno,age,sal
10,abcadefshnz,100,20,12700.5
... View more
06-14-2018
09:57 PM
1 Kudo
@Sudheer Velagapudi It's a bug in NiFi, couldn't able to refresh the kerberos ticket, to fix this issue please upgrade to HDF-3.1.2 (or) get in touch with support team regarding this issue to get Hot fix. Refer to this link regarding HDF-3.1.2 release https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_release-notes/content/ch_hdf_relnotes.html
... View more
06-14-2018
09:47 PM
@Sudheer Velagapudi Once you disable/enable HiveController service the issue is not going to happen unitil sometime right?
... View more
06-14-2018
09:21 PM
@Sudheer Velagapudi
Which version of NiFi/HDF are you using is it NiFi-1.5?
... View more
06-14-2018
01:00 PM
@rajat puchnanda Merge a group of flowfile (or) records is possible with MergeContent/MergeRecord processors. Example: if flowfile(ff1) having 123 records then ff2 having 345 by using mergecontent/record processors we can merge these flowfiles in to one like 123345. Merge means combining the group of records/flowfiles(union all) ,if you want to remove duplicates(i.e 3 is duplicate record) from the combined record flowfile content then you can use QueryRecord Processor with row_number window function to eliminate duplicates. This scenario is possible with NiFi without using lookup record processors. But as you mentioned in one of the answer Scenario2: InputFile 1 deptid firstname lastname 1 Aman Sharma 2 Raman Verma InputFile 2 deptid salary email 1 20000 abc@gmail.com 2 30000 bgf@gmail.com OutputFile(By merging file1 and file2):- deptid firstname lastname salary email 1 Aman Sharma 20000 abc@gmail.com 2 Raman Verma 30000 bgf@gmail.com This is not possible with MergeContent/Record but you can try with QueryRecord processor by implementing group and collect as set (or) some sort of sql logic in queryrecord processor to transpose the data into your desired format.This query Would be intensive if you are doing on larger number of records.
... View more