About bmoisson

bmoisson · ‎03-08-2022

Well I found the issue. I was the failing point 😁 I didn't put the option Treat First Line as Header to True so because my header was treat as a row, the schema was not Valid. Thanks for all, it seems to be a good way to change schema so I'll close this topic

bmoisson · ‎03-08-2022

@araujo Thanks for the answer. From this : https://community.cloudera.com/t5/Support-Questions/I-want-to-generate-Avro-Schema-from-CSV-file-using-Kite-SDK/td-p/309180 I took a convertRecord processor, putted my CSV inside this processor, and then I inferred schema with the CSV reader and wrote with CSV Writer with the option : Set 'avro.schema' Attribute. This allowed me to get an Avro Schema without having to type everything. I putted this schema as the Schema Text property of the CsvRecordReader setting the Schema Strategy to "Use Schema Text" as you mentioned, but it seems that this strategy do not allow me to transform String to Int (I get an error "Could not parse incoming Data", that I can remove if I replace the type int of my schema to string) I am not sure if the ConvertRecord processor is able to convert a String from the CSV, to an int, base on the schema i defined. May be it's my understanding of avro which is not good.

bmoisson · ‎03-07-2022

Hello all, I would like to know what would be the best approach, to change a schema of flowfile. I have got the following scenario : I ingest a csv file, and I would like then to rename column name inferred from the header and put them in a parquet file. Additionally, I want to cast some columns (String) to Integer or other types. I think it may be possible to do this job using the convertRecord component, or may be with the QueryRecord component but I am not sure if it is the best approach. Do you have any idea? If you have got few example it would be nice too. Thanks

bmoisson · ‎02-11-2022

Hello, I would like to connect my Airflow application with my Apache Nifi application though the Apache Nifi API. The aim is to let the Airflow orchestrate my entire flow. My Apache Nifi authentication is made via the OpenId protocol on a Keycloak application so I first get a token via a call to the auth/realms/myrealms/protocol/openid-connect/token and then with the generated JWT, I call the nifi-api to catch some processors, passing the JWT as a bearer token, I have got the following error : Bearer error="invalid_token", error_description="An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found", error_uri="https://tools.ietf.org/html/rfc6750#section-3.1" I tried the same curl call to the processor, using the token I get when I connect with the UserInterface, and I get a success. I supposed it may be due to the internal user policy of Nifi who rejecte me so I added username and email of the first token to the Nifi policy, but it change nothing. Does someone may help me please? Thanks

Online	Offline
Last Visited	‎03-10-2022 09:38 AM

Member Since	‎02-11-2022 08:19 AM
Last Visited	‎03-10-2022 09:38 AM
Posts	4
Kudos received	2

Cloudera Community

Re: Nifi : Best way to change schema

Re: Nifi : Best way to change schema

Nifi : Best way to change schema

Connect Airflow to Nifi and access the Nifi-Api th...