Created 12-05-2022 01:54 AM
Is there a way in NIFI to create Hive tables from using the schema from CDP schema registry.
Created 12-09-2022 06:03 AM
@Althotta I do not think this is possible for the create to reference schema registry, only insert/update. To do this table creation in NiFi you will need to do some flowfile content manipulation work to fabricate the CREATE TABLE statement. For example if the flowfile content is the avro schema (you could get this form the schema registry API) then you can parse out the columns and data types to get the middle of the create statement:
(col_name data_type [COMMENT 'col_comment'],, ...)
then a ReplaceText processor to add the top lines:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name
and bottom lines:
[COMMENT 'table_comment'] [ROW FORMAT row_format] [FIELDS TERMINATED BY char] [STORED AS file_format];
This would give you a full create statement which you can then execute against Hive.
Created 12-15-2022 05:10 AM
@Althotta, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
Regards,
Vidya Sargur,