Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Create Hive Tables in NIFI using Schema from schema Registry

avatar
Rising Star

Is there a way in NIFI to create Hive tables from using the schema from CDP schema registry.

2 REPLIES 2

avatar

@Althotta I do not think this is possible for the create to reference schema registry, only insert/update.   To do this table creation in NiFi you will need to do some flowfile content manipulation work to fabricate the CREATE TABLE statement.  For example if the flowfile content is the avro schema (you could get this form the schema registry API) then you can parse out the columns and data types to get the middle of the create statement:

(col_name data_type [COMMENT 'col_comment'],, ...) 

then a ReplaceText processor to add the top lines:

 

CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name

and bottom lines:

[COMMENT 'table_comment'] [ROW FORMAT row_format] [FIELDS TERMINATED BY char] [STORED AS file_format];

 This would give you a full create statement which you can then execute against Hive.

avatar
Community Manager

@Althotta, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: