- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Create Hive Tables in NIFI using Schema from schema Registry
- Labels:
-
Apache Hive
-
Apache NiFi
Created 12-05-2022 01:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is there a way in NIFI to create Hive tables from using the schema from CDP schema registry.
Created 12-09-2022 06:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Althotta I do not think this is possible for the create to reference schema registry, only insert/update. To do this table creation in NiFi you will need to do some flowfile content manipulation work to fabricate the CREATE TABLE statement. For example if the flowfile content is the avro schema (you could get this form the schema registry API) then you can parse out the columns and data types to get the middle of the create statement:
(col_name data_type [COMMENT 'col_comment'],, ...)
then a ReplaceText processor to add the top lines:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name
and bottom lines:
[COMMENT 'table_comment'] [ROW FORMAT row_format] [FIELDS TERMINATED BY char] [STORED AS file_format];
This would give you a full create statement which you can then execute against Hive.
Created 12-15-2022 05:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Althotta, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
Regards,
Vidya Sargur,Community Manager
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
