<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question hive insert into struct fields get_json_object in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/hive-insert-into-struct-fields-get-json-object/m-p/81028#M4105</link>
    <description>&lt;P&gt;Hello&lt;/P&gt;
&lt;P&gt;I have hdfs json files, specifically the files are tweets messages in json format, i'm trying to read that files with hive into a table.&lt;/P&gt;
&lt;P&gt;i'm using the function get_json_object to extract each element of the json an insert in a table field.&lt;/P&gt;
&lt;P&gt;Te problem is that i don't know how to manage struct fields, for example i have defined in th testination table a struct field with several fields for example, the field&amp;nbsp;&lt;SPAN&gt;retweeted_status&amp;nbsp; is a srutct field with two fields&lt;/SPAN&gt;&amp;nbsp;:&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;first i create a table to insert the tweets like a string:&lt;/P&gt;
&lt;P&gt;CREATE EXTERNAL TABLE `tweets_raw`(&lt;BR /&gt;`json_response` string)&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;then i create a second table where i want to split each json field in a table field. i created the next table:&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;CREATE TABLE tweets&lt;/P&gt;
&lt;P&gt;(&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp; in_reply_to_status_id_str string,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; in_reply_to_status_id string,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; created_at STRING,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; in_reply_to_user_id_str STRING,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; source string ,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; retweeted_status STRUCT&amp;lt;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; in_reply_to_status_id_str:STRING,&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; in_reply_to_status_id:STRING&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; &amp;gt;,&lt;/P&gt;
&lt;P&gt;&amp;nbsp; retweet_count STRING,&lt;/P&gt;
&lt;P&gt;....&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;then, using the function get_json_object , i'm trying to insert data with a sentence like this:&lt;/P&gt;
&lt;P&gt;INSERT overwrite table tweets&lt;BR /&gt;SELECT&lt;BR /&gt;cast(get_json_object(json_response, '$.in_reply_to_status_id_str') as STRING),&lt;BR /&gt;cast(get_json_object(json_response, '$.in_reply_to_status_id') as STRING),&lt;BR /&gt;cast(get_json_object(json_response, '$.created_at') as STRING),&lt;BR /&gt;cast(get_json_object(json_response, '$.in_reply_to_user_id_str') as STRING),&lt;BR /&gt;cast(get_json_object(json_response, '$.source') as STRING),&lt;/P&gt;
&lt;P&gt;&lt;BR /&gt;struct( cast(get_json_object(json_response, '$.retweeted_status.in_reply_to_status_id_str') as STRING),&lt;BR /&gt;cast(get_json_object(json_response, '$.retweeted_status.in_reply_to_status_id') as STRING)&lt;BR /&gt;&lt;BR /&gt;&lt;BR /&gt;) as retweeted_status ,&amp;nbsp;&lt;BR /&gt;cast(get_json_object(json_response, '$.retweet_count') as STRING),&lt;/P&gt;
&lt;P&gt;...&lt;/P&gt;
&lt;P&gt;the simple fileds are ok, bur i don't know how insert the struct field&amp;nbsp;&lt;SPAN&gt;retweeted_status&amp;nbsp;, i try several things but none work,&amp;nbsp;using&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN&gt;struct( cast(get_json_object(json_response, '$.retweeted_status.in_reply_to_status_id_str') as STRING),&lt;BR /&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; cast(get_json_object(json_response, '$.retweeted_status.in_reply_to_status_id') as STRING)&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp;)&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&lt;SPAN&gt;hive give me the next error:&lt;/SPAN&gt;&lt;/P&gt;
&lt;DIV class="snippet-execution-status"&gt;
&lt;DIV class="snippet-error-container alert alert-error"&gt;
&lt;UL&gt;
&lt;LI&gt;Error while compiling statement: FAILED: SemanticException [Error 10044]: line 1:23 Cannot insert into target table because column number/types are different 'tweets': Cannot convert column 5 from struct&amp;lt;col1:string,col2:string&amp;gt; to struct&amp;lt;in_reply_to_status_id_str:string,in_reply_to_status_id:string&amp;gt;.&lt;/LI&gt;
&lt;/UL&gt;
&lt;/DIV&gt;
&lt;/DIV&gt;
&lt;DIV class="snippet-code-resizer ui-draggable"&gt;&amp;nbsp;&lt;/DIV&gt;
&lt;DIV class="snippet-code-resizer ui-draggable"&gt;anyone know how to insert values in a struct field?&lt;/DIV&gt;
&lt;DIV class="snippet-code-resizer ui-draggable"&gt;Thanks inadvance for your help&lt;/DIV&gt;
&lt;DIV class="snippet-code-resizer ui-draggable"&gt;best regards&lt;/DIV&gt;
&lt;DIV class="snippet-code-resizer ui-draggable"&gt;diego&lt;/DIV&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
    <pubDate>Fri, 16 Sep 2022 13:48:02 GMT</pubDate>
    <dc:creator>dcond</dc:creator>
    <dc:date>2022-09-16T13:48:02Z</dc:date>
  </channel>
</rss>

