Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Correct json format to (re)create a Hive table

Correct json format to (re)create a Hive table

Contributor

I've created a hive table with the following format:

CREATE TABLE test (
  id bigint,
  sentiment INT,
  created_at string,
  source STRING,
   favorited BOOLEAN,
   retweeted_status STRUCT<
     text:STRING,
     user:STRUCT<screen_name:STRING,name:STRING>,
     retweet_count:INT>,
   entities STRUCT<
     urls:ARRAY<STRUCT<expanded_url:STRING>>,
     user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
     hashtags:ARRAY<STRUCT<text:STRING>>>,
  lang string,
  retweet_count int,
  text string,
  user STRUCT<
     screen_name:STRING,
     name:STRING,
     friends_count:INT,
     followers_count:INT,
     statuses_count:INT,
     verified:BOOLEAN,
     utc_offset:INT,
     time_zone:STRING>
       )
STORED AS TEXTFILE

This table is filled with data .
I've fetched the data to a pandas dataframe and edited a whole column.
Now I want to put the table back into hive.

First I started by using a csv but got problems due to the structs.
Now I wish to try using a json via dataframe.to_json but I'm having problems with the parameters.
First of all, is this possible?
Can anybody pelase help

Don't have an account?
Coming from Hortonworks? Activate your account here