Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hi, I need to create ETL pipeline, files are generated after hour and then these files are moved to HDFS now i want to load these files into Hive table using below command but i want to automate this process.

hi, I need to create ETL pipeline, files are generated after hour and then these files are moved to HDFS now i want to load these files into Hive table using below command but i want to automate this process.

New Contributor

LOAD DATA INPATH '/Project/' OVERWRITE INTO TABLE tb1;

3 REPLIES 3
Highlighted

Re: hi, I need to create ETL pipeline, files are generated after hour and then these files are moved to HDFS now i want to load these files into Hive table using below command but i want to automate this process.

Contributor

Let me know how you generate the files.

Re: hi, I need to create ETL pipeline, files are generated after hour and then these files are moved to HDFS now i want to load these files into Hive table using below command but i want to automate this process.

New Contributor

JSON files are generated from twitter API using python now i want to load these files in HIVE tables but need to automate this process.

Re: hi, I need to create ETL pipeline, files are generated after hour and then these files are moved to HDFS now i want to load these files into Hive table using below command but i want to automate this process.

Contributor

You can write your processes into script file.

test.sh

# Run the python to generate JSON file

# hive -e "LOAD DATA INPATH '/Project/' OVERWRITE INTO TABLE tb1;"

# hive -e "select * from your table"

Run test.sh !

Don't have an account?
Coming from Hortonworks? Activate your account here