Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can I write a parquert file using Flume, Morphline, etc.?

Can I write a parquert file using Flume, Morphline, etc.?

Explorer

The data that I collect contains complex types and should guarantee a response time of less than 5 seconds.

I might use Hbase, but I want to use Impala.

I know that Impala does not support complex types.

What I want is for Impala to skipping complex types.

As a result of my checking, Impala skipping a complex type in a parqut format file.

 

How do I write a parquert format file to hdfs, hive, impala etc.?

Can I write a parquert file using Flume, Morphline, etc.?

 

My system's data collection flow is as follows.

Kafka -> Flume -> hdfs(avro file) -> hive