Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to automatically use the schéma inferred by databriks/csv as catalog for Hbase?

Highlighted

How to automatically use the schéma inferred by databriks/csv as catalog for Hbase?

Rising Star

hi,

I'm trying to ingest csv files with variable (and unpredictable) schéma to Hbase.

Based on @Binu Mathew advice, I started to use spark with Databricks. So far I can infer the schema and read the file into a data frame.

I'm trying to write the data frame to a Hbase table, while using the inferred schéma as catalog. But I could not find any way to transform the schéma (df.dtypes) into a valid catalog.

Has anyone any advices or link to advice on how to proceed ?

Thanks

Christophe