Support Questions
Find answers, ask questions, and share your expertise

How to automatically use the schéma inferred by databriks/csv as catalog for Hbase?

Rising Star


I'm trying to ingest csv files with variable (and unpredictable) schéma to Hbase.

Based on @Binu Mathew advice, I started to use spark with Databricks. So far I can infer the schema and read the file into a data frame.

I'm trying to write the data frame to a Hbase table, while using the inferred schéma as catalog. But I could not find any way to transform the schéma (df.dtypes) into a valid catalog.

Has anyone any advices or link to advice on how to proceed ?