Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Saving snappy compressed Avro data without compression?

Saving snappy compressed Avro data without compression?

I have a Snappy compressed avro data file that I imported through sqoop

 

1.png

 

save it uncompressed (Spark 1.6)

 

>>> sqlContext.getConf("spark.sql.avro.compression.codec","")
u''

>>> df = sqlContext.read.format("com.databricks.spark.avro").load("/user/cloudera/myavro/cat_avro3/")

>>> sqlContext.setConf("spark.sql.avro.compression.codec","uncompressed")
18/01/23 04:58:49 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.1.0-cdh5.12.0
18/01/23 04:58:49 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException

>>> df.write.format("com.databricks.spark.avro").save("/user/cloudera/myavro/uc1/")

 avro-tools shows that the codec is null but when I open the file in filebrowser, it shows Output rendered from compressed avro file

22.png

 

3.png

 

 

Which one is reliable?

 

thanks