Skip to main content
PetiaLeshiy
PetiaLeshiy
Silver ⭐️⭐️
September 20, 2023
Question

How not show null fields in hive(Hue, beeline)?

I'm new in hive so it can be very stupid question) In our project we have very difficult model that contains a lot of arrays and nested fields. This model describes interaction with other systems so its filling can be very different. It is something like logs. We save this data in Hadoop using external table and orc files. It is a very big struct field. Apache ORC Tools is used by us for creating orc files and making orc schema. We don't have problem with writing, but when we read data we get a lot of "null" values. I understand that reasons of it are difficult model and different filled fields. But output in beeline or Hue is unreadable:

...{""id"":""mqqqnc"",""value"":[{""Ipv4Value"":null,""StringValue"":{""value"":""99""}}]},{""id"":""eee"",""value"":[{""Ipv4Value"":null,""StringValue"":{""value"":""254103""}}]},{""id"":""uli"",""value"":[{""Ipv4Value"":null,""StringValue"":{""value"":""asfsfsd""}} ...

This example is a little part of our struct field but we have three null field.

Is it possible to not show fields with null value? Thak everyone for help!!!

5 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
September 20, 2023

@PetiaLeshiy, Welcome to our community! To help you get the best possible answer, I have tagged in our Hive experts @asish @smruti @tjangid  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
asish
Gold ⭐️⭐️⭐️
asish
Gold ⭐️⭐️⭐️
September 20, 2023

@PetiaLeshiy If I understand correctly, you want to dispaly NOT NULL

 

You can use SELECT * FROM table_name WHERE column_name IS NOT NULL;

Translate
    PetiaLeshiy
    PetiaLeshiyAuthor
    Silver ⭐️⭐️
    September 25, 2023

    Thx for your answer, but i try to find a solution that can drop nested null-field in struct in the output. And I want that happens autmomatically)

    Translate
    smruti
    Gold ⭐️⭐️⭐️
    smruti
    Gold ⭐️⭐️⭐️
    September 20, 2023

    @PetiaLeshiy Adding to @asish 's comment, as its a struct column, we could write the query something like this:

    SELECT * FROM TABLE_NAME LATERAL VIEW explode(struct_col_name.list_name) exploded_column AS xyz WHERE xyz IS NOT NULL;

    You may make changes where required.

    Translate
    PetiaLeshiy
    PetiaLeshiyAuthor
    Silver ⭐️⭐️
    September 25, 2023

    Thx for your answer, but I have problem that i don't know what fields can be null( my query will have a different columns in WHERE statment  

    Translate
    Terms & ConditionsAccessibility statement