There is a workaround to solve this. Is not a definitive solutions but it can help: The final result would be like this:  Firstly, I created a table like your example (I used ";" as separator): insert overwrite t_1 select 'Asia' as cont,'Japan;China;Singapore;' Country_list union select 'Europe' as cont,'UK;Spain;Italy;German;Norway;' Country_list After, I created a external table. It must be stored as textfile: CREATE EXTERNAL TABLE IF NOT EXISTS t_transpose ( field_transpose string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ";" STORED AS TEXTFILE; Then insert on this table like this: insert overwrite t_transpose select REGEXP_REPLACE(Country_list, ';', concat("|", cont, '\n' ) ) as transpose from t_1; After you can select like in my example before:   select split_part(field_transpose,"|",1), split_part(field_transpose,"|",2) from t_transpose; Ps: The final result could have some blank lines, just filter/ignore it. I also put one more ";" in the line comparing with the example informed. ... View more

jdbc:hive2://<host>:10000 Error: Could not open client transport with JDBC Uri: .... Invalid status 21 If i use just 'beeline' command, i am able to connect to hive. However if i use below command, i am getting exception. beeline -u jdbc:hive2://<host>:10000/default -n user -p pwd 23/10/23 23:22:31 [main]: WARN transport.TSaslTransport: Could not send failure response org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset 23/10/23 23:22:31 [main]: WARN jdbc.HiveConnection: Failed to connect to host:10000 log4j:ERROR Could not create an Appender. Reported error follows. java.lang.ClassCastException: org.apache.log4j.ConsoleAppender cannot be cast to com.cloudera.hive.jdbc42.internal.apache.log4j.Appender at com.cloudera.hive.jdbc42.internal.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:248) at com.cloudera.hive.jdbc42.internal.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176) log4j:WARN No appenders could be found for logger (com.cloudera.hive.jdbc42.internal.apache.thrift.transport.TSaslTransport). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Unknown HS2 problem when communicating with Thrift server. Error: Could not open client transport with JDBC Uri: jdbc:hive2://host:10000/default: Invalid status 21 Also, could not send response: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset (state=08S01,code=0)   ... View more

Check the possibility of using hive managed table.  As part of hive managed tables, you won't require separate merge job , as hive compaction takes care by default if compaction is enabled.  You can access managed tables through HWC (Hive Warehouse Connector) from Spark.    ... View more