Try this: CREATE external TABLE mytable ( col1 INT, col2 STRING, col3 STRING, col4 STRING, col5 INT, col6 STRING, col7 STRING ... ) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES ( "separatorChar" = ",", "quoteChar" = "\"" ) STORED AS TEXTFILE; It should work. ... View more

@Leopold I just checked. Your observation is correct. For external tables, it does not use a fetch task. In the logs, I see the following message: 2024-03-12 10:48:37,247 INFO org.apache.hadoop.hive.ql.optimizer.StatsOptimizer: [b226e7aa-9a42-4af3-b99b-be4a6592fb7f HiveServer2-Handler-Pool: Thread-31145]: Table t7 is external. Skip StatsOptimizer. But enabling  "hive.fetch.task.aggr=true" will help avoid the Reducer phase that is used for final aggregation. It will be a Map-only job. ... View more

@frbelotto I have not tried with pyhive, I think it requires additional modules if you want to connect using zookeeper quorum. But you could use jaydebeapi python module to connect to Hive3. It works for any type of connection string knox/ZK.  You would require Hive driver that you could download from here. An example on how to to make use of jaydebeapi module to connect to Hive: import jaydebeapi # Connection parameters jdbc_url = 'jdbc:hive2://knox.host:8443/default' # JDBC URL for HiveServer2 username = 'your-username' password = 'your-password' jar_file = '/path/to/hive-jdbc-driver.jar' # Path to the Hive JDBC driver JAR file # Establish connection to Hive conn = jaydebeapi.connect( 'org.apache.hive.jdbc.HiveDriver', jdbc_url, [username, password], jar_file ) # Create cursor cursor = conn.cursor() # Execute Hive query cursor.execute('SELECT * FROM hive_table') # Fetch results result = cursor.fetchall() # Close cursor and connection cursor.close() conn.close() ... View more