Hive Batch Queries (Hive on Tez) and Concurrency

pgood · ‎08-08-2018

Hi, We have started using HDP 3.0 and we are connecting to hive through beeline but we are encountering concurrency issues. All sessions are using the hive user to connect and we are finding that when a second session tries to execute a query it remains in a pending status until the second session is closed. Any ideas where to start looking to determine why two or more sessions using the same user cannot execute queries simultaneously? Here is the configuration details of our HDP 3.0 cluster: Number of nodes 5 (2 master, 3 data) 16GB memory on each node 12GB memory for YARN containers per data node Maximum container size memory is set to 12GB to satisfy Hive LLAP requirements Maximum container size vcores is set to 3 Thanks Paul

pgood · ‎06-21-2018

Thanks Bryan, I am providing a schema to the CSVReader and it was simple as using the field name defined in the schema in the Message Key Field property. Cheers, Paul

pgood · ‎06-21-2018

Hi, I am using the PublishKafkaRecord processor to write messages to a Kafka topic. I am using the CSVReader for the Record Reader property and the AvroRecordSetWriter for the Record Writer property. I would like to set the message key of each message written to Kafka using one of the fields of the csv file by setting the Message Key Field property (see the image below). Please let me know if this is possible or an alternative approach where it can be set on a per record basis? processor.png Many thanks, Paul

Online	Offline
Last Visited	‎07-22-2019 01:39 PM

Date Registered	‎06-21-2018 01:01 PM
Last Visited	‎07-22-2019 01:39 PM
Total Messages Posted	5

Hive Batch Queries (Hive on Tez) and Concurrency

Re: How can you set the Message Key Field property...

How can you set the Message Key Field property in ...