Hi @Iga21207 Please find below the definition of parameters you have mentioned. We don't think that tweaking these parameters will cause much impact on the loading of metadata to impala daemon from catalog. ================================================================== catalog_max_lock_skipped_topic_updates, default value is 3: Maximum number of topic updates skipped for a table due to lock contention in catalogd after which it must be added to the topic the update log. This limit only applies to distinct lock operations which block the topic update thread. topic_update_tbl_max_wait_time_ms, default value is 120000: Maximum time (in milliseconds) catalog's topic update thread will wait to acquire lock on table. If the topic update thread cannot acquire a table lock it skips the table from that topic update and processes the table in the next update. However to prevent starvation it only skips the table catalog_max_lock_skipped_topic_updates many times. After that limit is hit, topic thread block until it acquires the table lock. A value of 0 disables the timeout based locking which means topic update thread will always block until table lock is acquired. max_wait_time_for_sync_ddl_s, default value is 0, Maximum time (in seconds) until which a sync ddl operation will wait for the updated tables to be the added to the catalog topic. A value of 0 means sync ddl operation will wait as long as necessary until the update is propogated to all the coordinators. This flag only takes effect when topic_update_tbl_max_wait_time_ms is enabled. A value greater than 0 means catalogd will wait until that number of seconds before throwing an error indicating that not all the coordinators might have applied the changes caused due to the ddl. ================================================================== However, you may increase the num_metadata_loading_threads this determines how much parallelism Impala devotes to loading metadata in the background. The default is 64. You might increase this value for systems with huge numbers of databases, tables, or partitions. NOTE: Increasing this value will cause more CPU utilization.
... View more
