Member since
07-25-2022
3
Posts
0
Kudos Received
0
Solutions
02-27-2024
11:48 AM
1 Kudo
Hi @Iga21207 Please find below the definition of parameters you have mentioned. We don't think that tweaking these parameters will cause much impact on the loading of metadata to impala daemon from catalog. ================================================================== catalog_max_lock_skipped_topic_updates, default value is 3: Maximum number of topic updates skipped for a table due to lock contention in catalogd after which it must be added to the topic the update log. This limit only applies to distinct lock operations which block the topic update thread. topic_update_tbl_max_wait_time_ms, default value is 120000: Maximum time (in milliseconds) catalog's topic update thread will wait to acquire lock on table. If the topic update thread cannot acquire a table lock it skips the table from that topic update and processes the table in the next update. However to prevent starvation it only skips the table catalog_max_lock_skipped_topic_updates many times. After that limit is hit, topic thread block until it acquires the table lock. A value of 0 disables the timeout based locking which means topic update thread will always block until table lock is acquired. max_wait_time_for_sync_ddl_s, default value is 0, Maximum time (in seconds) until which a sync ddl operation will wait for the updated tables to be the added to the catalog topic. A value of 0 means sync ddl operation will wait as long as necessary until the update is propogated to all the coordinators. This flag only takes effect when topic_update_tbl_max_wait_time_ms is enabled. A value greater than 0 means catalogd will wait until that number of seconds before throwing an error indicating that not all the coordinators might have applied the changes caused due to the ddl. ================================================================== However, you may increase the num_metadata_loading_threads this determines how much parallelism Impala devotes to loading metadata in the background. The default is 64. You might increase this value for systems with huge numbers of databases, tables, or partitions. NOTE: Increasing this value will cause more CPU utilization.
... View more
09-05-2022
05:05 AM
1 Kudo
Hi @Iga21207 , So how it works in catalod is when you run any refresh commands then that is executed sequentially and once that is completed then it goes to next one. It doesn't run in parallel as per the catalogd which is a single threaded operation. There is a lock that catalogd thread creates on class getCatalogObjects(). So when you are refreshing(means they have not completed yet sequentially) and after that when the new request came in then the Catalog threw the error on that table as it can't get the lock because the lock is already there on previous table on which the refresh command was running. Not sure on your CDH version, This may resolved in Higher version of CDP/CDH. Note: If i answered your question please give a thumbs up and accept it as a solution. Regards, Chethan YM
... View more
07-25-2022
02:47 AM
There is a problem with entering a host to the maintenance state using cloudera manager. The problem is related to replication of some blocks. The most files in hdfs have a factor replication of 3, but a few of them have changed at 2. When maintenance is starting the first of the blocks with replication factor of 2 trying to replicate to another DataNode and can't do that. The value dfs.namenode.maintenance.replication.min is 1. In the NameNode log file there is a message (it repeats): INFO BlockStateChange: Block: blk_2355920052_1283317658, Expected Replicas: 2, live replicas: 0, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 0, maintenance replicas: 1, live entering maintenance replicas: 1, excess replicas: 1, Is Open File: false, Datanodes having this block: ... ... , Current Datanode: ..., Is current datanode decommissioning: false, Is current datanode entering maintenance: true and simultaneously the fsck command by that block (hdfs fsck / -blockId blk_2355920052) shows: No. of Expected Replicas: 2 No. of live Replica: 1 No. of excess Replica: 0 No. of stale Replica: 0 No. of decommissioned Replica: 0 No. of decommissioning Replica: 0 No. of corrupt Replica: 0 Block replica on dnanode ... is HEALTHY Block replica on dnanode ... is HEALTHY On the one hand NameNode shows that live replicas: 0 and excess replicas: 1 and on the other hand the fsck command shows that live Replica: 1 and excess Replica: 0. Why does the problem is occur? Does anyone know how to deal with this problem? Thank you in advance! Cloudera manager shows this messages:
... View more
Labels:
- Labels:
-
Apache Hadoop
-
HDFS