I don't understand what the benefit of doing it this way is. As far as I know, when creating a table in Hive, a new entity of type hive_table is automatically created in Atlas. This happens automatically, in contrast to your manual approach. Am I misunderstanding something? Could you please explain it to me? ... View more

To troubleshoot the issue of ranger policies not getting synced, you can check the following log files in HDFS: 1. ranger_admin.log: This log file contains the logs related to the Ranger Admin service. It can be found in the Ranger Admin node at the location: `/var/log/ranger/ranger-admin`. 2. ranger_admin_audit.log: This log file contains the logs for auditing actions performed by Ranger Admin. It can be found at the same location as ranger_admin.log.= 3. hdfs.log: This log file contains the logs for HDFS operations. It can be found in the Hadoop log folder, which is usually located at: `/var/log/hadoop/hdfs` or `/var/log/hadoop-hdfs` ... View more

Thank you, my friend. A week ago, I read through your configurations in the official documentation and experimented with them. However, I encountered an error along the lines of 'class not found.' Currently, I've identified the root cause: I'm using HDP 3.1.0, which includes PySpark 2.3.2.3.1.0.0-78. Therefore, I upgraded it to PySpark 3, while still using the standalone-metastore-1.21.2.3.1.0.0-78-hive3.jar file by default. That's the reason why, when using the configuration, I encountered the 'class not found' error. Now, I've replaced that JAR file with hive-metastore-2.3.9.jar. Everything is working fine now. Once again, thank you, my friend. ... View more

To calculate the DataFrame size, you can use SizeEstimator class. ... View more

It works for me ... View more

Hi @sonnh  You can go ahead the raise the hdfs case by uploading all required logs like NM logs and DN logs. ... View more