Member since
11-11-2019
636
Posts
34
Kudos Received
27
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 966 | 10-09-2025 12:29 AM | |
| 8367 | 02-19-2025 09:43 PM | |
| 2439 | 02-28-2023 09:32 PM | |
| 5074 | 02-27-2023 03:33 AM | |
| 26757 | 12-24-2022 05:56 AM |
09-13-2021
11:48 PM
Could you please attach hive on Tez logs. Please start Hive on Tez and wait till it crash completely and the provide the logs. Please attach the logs from below location: a. /var/logs/hive b. /var/run/cloudera-scm-agent/process/<hove on tez>/logs Please also try setting hive.server2.tez.initialize.default.sessions=false in CM and check.
... View more
09-09-2021
09:34 AM
Is the query failing or you are just noticing the errors in the logs? It seems to be ACID managed table. Please provide the hiveserver2 logs and the exact query fired.
... View more
09-09-2021
09:07 AM
DISTCP or import/export is not supported for ACID tables. You need to follow below mechanism: Distscp for ACID is not supported ,you have 2 approaches: Approach 1 ============= 1. Assuming that you have ACID in source and target clusters. 2. Create a external in source and target clusters. 3. Copy the data from ACID TO external in SOURCE CLUSTER INSERT into external select * from acid. 4. Perfrom distscp from source to target for external table. 5. Copy the data from external TO ACID IN SOURCE CLUSTER INSERT into acid select * from external. Approach 2 ========= Use DLM Refrence: https://community.cloudera.com/t5/Support-Questions/HIVE-ACID-table-Not-enough-history-available-for-0-x-Oldest/td-p/204551
... View more
09-09-2021
08:06 AM
I see you are getting belw issue java.io.IOException: Previous writer likely failed to write hdfs://sunny/tmp/hive/hive/_tez_session_dir/1b689cf2-9a2e-4afc-96a7-bdeef34ed887/hive. Failing because I am unlikely to write too. Have you copied Managed table data using distcp or import/export from other cluster ?
... View more
08-25-2021
01:07 AM
You need to provide ALL access to the User in the Ranger. Please check the ranger permission settings in Ranger UI
... View more
08-24-2021
02:06 AM
Is hive generating slow response? Is the query slow??
... View more
08-03-2021
03:36 AM
DOes the same issue happen when you run via beeline ? I researched ,I found this occurs only in hue.
... View more
08-03-2021
03:25 AM
Hi Megh,
Thanks for the logs.
I found a discrepancy in HDP logs,as per "Query counters on HDP Cluster.txt" ,the Mapper spawned is 64 but from the application_1627033595344_12159_hdp.log,I observe it is 127.
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 64 64 0 0 0 0
Reducer 2 ...... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
grep "Event:TASK_STARTED]: vertexName=Map 1" syslog_dag_1627033595344_12159_1 | wc -l
127
grep "Event:TASK_FINISHED]: vertexName=Map 1" syslog_dag_1627033595344_12159_1 | wc -l
127
Question: have you provided correct application log for HDP ? As I see a discrepancy.
Assuming you have provided correct set of logs for HDO,I compared between CDP and HDP,I see that input records for each maper is more in CDP then HDP that is the reason it took more tie in CDP.
I have collected samle taskID and INPUT_RECORDS_PROCESSED for each maper.
application_1627885418606_0016_cdp.log
======
each Mapper took close to 1 min to finish
taskId=task_1627885418606_0016_1_00_000059 INPUT_RECORDS_PROCESSED=2012469
taskId=task_1627885418606_0016_1_00_000061 INPUT_RECORDS_PROCESSED=2010369
taskId=task_1627885418606_0016_1_00_000026 INPUT_RECORDS_PROCESSED=2021105
application_1627033595344_12159_hdp.log
==========
taskId=task_1627033595344_12159_1_00_000062 INPUT_RECORDS_PROCESSED=1010250
taskId=task_1627033595344_12159_1_00_000126 INPUT_RECORDS_PROCESSED=376062
Thanks,
Asish
... View more
07-20-2021
01:06 AM
Can you check "show locks" if the table is acquired EXCLUSIVE lock ?
... View more
07-19-2021
10:26 AM
Hi Sai, Please follow https://community.cloudera.com/t5/Support-Questions/Does-hive-support-Photo-or-images-datatypes/td-p/221473 Copy the data to hdfs instead of loadpath and check, Thanks, Asish
... View more