Hi,   -> Can you check user has sudo/root permissions on these hosts? -> Can you do agent hard restart on these hosts and try? -> recheck private key is valid and correct one.   Regards, Chethan YM ... View more

Hi,   Can you create a separate user in the database for each service and retry? You should NOT use the root user for all databases.   Refer below doc: https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_extrnl_pstgrs.html#cmig_topic_5_6_2   Regards, Chethan YM ... View more

Hi,   Where you are redirecting the output to? for csv file? What is the output from impala-shell terminal if you do not redirect the output?   It looks like caused by the csv module when impala is using it to export the data.       # csv.writer expects a file handle to the input.     # cStringIO is used as the temporary buffer.     temp_buffer = StringIO()     writer = csv.writer(temp_buffer, delimiter=self.field_delim,                         lineterminator='\n', quoting=csv.QUOTE_MINIMAL)     writer.writerows(rows)   Seems to be we cannot change this since it is needs to modified at code level.   https://github.com/apache/impala/blob/014c455aaaa38010ae706228f7b439c080c0bc7d/shell/shell_output.py...   Regards, Chethan YM   ... View more

Hi,   -> Can you try with  -Doraoop.timestamp.string=false property? -> Can you try with sqoop type mapping parameters as per below doc and see? https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_java_sql_timestamp:~:text=on%20all%20databases.-,7.2.8.%C2%A0Controlling%20type%20mapping,-Sqoop%20is%20preconfigured   Regards, Chethan YM   ... View more

Hi, > Add the below configuration in HDFS from CM UI: CM -> hdfs -> Configuration     hadoop.proxyuser.hive.hosts = * hadoop.proxyuser.hive.groups = *     >  If the above is already set, please disable the DB notification API auth.   In Hive and Hive_On_Tez Service: In Hive Service Advanced Configuration hive-site safety value:     Name: hive.metastore.event.db.notification.api.auth Value: false   Restart stale configs and check if it works.   Regards, Chethan YM   ... View more

Hi,   It looks like known- OPSAPS-60161 unresolved issue. Can you disable the canary health check? https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ht_hive_metastore_server.html#concept_6qo_fpn_yk   Regards, Chethan YM ... View more

Hi,   > As per the document it is service down time, So i think it is complete impala service down time. (However I haven't seen the issue on live) > No metrics/graph to check "  inc_stats_size" > If 1GB is insufficient, Try to use "compute stats" instead of  "compute incremental stats"   Regards, Chethan YM ... View more

Hi @yassan ,   I would like to let you know that, the default value on the flag(inc_stats_size_limit_bytes) is set to 200 MB, as a safety check to prevent Impala from hitting the maximum limit for the table metadata.   Whereas, the error reported usually serves as an indication that 'COMPUTE INCREMENTAL STATS' should not be used on the particular table and consider spitting the table thereby, using regular 'COMPUTE STATS'   statement if possible.     However, incase if you are not able to use the 'Compute Stats' statement then you could try to increase the default limit on the flag(inc_stats_size_limit_bytes) where, it should be set less than 1 GB limit and the value is measured in bytes.   Below is the seteps:   1. CM > Impala Service > Configuration > Search "Impala Command Line Argument Advanced Configuration Snippet (Safety Valve)" 2. Add --inc_stats_size_limit_bytes=     #####Please note that the above value is in bytes.   For example, if you want to set 400 Mb, please input 419430400(400*1024*1024). 3. Please save and restart Impala service.   Note: If I answered your question please give a thumbs up and Accept it as a solution.   Regards, Chethan YM   ... View more

Hi,   The alert message does not give more information to check, review the HMS and CM service monitor logs related to this issue and provide the stack-traces.   Regards, Chethan YM     ... View more

Hi,   As per the below Git, PauseTransitRunnable is the runnable which is scheduled to run at the configured interval, it checks all bundles. to see if they should be paused, un-paused or started.   https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/PauseTransitService.java#:~:text=PauseTransitRunnable%20is%20the%20runnable%20which%20is%20scheduled%20to%20run%20at%20the%20configured%20interval%2C%20it%20checks%20all%20bundles   It also released the lock, Are you running this sqoop-import from oozie? If yes try to rerun outside of oozie and check if it still get stucks and review the corresponding NM and RM logs if anything interrupting.   Regards, Chethan YM ... View more

Hi,   It looks like waiting for inserting the data, It may get finish after few minutes. Is it worked or still hangs for hours?Create table when we upload CSV file usually takes more time.   Regards, Chethan YM ... View more

Hi,   Below is the suspected causes for this issue: https://issues.apache.org/jira/browse/YARN-3055 https://issues.apache.org/jira/browse/YARN-2964   Yes, You can set that parameter at workflow level and test.   Regards, Chethan YM     ... View more

Hi @coco    Can you follow the below steps in Hue, if you are running the job from Hue.   1. Login to Hue 2. Go to Workflows -> Editors -> Workflows 3. Open the Workflow to edit.   4. On the left hand pane, Click 'Properties'   5. Under section 'Hadoop Job Properties', in the Name box enter 'mapreduce.job.complete.cancel.delegation.tokens' and in value enter 'true'.   6. Save the workflow and submit.     If you are running from terminal add the above property in configurations section then rerun the workflow and see if it helps.   If this works please accept it as a solution.   Regards, Chethan YM ... View more

Hi @vz    If both columns are in string i think you can use concat or concat_ws, Can you check below articles and see if this helps? https://community.cloudera.com/t5/Support-Questions/HIVE-Concatenates-all-columns-easily/td-p/180208 https://blog.fearcat.in/a?ID=01600-32e80587-5a71-411e-835b-ed905cb1b61a https://stackoverflow.com/questions/51211278/concatenate-multiple-columns-into-one-in-hive   Note: If my reply answers your question please give a thumbs up and accept it as a solution.   Regards, Chethan YM   ... View more

Hi,   Please review the below documentation: https://docs.cloudera.com/HDPDocuments/DAS/DAS-1.4.5/index.html   As per this it looks like it only works on Hive and need  PostgreSQL db.   Regards, Chethan YM ... View more

Hi,   > Running workflow for more than 7 days means, does it run entire 7 days all the time and fails? Can you provide the script you are running? > Is the shell script works outside of oozie without issues? > Please provide the complete error stack trace you are seeing.   Regards, Chethan YM ... View more

Hi,   Yes,  Hive metastore is a  component that stores all the structure information(metadata) of objects like tables and partitions in the warehouse including column and column type information etc...   Regards, Chethan YM   Note: If this answered your question please accept the reply as a solution. ... View more

Hi,   I do not think we have such configuration to validate the data, We need to ensure that data matches with the table that we have created.   Regards, Chethan YM ... View more

> Below is the document which has some more details on the same: https://impala.apache.org/docs/build/html/topics/impala_upsert.html   > Please let us know what is your concerns. ... View more

Hi @PNW    If you want to disable the automatic restart of cloudera services v isit the configuration page of the service you'd like to manage, then search for "Automatically Restart Process". You should see this option for each role within the service, then uncheck it.   Regards, Chethan YM ... View more

Hello Syed,   Can you try the alternatives command to change the python version? Check the below link. https://medium.com/coderlist/how-to-change-default-python-version-on-linux-fedora-28-c22da18bdd6   Regards, Chethan YM ... View more

Hi,   Attached screenshot does not give more details, Do check application logs, HS2 logs and metastore logs to find out the more information.  Check if you have required resources in the cluster to run the query. Check if any JVM pauses due to out of memory. Check if the tables involved in this job is not using by anyone for writing etc...   Try to restart Hive and rerun the job again and check.   Regards, Chethan YM ... View more

Hello,   You need to enable trace level logging for ODBC driver and should check the logs after reproducing the issue. Check if you have any socket timeout errors in the trace logs if yes refer the below doc to set the socket timeout. https://docs.cloudera.com/documentation/other/connectors/impala-odbc/2-6-5/Cloudera-ODBC-Driver-for-Impala-Install-Guide.pdf   Regards, Chethan YM     ... View more