Member since
09-30-2019
27
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5535 | 07-14-2022 07:33 AM | |
20482 | 08-29-2021 11:56 PM |
07-22-2022
06:05 AM
Hi All, I have two cases : Case A : i have a kerberised production cluster i want to copy files from this linux box to gcp storage ( non kerberised). Currently performing manully : downloading linux files to local system using winscp and uploading to google cloud storage if can be done using Distcp provinding steps will be helpful Case B : from the same kerberised cluster want to copy data of HDFS to Google cloud storage( non kerberised) Thanks, Syed.
... View more
Labels:
- Labels:
-
Cloudera Data Platform (CDP)
-
HDFS
07-14-2022
07:33 AM
1 Kudo
Hi Vidya , Issue has been resolved by setting path : export PATH=/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin:/usr/bin/python2:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/anaconda3/bin:/opt/anaconda3/condabin
... View more
06-28-2022
05:37 AM
Hi, Me sharing below line from bashrc of a particular user # .bashrc export PYSPARK_DRIVER_PYTHON="jupyter" export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export SPARK_HOME="/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/spark/" export PATH="$PATH:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/" # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # >>> conda initialize >>> # !! Contents within this block are managed by 'conda init' !! __conda_setup="$('/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" if [ $? -eq 0 ]; then eval "$__conda_setup" else if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then . "/opt/anaconda3/etc/profile.d/conda.sh" else export PATH="/opt/anaconda3/bin:$PATH" export PATH="/opt/anaconda3/condabin:$PATH" fi fi unset __conda_setup # Uncomment the following line if you don't like systemctl's auto-paging feature: # export SYSTEMD_PAGER= # User specific aliases and functions alias python=/usr/bin/python2 export PYSPARK_PYTHON=/usr/src/Python-3.7.0/python but still im getting same error Thanks, Syed
... View more
06-23-2022
09:08 AM
Python 2.7.5 is already installed and multiple version are available tried to set path but not working , any alternate solution.. .!!!
... View more
06-22-2022
08:11 AM
How to fix it permanently that default to python2.7.5 and impala shell to be work ?
... View more
06-22-2022
07:43 AM
Thanks for the solution , every time i have to come in venv and run impala shell ? or it will work without venv !!
... View more
06-22-2022
01:40 AM
Hi , im unable to connect impala shell ( kerberised cluster ) with below: command on Edge node impala-shell -i hostname.abc.com -d default -k case scenario : we have two Edge node A and B , able to connect on edgenode A and unable to connect on edgenode B with same credentials. checked with different user is it also same as working on Edge node A and edgenode B . observed that both have different python version A has 2.7.5 as default and B has 3.7.0 is this the cause of connection , if yes how to rectify. ERROR screenshot shared.
... View more
Labels:
01-19-2022
11:24 PM
Hi Team , We have upgraded from CDH to CDP 7.4.4, Atlas has been installed and Navigator also running parallely . Is it safe to remove and clean up databases of cloudera navigator and from CM ? Please share the steps involved. Thanks, Syed
... View more
Labels:
01-05-2022
01:34 AM
Hi Mszurap , thanks for the reply . Impala query profile? --showing same error its not a dynamic partitioning and there is only one insert operation mentioned in query. CM > Impala > Queries page-- showing same error no multiple inserts Thanks, Syed
... View more
01-05-2022
12:33 AM
Hi Team , we got error below during load : Error(s) moving partition files. First error (of 1) was: Hdfs op (RENAME hdfs://abc/app/_impala_insert_staging/1e49d89e9dec169b_276b0d4500000000/.1e49d89e9dec169b-276b0d4500000015_1583901315_dir/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt TO hdfs://abc/app/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt) failed, error was: hdfs://abc/app/_impala_insert_staging/1e49d89e9dec169b_276b0d4500000000/.1e49d89e9dec169b-276b0d4500000015_1583901315_dir/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt Error(5): Input/output error
... View more
Labels:
- Labels:
-
Apache Impala
-
HDFS