About syedshakir

syedshakir · ‎07-22-2022

Hi All, I have two cases : Case A : i have a kerberised production cluster i want to copy files from this linux box to gcp storage ( non kerberised). Currently performing manully : downloading linux files to local system using winscp and uploading to google cloud storage if can be done using Distcp provinding steps will be helpful Case B : from the same kerberised cluster want to copy data of HDFS to Google cloud storage( non kerberised) Thanks, Syed.

syedshakir · ‎07-14-2022

Hi Vidya , Issue has been resolved by setting path : export PATH=/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin:/usr/bin/python2:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/anaconda3/bin:/opt/anaconda3/condabin

syedshakir · ‎06-28-2022

Hi, Me sharing below line from bashrc of a particular user # .bashrc export PYSPARK_DRIVER_PYTHON="jupyter" export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export SPARK_HOME="/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/spark/" export PATH="$PATH:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/" # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # >>> conda initialize >>> # !! Contents within this block are managed by 'conda init' !! __conda_setup="$('/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" if [ $? -eq 0 ]; then eval "$__conda_setup" else if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then . "/opt/anaconda3/etc/profile.d/conda.sh" else export PATH="/opt/anaconda3/bin:$PATH" export PATH="/opt/anaconda3/condabin:$PATH" fi fi unset __conda_setup # Uncomment the following line if you don't like systemctl's auto-paging feature: # export SYSTEMD_PAGER= # User specific aliases and functions alias python=/usr/bin/python2 export PYSPARK_PYTHON=/usr/src/Python-3.7.0/python but still im getting same error Thanks, Syed

syedshakir · ‎06-23-2022

Python 2.7.5 is already installed and multiple version are available tried to set path but not working , any alternate solution.. .!!!

syedshakir · ‎06-22-2022

How to fix it permanently that default to python2.7.5 and impala shell to be work ?

syedshakir · ‎06-22-2022

Thanks for the solution , every time i have to come in venv and run impala shell ? or it will work without venv !!

syedshakir · ‎06-22-2022

Hi , im unable to connect impala shell ( kerberised cluster ) with below: command on Edge node impala-shell -i hostname.abc.com -d default -k case scenario : we have two Edge node A and B , able to connect on edgenode A and unable to connect on edgenode B with same credentials. checked with different user is it also same as working on Edge node A and edgenode B . observed that both have different python version A has 2.7.5 as default and B has 3.7.0 is this the cause of connection , if yes how to rectify. ERROR screenshot shared.

syedshakir · ‎01-19-2022

Hi Team , We have upgraded from CDH to CDP 7.4.4, Atlas has been installed and Navigator also running parallely . Is it safe to remove and clean up databases of cloudera navigator and from CM ? Please share the steps involved. Thanks, Syed

syedshakir · ‎01-05-2022

Hi Mszurap , thanks for the reply . Impala query profile? --showing same error its not a dynamic partitioning and there is only one insert operation mentioned in query. CM > Impala > Queries page-- showing same error no multiple inserts Thanks, Syed

syedshakir · ‎01-05-2022

Hi Team , we got error below during load : Error(s) moving partition files. First error (of 1) was: Hdfs op (RENAME hdfs://abc/app/_impala_insert_staging/1e49d89e9dec169b_276b0d4500000000/.1e49d89e9dec169b-276b0d4500000015_1583901315_dir/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt TO hdfs://abc/app/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt) failed, error was: hdfs://abc/app/_impala_insert_staging/1e49d89e9dec169b_276b0d4500000000/.1e49d89e9dec169b-276b0d4500000015_1583901315_dir/organizationgroupid=INTEA613%2F/batch_id=20220105025807/1e49d89e9dec169b-276b0d4500000015_814403073_data.0.txt Error(5): Input/output error

syedshakir · ‎12-13-2021

Hi Team , Unable to change interpreter by user , need to enable multiple user support .one admin user to be activate. Error attached on interpreter

syedshakir · ‎08-29-2021

Hi, Issue resolved by increasing number of conection and session timeout as a workaround. Actually need to use dedicated disk for DB's. Thanks, Syed

syedshakir · ‎08-04-2021

Hi, yes its happening at hue browser only

syedshakir · ‎07-26-2021

Hi, No its not acquiring lock , result was empty set in hue DB. Thanks, syed

syedshakir · ‎07-19-2021

Hi, Im also facing same issue , increased connections numbers to HUE DB, still issue exits. Thanks, SS

syedshakir · ‎12-09-2020

After making changes as ssh-copy-id -i /home/oozie/.ssh/id_rsa.pub cloudera@example.local.com getting permission issue in oozie workflow oozi-W@ssh-34f1] Error starting action [ssh-34f1]. ErrorType [NON_TRANSIENT], ErrorCode [AUTH_FAILED], Message [AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 abc.com mkdir -p oozie-oozi/0000000-201209133111712-oozie-oozi-W/ssh-34f1--ssh/ ] | ErrorStream: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive).

syedshakir · ‎08-06-2020

CDH version is 5.15.1 impala version : impalad version 2.12.0 audit logs in which location ? provide steps please. impala cant restart as its a prod . Thanks, syed

syedshakir · ‎08-06-2020

Hi Folks , Running workflow with impala shell. got error ERROR: Disk I/O error: Failed to open HDFS file Disabled invalidate metadata hdfs:///batch_id=NWMISSPAYWRADJ/aa4fbef1c0bb3fd5-85012b8600000018_1953707135_data.0.parq Error(2): No such file or directory Root cause: RemoteException: File does not exist /app/abc/footable/tablename/account/orgid=abcd/batch_id=NWMISSPAYWRADJ/aa4fbef1c0bb3fd5-85012b8600000018_1953707135_data.0.parq at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66) at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2157) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2127) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2040) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:583) at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:94) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:377) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2275) Thanks,

syedshakir · ‎06-24-2020

Hi Paras, i have existing database but in article it s for new database . my question how to apply utf8 to existing database Thanks, Syed

syedshakir · ‎05-26-2020

Hi all, im getting configuration issues recently changed thrift server timeout and limit beewax to 10000

syedshakir · ‎03-26-2020

i have tick mark the three option to the particular, still issue persist, user unable to kill jobs

syedshakir · ‎03-01-2020

Hi , im facing same issue , attached is the error screenshot

syedshakir · ‎02-21-2020

Hi Wang , i did not find any option like oozie.access , or provide the step by step process. thanks, syed

syedshakir · ‎12-08-2019

Hi , dashboard.access is already given but search.access is not visible on screen attached screenshot

syedshakir · ‎12-05-2019

Hi All, using CDH 5.15.1 security : kerberos enabled whenever i clicked on oozie workflow under my documents getting error 403 forbidden i want to edit oozie workflow

Online	Offline
Last Visited	‎10-17-2022 06:55 AM

Date Registered	‎09-30-2019 11:47 PM
Last Visited	‎10-17-2022 06:55 AM
Total Messages Posted	27
Total Kudos Received	1

Re: Impala shell error

Re: You can't execute queries until the end of the...

Copy HDFS data to GCP

Re: Impala shell error

Re: Impala shell error

Re: Impala shell error

Re: Impala shell error

Re: Impala shell error

Impala shell error

Steps to remove/clean up Navigator

Re: Loading Error

Loading Error

zeppelin interpreter unable to change

Re: You can't execute queries until the end of the...

Re: You can't execute queries until the end of the...

Re: You can't execute queries until the end of the...

Re: You can't execute queries until the end of the...

oozie job getting suspended

Re: Failed to open HDFS file

Failed to open HDFS file

Re: How to resolve hue configuration

How to resolve hue configuration

Re: oozie workflow issue 402

Re: You can't execute queries until the end of the...

Re: oozie workflow issue 402

Re: oozie workflow issue 402

oozie workflow issue 402