About Bildervic

Bildervic · ‎08-06-2019

hi @eMazarakis as far as I know Impala does not support HDFS Impersonation . (security reasons I guess ) means you can't delete hdfs files using hue user cheers !

Bildervic · ‎08-05-2019

hi @Harish19 check here : https://community.cloudera.com/t5/forums/forumtopicprintpage/board-id/Questions/message-id/13133/print-single-message/true/page/1

Bildervic · ‎08-05-2019

@Amritha as I have checked some Kudu manuals, Chrony is not fully tested with KUDU for network time synchronization. " Kudu releases are only tested with NTP. Other time synchronization providers like Chrony may or may not work." so you can either try to install NTP. or just try this tip I've found : "In order to use chrony for synchronization (for KUDU), chrony.conf must be configured with the rtcsync option" good luck

Bildervic · ‎08-02-2019

hello @GopiG , have you tried setting the executor's and the driver's params in spark-defaults.conf ? spark.driver.extraJavaOptions -Duser.timezone=UTC spark.executor.extraJavaOptions -Duser.timezone=UTC you can set the default time zone UTC or any example you want like GMT+8 etc... cheers.

Bildervic · ‎08-02-2019

hi @Amritha , when you're running the command ntpq -np , (ntp should be already installed I guess ) what does it give as result? greetings .

Bildervic · ‎05-06-2019

hi @MartinP , one way to do it is to build a new cluster in your CentOS platforme , then copy all your data to the new cluster as mentionned here : https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_admin_distcp_data_cluster_migrate.html just pay attention to the CDH distribution source and destination ,if they are supported for the migration or not . cheers,

Bildervic · ‎04-25-2019

hi @Harish19 , can you tell us what version of java,spark and CDH you're running ? otherwise try to add spark-assembly-XXX.jar to hdfs (copy from local to hdfs) then add that parameter : spark.yarn.jars hdfs://IP/spark/spark-assembly-XXX.jar to the file spark-defaults.conf and don't forget to restart yarn. I hope this will work for you ,

Bildervic · ‎04-24-2019

hi @DataMike , The Balancer role is normally added (by default) when the HDFS service is installed, so the "Balancer" Resides usally in your nameNode , but to make sure where it's assigned you can check HDFS->Instances then check the role Type , you'll find the role 'balancer' assigned to a host ( usally it's the nameNode ) . for your second question I guess it's better to use the nameNode just to maintain the architecture since we're talking about checking all the other DataNodes,moving blocks...

Bildervic · ‎04-24-2019

@bgooley , thank you for your feedback and your clear explanation , in fact the problem was resolved by removing the contents of /var/lib/hadoop-yarn/yarn-nm-recovery/ directory and then the Nodemanager role started successfully. the solution that I've found was from : https://community.cloudera.com/t5/Batch-Processing-and-Workflow/Yarn-NodeManager-fails-to-start-and-crashing-with-SIGBUS/m-p/66590#M3611

Bildervic · ‎04-23-2019

hello folks , the nodeManager has suddently stopped in a instance (while stille running for other nodes/intances ). so when I try to start/restart it -via cloudera manager - , an error is shown in the first step : Failed to start role. and I'm using CentOS release 6.10 (Final) please what do you suggest me to look or check in order to resolve this problem ? here's my stdout log : Tue Apr 23 10:18:56 PDT 2019 JAVA_HOME=/usr/java/jdk.1.8.0_144 using /usr/java/jdk.1.8.0_144 as JAVA_HOME using 5 as CDH_VERSION using /opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop-yarn as CDH_YARN_HOME using /opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop-mapreduce as CDH_MR2_HOME using /var/run/cloudera-scm-agent/process/23960-yarn-NODEMANAGER as CONF_DIR CONF_DIR=/var/run/cloudera-scm-agent/process/23960-yarn-NODEMANAGER CMF_CONF_DIR=/etc/cloudera-scm-agent # # A fatal error has been detected by the Java Runtime Environment: # # SIGBUS (0x7) at pc=0x00007f8c1fde51a1, pid=3004, tid=0x00007f8c4f44c700 # # JRE version: Java(TM) SE Runtime Environment (8.0_144-b01) (build 1.8.0_144-b01) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libleveldbjni-64-1-8170950501904951615.8+0x491a1] leveldb::ReadBlock(leveldb::RandomAccessFile*, leveldb::ReadOptions const&, leveldb::BlockHandle const&, leveldb::BlockContents*)+0x191 # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # and this is my log.out error : NodeManager Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner. AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl AsyncDispatcher Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher

Bildervic · ‎02-13-2019

try to ping on that ip first (127.0.0.1 or for my case I had to add another bridge ip) suits well with port 22 as mentioned in @nauseous the port 22 sould be enabled in iptables (firewall) , and so for ssh ( /etc/ssh/sshd_config) check if the port is still commented with # (that means still on default port 22 ) otherwise try to add a new port of your own .

Bildervic · ‎02-11-2019

very helpful , thank you for your efforts

Bildervic · ‎02-07-2019

for my case I was just working on cloudera VM , I had to configure the node ip , => ifconfig LOCAL_NODE_IP eth1:2 netmask XXXX after that pinging to that ip is going well thanks a lot

Online	Offline
Last Visited	‎08-21-2019 02:36 AM

Date Registered	‎01-31-2019 01:50 AM
Last Visited	‎08-21-2019 02:36 AM
Total Messages Posted	13
Total Kudos Received	1

Re: Kudu to HDFS data load timestamp issue.

Re: Which Host Should HDFS "Balancer" Role Reside?

Re: Failed to start role -YARN- NodeManager (n...

Re: How to properly delete a hive table through HU...

Re: Impala ODBC connection fails with ssl errors.

Re: KUDU time sync error

Re: Kudu to HDFS data load timestamp issue.

Re: KUDU time sync error

Re: Migrating OS distribution

Re: can't access pyspark shell

Re: Which Host Should HDFS "Balancer" Role Reside?

Re: Failed to start role -YARN- NodeManager (n...

Failed to start role -YARN- NodeManager (node)

Re: unable to Ssh using putty from windows to clou...

Re: Tutorial exercise 3 explained in details

Re: data nodes can't reach name node