About Fawze

Fawze · ‎03-05-2017

Any new insights?

Fawze · ‎03-03-2017

i searched all the config files at the resource managers for spark and all of them directs the history spark server to the right one, where i can track the error of the redirection.

Fawze · ‎03-01-2017

only this job failed on this error, Since i have only 10G for ts so i have history for 3 days, I restarted the DN at that time and the job didn't failed since 3 days, but still curious to understand the cause and how to handle it next time

Fawze · ‎03-01-2017

No, In one RM it was but the other RM was not, i changed it manually in the spark default at the 2 RMs and kept if the spark client configuration through CM

Fawze · ‎03-01-2017

Tried both with no success Under /etc/spark/conf.cloudera.spark_on_yarn/spark-defaults.conf at the 2 RM i added also tried to add it using CM and still not directing to the right history server. spark.yarn.historyServer.address=http://avvr-ahc101.lpdomain.com:18088

Fawze · ‎02-28-2017

jdbc:impala://ropr-dhc001:21050/;UID=fawze Cloudera_ImpalaJDBC_2.5.5.1007.zip No erros but the queries are running without user.

Fawze · ‎02-28-2017

since the job failed the file will not be available and deleted, also in the error i see it's alerting on a file including the job id under the folder but don't find this, also tried fsck but the file isn't exists

Fawze · ‎02-28-2017

I have Spark Gateway on all DNs and the priephrals. de selecting and re selecting the MR will have no affect and will not have any client configurations changes needed to deployed, also trying de selecting it ad run client configuration deploy will not work since it's alerting on the missing MR. Is there any work around for this?

Fawze · ‎02-28-2017

YARN (MR2 Included) Service Spark (Service-Wide) YARN (MR2 Included) Missing required value: YARN (MR2 Included) Service Cann't find spark.yarn.historyServer.address at any DataNode /etc/spark/conf files

Fawze · ‎02-28-2017

Since i cann't catch the issue on time, i cann't know the host name that had the issue ... Can i conclude it from the block name or something like? is such issue can be investigated in retrospectivie without monitoring the job till it failed? I looked at all the DNs and all the disks are well distributed.

Online	Offline
Last Visited	‎10-19-2023 10:11 PM

Member Since	‎01-25-2017 01:09 PM
Last Visited	‎10-19-2023 10:11 PM
Posts	396
Kudos received	27

Cloudera Community

Re: How to make Yarn deploy resources to new added...

Re: Upgrade to CDH 6.0.x from 5.15

Re: How to define concrete resource consumption fo...

Re: Excution of the following command gives warnin...

Re: Excution of the following command gives warnin...

Re: Wrong spark history redirection for finished j...

Re: Wrong spark history redirection for finished j...

Re: Mapreduce failed on Could not deallocate conta...

Re: Wrong spark history redirection for finished j...

Re: Wrong spark history redirection for finished j...

Re: Impala queries through impala JDBC is running ...

Re: Mapreduce failed on Could not deallocate conta...

Re: Wrong spark history redirection for finished j...

Re: Wrong spark history redirection for finished j...

Re: Mapreduce failed on Could not deallocate conta...