hadoop

gvbnn — Thu, 06 Feb 2020 08:52:35 GMT

while i executing the program i getting these error can anyone help me with that

[cloudera@quickstart ~]$ spark-shell --master yarn-client
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/flume-ng/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/parquet/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/avro/avro-tools-1.7.6-cdh5.13.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.6.0
/_/

Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67)
Type in expressions to have them evaluated.
Type :help for more information.
20/02/05 22:03:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20/02/05 22:03:42 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
Spark context available as sc (master = yarn-client, app id = application_1580968178673_0001).
SQL context available as sqlContext.

Re: hadoop

jsensharma — Thu, 06 Feb 2020 06:24:25 GMT

@gvbnn

Those are just WARNING messages. Where do you see the errors?

Can you please share more details if you are noticing any error?

Re: hadoop

jsensharma — Thu, 06 Feb 2020 06:33:49 GMT

@gvbnn
Those are just warnings you can ignore them ... it should not be causing Job failure.

As we also see that the application_id "application_1580968178673_0001" is also generated ...
So you should be able to check the status of your Yarn application in ResourceManagr UI.

http://$RM_ADDRESS:8088/cluster/apps

If your cluster has enough resources then you should see the progress as well for your application_id ...

question hadoop in Support Questions

hadoop

Re: hadoop

Re: hadoop