Member since
10-01-2016
156
Posts
8
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4488 | 04-04-2019 09:41 PM | |
1568 | 06-04-2018 08:34 AM | |
720 | 05-23-2018 01:03 PM | |
1642 | 05-21-2018 07:12 AM | |
933 | 05-08-2018 10:48 AM |
03-25-2020
06:28 AM
Thank you very much. This is the one that satisfies me. Documents are expected to make clear and simple things, not complicated.
... View more
03-23-2020
01:39 AM
I know thank you but I still don't understand why we add same property (dfs.datanode.balance.max.concurrent.moves) on a different section DataNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml Balancer Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml although same property (dfs.datanode.balance.max.concurrent.moves) already exists in Cloudera Manager. Is CM supposed to refuse this addition?
... View more
03-22-2020
12:15 PM
Yes, you are right. I have not realized that. But if dfs.datanode.ec.reconstruction.xmits.weight is already in hdfs-site.xml why Cloudera document makes us add the same property for balancer and DataNode again, what is the point?
... View more
03-22-2020
02:49 AM
I am trying to rebalance hdfs with Cloudera Manager 6.3 with HDFS Balancer Document
It says add the same property dfs.datanode.balance.max.concurrent.moves into different section
DataNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
Balancer Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
But before adding the property I searched and saw dfs.datanode.balance.max.concurrent.moves was already there. Nevertheless, I did what the document says. After adding properties Cloudera Manager asked me to restart/redeploy stale configurations. Before restart, I saw totally different properties added.
I don't understand although we seem to add the same property why different properties are added to hdfs-site.xml?
... View more
Labels:
- Labels:
-
Cloudera Manager
-
HDFS
03-06-2020
03:36 AM
Thank you when I copy sigar-1.6.4.jar to /opt/cloudera/parcels/CDH/lib/spark/jars/ error disappeared. But deleting the metrics.properties file still a problem.
... View more
03-04-2020
11:51 PM
I am trying to follow https://github.com/ibm-research-ireland/sparkoscope Enabling Spark Optimization through Cross-stack Monitoring and Visualization. I can't find spark.properties configuration on Cloudera Manager or as a file in /etc/spark/conf. So I created a file named metric.properties and put metrics in it as following
executor.source.jvm.class=org.apache.spark.metrics.source.SigarSource
executor.sink.hdfs.class=org.apache.spark.metrics.sink.HDFSSink
executor.sink.hdfs.pollPeriod = 1
executor.sink.hdfs.dir = hdfs://cloudera.impketra.com:8020/custom-metrics
executor.sink.hdfs.unit = seconds
executor.sink.mqtt.class=org.apache.spark.metrics.sink.MQTTSink
executor.sink.mqtt.pollPeriod = 1
executor.sink.mqtt.host = cloudera.impketra.com
executor.sink.mqtt.port = 1883
executor.sink.mqtt.unit = seconds
But after a while ( I think after restarting Spark service) the metrics.properties file is deleted.
Additionally when I submit
spark-submit --class org.apache.spark.examples.SparkPi --master yarn /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/jars/spark-examples_2.11-2.4.0-cdh6.3.1.jar 10000
I get this ERROR
20/03/05 10:17:05 ERROR metrics.MetricsSystem: Sink class org.apache.spark.metrics.sink.MQTTSink cannot be instantiated
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:70)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:281)
at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.metrics.sink.MQTTSink
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:242)
at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:198)
at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:194)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:130)
at org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:194)
at org.apache.spark.metrics.MetricsSystem.start(MetricsSystem.scala:102)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:365)
at org.apache.spark.SparkEnv$.createExecutorEnv(SparkEnv.scala:201)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:221)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:71)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:70)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
... 4 more
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera Manager
02-28-2020
05:06 AM
In CDH 6.X I can't find Advanced spark2-metrics-properties in Spark config. Should I create manually?
... View more
10-28-2019
06:45 PM
Here are newly imported HDP 2.6.5 Sandbox spark-shell --master yarn [root@sandbox-hdp ~]# spark-shell --master yarn
SPARK_MAJOR_VERSION is set to 2, using Spark2
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at <a href="http://sandbox-hdp.hortonworks.com:4040" target="_blank">http://sandbox-hdp.hortonworks.com:4040</a>
Spark context available as 'sc' (master = yarn, app id = application_1572283124735_0001).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.3.0.2.6.5.0-292
/_/
Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_171)
Type in expressions to have them evaluated.
Type :help for more information.
scala>
... View more
10-27-2019
11:01 AM
Where did you paste log4j2.xml? I solved the problem by adding the following : -Dlog4j.configurationFile=/usr/hdp/current/flume-server/conf/log4j2.xml My whole command is: /usr/hdp/current/flume-server/bin/flume-ng agent --conf conf --conf-file /home/maria_dev/flume_uygulama_01/flumelogs.conf --name a1 -Dflume.root.logger=INFO,console -Dlog4j.configurationFile=/usr/hdp/current/flume-server/conf/log4j2.xml
... View more
10-27-2019
05:21 AM
Hi @Shelton No specific steps actually. I just open HDP 2.6.5 Sandbox, connect it via ssh then run spark-shell --master yarn Alternatively, I tried to start Spark on Zeppelin and examined the logs. ERROR was the same. Spark-shell has opened in local mode, no problem. But I can't start it yarn mode. But I managed to start a newly imported Sandbox.
... View more
10-26-2019
10:42 PM
Unfortunately, after a while, the same problem occurred again.
... View more
10-19-2019
04:18 AM
I tried to add kafka-clients-1.0.0.jar into /usr/hdp/2.6.5.0-292/kafka/libs folder but it is no use.
... View more
10-19-2019
02:45 AM
I deleted Sandbox and freshly imported it. It worked.
... View more
10-19-2019
01:14 AM
In HDP 2.6.5 I tried
[root@sandbox-hdp ~]# spark-shell --master yarn
But got following errors. It opens scala shell but cannot create spark session.
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
19/10/19 06:04:17 ERROR YarnClientSchedulerBackend: Yarn application has already exited with state FAILED!
19/10/19 06:04:17 ERROR TransportClient: Failed to send RPC 5080948039683175202 to /172.18.0.2:52542: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
19/10/19 06:04:17 ERROR YarnSchedulerBackend$YarnSchedulerEndpoint: Sending RequestExecutors(0,0,Map(),Set()) to AM was unsuccessful
java.io.IOException: Failed to send RPC 5080948039683175202 to /172.18.0.2:52542: java.nio.channels.ClosedChannelException
....
....
Caused by: java.nio.channels.ClosedChannelException
...
19/10/19 06:04:17 ERROR Utils: Uncaught exception in thread Yarn application state monitor
org.apache.spark.SparkException: Exception thrown in awaitResult:
...
Caused by: java.io.IOException: Failed to send RPC 5080948039683175202 to /172.18.0.2:52542: java.nio.channels.ClosedChannelException
...
19/10/19 06:04:17 ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalStateException: Spark context stopped while waiting for backend
...
console>:14: error: not found: value spark
import spark.implicits._
^
<console>:14: error: not found: value spark
import spark.sql
Then I looked at yarn logs:
19/10/19 06:21:35 INFO RMProxy: Connecting to ResourceManager at sandbox-hdp.hortonworks.com/172.18.0.2:8030
19/10/19 06:21:35 INFO YarnRMClient: Registering the ApplicationMaster
19/10/19 06:21:35 INFO YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead)
....
19/10/19 06:21:35 INFO RMProxy: Connecting to ResourceManager at sandbox-hdp.hortonworks.com/172.18.0.2:8030
19/10/19 06:21:35 INFO YarnRMClient: Registering the ApplicationMaster
19/10/19 06:21:35 INFO YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead)
... View more
Labels:
10-12-2019
10:49 PM
This (NULL issue) can be caused by data schema mismatch.
... View more
10-12-2019
10:18 PM
I explained how to manually set a parameter at runtime beeline. Before to do this you have to set sth on Ambari Hive settings. Please refer: https://community.cloudera.com/t5/Support-Questions/params-that-are-allowed-to-be-modified-at-runtime-beeline/m-p/280063/highlight/true#M208647 Just use hive.security.authorization.sqlstd.confwhitelist.append=mapreduce.job.reduces
... View more
10-12-2019
10:10 PM
3 Kudos
Go to Ambari -> Hive -> Configs -> Advanced -> Custom hive-site Click Add Property Insert followings into the opening window: hive.security.authorization.sqlstd.confwhitelist.append=mapred.compress.map.output After saving, restart hive services. Then connect to beeline and set your parameter. I experienced a similar problem for mapreduce.job.reduces parameter and it worked.
... View more
10-08-2019
08:19 AM
My solution was a similar: in command line as a root systemctl stop mysqld
systemctl set-environment MYSQLD_OPTS="--skip-grant-tables --skip-networking"
systemctl start mysqld
mysql -u root in mysql shell use mysql;
update user set authentication_string=PASSWORD("hadoop") where User='root';
flush privileges;
quit After this change, I got trouble hive and ranger due to connection error to MySQL. Then I had to change hive and ranger mysql database root user password from Ambari. It is ok now.
... View more
10-04-2019
12:51 PM
Thank you @Shelton . I have just deleted the line specified in error message in /c/Users/user/.ssh/known_hosts It worked. I had used HDP 2.6.4 before this one. I think there was already a key for the same hostname, once delete it i was able to connect via ssh. This time added new key that belongs to HDP 2.6.5 Sandbox. In this case, Should I accept your solution? Because you offer additional solutions that I have not verified.
... View more
10-04-2019
07:13 AM
I have newly imported HDP 2.6.5 Sandbox after removing the 2.6.4, and tried to connect via ssh. My host machine Windows 10. My command:
ssh root@sandbox-hdp.hortonworks.com -p 2222
The error output:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
SHA256:8F0VOvcJIA2FD3Ld90kv5FiD3270onl/+uDeDBNffQ0.
Please contact your system administrator.
Add correct host key in /c/Users/user/.ssh/known_hosts to get rid of this message.
Offending RSA key in /c/Users/user/.ssh/known_hosts:3
RSA host key for [sandbox-hdp.hortonworks.com]:2222 has changed and you have requested strict checking.
Host key verification failed.
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
07-31-2019
06:46 AM
Hi @Shu thank you. Adding spark-yarn_x.x.jar to maven repository solved the problem. But I have come across other errors. Anyway in here the problem was parsing the yarn and it is solved.
... View more
07-30-2019
09:34 AM
Hi @Shu I tried yarn-client and spark-yarn.jar But it can't pass the Could not parse Master URL: 'yarn' ERROR
... View more
07-26-2019
05:19 PM
I use IntelliJ IDE installed on my Windows 10 laptop and try to run spark job in yarn mode on my 5 node HDP 3.1.1 cluster. My codes: package p1
import org.apache.spark.sql.{SparkSession, functions => F}
import org.apache.log4j.{Logger, Level}
object SparkDeneme extends App {
Logger.getLogger("org").setLevel(Level.INFO)
val spark = SparkSession.builder()
.appName("SparkDeneme")
.master("yarn")
.config("spark.hadoop.fs.defaultFS","hdfs://node1.impektra.com:8020")
.config("spark.hadoop.yarn.resoursemanager.address","node1.impektra.com:8030")
.getOrCreate()
}
import spark.implicits._
val sc = spark.sparkContext
val dfFromList = sc.parallelize(List(1,2,3,4,5,6)).toDF("rakamlar")
// dfFromList.printSchema()
dfFromList.show() When I run get following error: 19/07/26 20:00:32 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Could not parse Master URL: 'yarn'
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2744)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:492)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2493)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:933)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:924)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:924)
at p1.SparkDeneme$.delayedEndpoint$p1$SparkDeneme$1(SparkDeneme.scala:17)
at p1.SparkDeneme$delayedInit$body.apply(SparkDeneme.scala:8)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at p1.SparkDeneme$.main(SparkDeneme.scala:8)
at p1.SparkDeneme.main(SparkDeneme.scala) I tried to get help from this tutorial Anyone who has succeeded to run Spark YARN mode in IntelliJ?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
05-27-2019
04:30 PM
I checked my tez directory [root@node2 ~]# ls -l /usr/hdp/3.1.0.0-78/tez/
total 0
drwxr-xr-x 2 tez hadoop 44 May 27 00:57 conf
lrwxrwxrwx 1 root root 13 May 27 01:12 conf;5ceb0f32 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 01:34 conf;5ceb1477 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 02:09 conf;5ceb1c24 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 03:58 conf;5ceb3650 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 06:50 conf;5ceb5dd1 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 18:55 conf;5cec0874 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 19:13 conf;5cec0c93 -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 19:17 conf;5cec0d9c -> /etc/tez/conf
lrwxrwxrwx 1 root root 13 May 27 19:21 conf;5cec0e81 -> /etc/tez/conf I think every unsuccessful attempt added a new line. I removed all [root@node2 ~]# rm -rf /usr/hdp/3.1.0.0-78/tez/* Using Ambari I reinstalled tez client for this node only. It worked.
... View more
05-25-2019
06:17 PM
Thank you @Stylianos Sideridis. The issue cooled. Next time I'll keep it in mind, thanks again.
... View more
05-09-2019
01:36 AM
thank you very much @Geoffrey Shelton Okot. The job worked. One last thing: I can't see any table in hive azhadoop. my query: sqoop import --connect jdbc:mysql://sandbox-hdp.hortonworks.com/azhadoop --driver com.mysql.jdbc.Driver --username root --password hadoop --query "select * from iris_mysql WHERE \$CONDITIONS" --m 1 --hive-import --hive-table azhadoop.iris_hive --target-dir /tmp/hive_temp the result of mr job 19/05/08 21:33:10 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=172694
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=87
HDFS: Number of bytes written=4574
HDFS: Number of read operations=4
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Other local map tasks=1
Total time spent by all maps in occupied slots (ms)=26964
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=3852
Total vcore-milliseconds taken by all map tasks=3852
Total megabyte-milliseconds taken by all map tasks=5916672
Map-Reduce Framework
Map input records=151
Map output records=151
Input split bytes=87
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=135
CPU time spent (ms)=1310
Physical memory (bytes) snapshot=241512448
Virtual memory (bytes) snapshot=3256225792
Total committed heap usage (bytes)=152567808
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=4574
19/05/08 21:33:10 INFO mapreduce.ImportJobBase: Transferred 4.4668 KB in 26.0204 seconds (175.7852 bytes/sec)
19/05/08 21:33:10 INFO mapreduce.ImportJobBase: Retrieved 151 records.
... View more
05-07-2019
09:22 PM
Hi @Geoffrey Shelton Okot thanks again. Interestingly adding --driver made the ERROR disappear. But another problem showed up [root@sandbox-hdp ~]# sqoop import --connect jdbc:mysql://sandbox-hdp.hortonworks.com/azhadoop --driver com.mysql.jdbc.Driver --username root --password hadoop --query "select * from iris_mysql WHERE \$CONDITIONS" --m 1 --hive-import --hive-table azhadoop.iris_hive --target-dir /tmp/hive_temp
Warning: /usr/hdp/2.6.4.0-91/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/05/07 21:04:19 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.4.0-91
19/05/07 21:04:19 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/05/07 21:04:19 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
19/05/07 21:04:19 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
19/05/07 21:04:20 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
19/05/07 21:04:20 INFO manager.SqlManager: Using default fetchSize of 1000
19/05/07 21:04:20 INFO tool.CodeGenTool: Beginning code generation
19/05/07 21:04:20 INFO manager.SqlManager: Executing SQL statement: select * from iris_mysql WHERE (1 = 0)
19/05/07 21:04:20 INFO manager.SqlManager: Executing SQL statement: select * from iris_mysql WHERE (1 = 0)
19/05/07 21:04:20 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.4.0-91/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/3e81cb85d0e8a571138759f1babfc886/QueryResult.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/05/07 21:04:22 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/3e81cb85d0e8a571138759f1babfc886/QueryResult.jar
19/05/07 21:04:22 INFO mapreduce.ImportJobBase: Beginning query import.
19/05/07 21:04:23 INFO client.RMProxy: Connecting to ResourceManager at sandbox-hdp.hortonworks.com/172.17.0.2:8032
19/05/07 21:04:23 INFO client.AHSProxy: Connecting to Application History server at sandbox-hdp.hortonworks.com/172.17.0.2:10200
19/05/07 21:04:26 INFO db.DBInputFormat: Using read commited transaction isolation
19/05/07 21:04:26 INFO mapreduce.JobSubmitter: number of splits:1
19/05/07 21:04:27 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1557245169101_0001
19/05/07 21:04:27 INFO impl.YarnClientImpl: Submitted application application_1557245169101_0001
19/05/07 21:04:27 INFO mapreduce.Job: The url to track the job: http://sandbox-hdp.hortonworks.com:8088/proxy/application_1557245169101_0001/
19/05/07 21:04:27 INFO mapreduce.Job: Running job: job_1557245169101_0001
19/05/07 21:04:40 INFO mapreduce.Job: Job job_1557245169101_0001 running in uber mode : false
19/05/07 21:04:40 INFO mapreduce.Job: map 0% reduce 0% It doesn't move stuck with mapreduce job. No progress, job takes 2550 memory from YARN and status running. No error but no progress. How can anyone import a query from mysql to hive in a sandbox?
... View more
04-29-2019
08:27 AM
Hi @Geoffrey Shelton Okot thanks for your answer. But are you sure this is a driver problem? I think it works fine because I am able to import from mysql to hdfs.
... View more