Member since
05-22-2021
6
Posts
0
Kudos Received
0
Solutions
07-31-2021
10:47 AM
@flowerbirds From the error it looks issues with python packages, can you verify with the other hosts if there are any missing python packages in ../../lib/python2.7/site-packages on the agent host? - Is you python version same on all the hosts? - Run the command below and share the output from the issue host and healthy host rpm -qa | grep cloudera* - parser module is known to cause, review the below https://stackoverflow.com/questions/18577975/python-attributeerror-dep-map https://stackoverflow.com/questions/54908171/pip-installation-of-aiohttp-fails-with-odd-attributeerror-dep-map
... View more
07-13-2021
02:15 AM
Same problem: 21/07/13 15:56:34 ERROR ExecutionImpl: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
at org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:530)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:172)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:662)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:606)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:97)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:205)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:313)
at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:202)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:513)
... 84 more
Caused by: java.lang.RuntimeException: Could not find any configured addresses for URI hdfs://nameservice1/user/hive/warehouse/test.db/test/year_month=20187
at org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider.<init>(ConfiguredFailoverProxyProvider.java:93)
... 89 more
... View more
07-12-2021
09:13 PM
@sur I believe you have to Set below properties before executing create table statement. SET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager; SET hive.support.concurrency=true; This seems a very known behaviour and I did a simple Googling and found the solutions on several places. Below are the references. https://stackoverflow.com/questions/42669171/semanticexception-error-10265-while-running-simple-hive-select-query-on-a-tran https://dbmstutorials.com/hive/hive-errors.html https://knowledge.informatica.com/s/article/526538?language=en_USv Hope this helps.
... View more
07-12-2021
07:45 PM
I think you should check yarn logs for more details info.
... View more
07-12-2021
07:14 AM
2 Kudos
@flowerbirds For CM server DB you can try something like below: In db.properties: com.cloudera.cmf.db.type = mysql
com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://<db_host>/scm?useSSL=true
com.cloudera.cmf.orm.hibernate.connection.username=scm
com.cloudera.cmf.orm.hibernate.connection.password=cloudera In /etc/default/cloudera-scm-server add the following to export CMF_JAVA_OPTS: -Djavax.net.ssl.trustStore=/opt/cloudera/security/jks/truststore.jks Restart the Cloudera Manager server. NOTE: Just remember in some case we set ssl=true&sslmode=require so if the normal method means com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://<db_host>/scm?useSSL=true failed please try the `ssl=true&sslmode=require as well and see.
... View more