Member since
09-06-2017
14
Posts
1
Kudos Received
0
Solutions
01-29-2019
08:28 AM
Hi, I am running a spark job to compress a file, most of the times its successful, but every once in a while this issue occurs and sometimes its stuck indefinitely. Since the job is still in running phase , we don't find any error in the yarn logs. It gets stuck right after the driver requests to kill the executor. In this particular job it got stuck in this phase for about 6 hours although it easily completed for data much smaller in size. Also, dynamic allocation is enabled in our cluster. 19/01/28 09:19:40 INFO YarnAllocator: Driver requested a total number of 1 executor(s).
19/01/28 09:19:40 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 2, 1.
19/01/28 09:19:41 INFO YarnAllocator: Driver requested a total number of 0 executor(s).
19/01/28 09:19:41 INFO ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 3.
19/01/28 15:14:39 INFO ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down. 19/01/28 15:14:39 INFO ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down.
... View more
Labels:
10-16-2018
01:30 PM
I am trying to update multiple policies in a particular service simultaneously. Using the below command i am getting all the policies in a particular service curl -vk -H "Content-type:application/json" --netrc-file my-password-file -X GET http://localhost:6080/service/public/v2/api/service/POC_hadoop/policy -o rangerpolicies.json In the file obtained i.e. rangerpolicies.json , i am updating a few properties. While updating, using the put command, i am facing error which i have attached as attachment. error.txt The command used:- curl -vk "Content-type:application/json" --netrc-file my-password-file -X PUT -d @rangerpolicies.json http://localhost:6080/service/public/v2/api/service/POC_hadoop/policy/ A single policy-wise, i am able to update, but is this option not available at service level? Also, is there any workaround available, because to update all policies one by one in case of 100+ policies is not feasible. Ranger version :0.7 Thanks in advance.
... View more
Labels:
10-16-2018
01:30 PM
What is the the use of the the active status- enable/disable tab while creating a new ranger service? If the service is disabled, will the policies in the service not work anymore? I tried it, but they still work even after disabling the service. Ranger version=0.7
... View more
Labels:
05-16-2018
09:22 AM
I am trying to access pyspark where SPARK_MAJOR_VERSION is set to 2 by a certain user but I am getting the following error: at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
18/05/14 10:18:18 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
Traceback (most recent call last):
File "/usr/hdp/current/spark2/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/usr/hdp/current/spark2/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o34.sessionState.b
: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$reflect(SparkSession.scala:983)
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$reflect(SparkSession.scala:980)
... 13 more
Caused by: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveExternalCatalog':
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$reflect(SharedState.scala:176)
at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
at org.apache.spark.sql.SparkSession$anonfun$sharedState$1.apply(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession$anonfun$sharedState$1.apply(SparkSession.scala:101)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
... 18 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$reflect(SharedState.scala:173)
... 26 more Although the same user is able to access pyspark from a different node in the same environment, it is only the edge node that is facing this error. Interestingly, other users are able to access pyspark from edge node too. I believe it is related to some permission issue but i am not sure where else to check for permission. Please suggest where can be the possible places to cause such an error. Thanks in advance.
... View more
04-02-2018
01:12 PM
Hi, Referring to https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-operations/content/archiving_and_purging_data.html it says that archiving and purging of logs is possible in Ambari. But is it possible for all components or only Atlas, Ranger, and Log Search? actually my aim is to run a script to archive and delete all logs for major hadoop components post a stipulated date so that my cluster doesn't get cluttered. But i came across this, will Ambari-infra help in achieving this purpose? Thanks for your help in advance.
... View more
Labels:
03-22-2018
04:56 AM
@Frank Lu @Kit Menke Please can you help with this one?
... View more
03-21-2018
01:29 PM
When I Sqoop data from Teradata to Hive, I am running the command sqoop import --connect jdbc:teradata://link/database=db --connection-manager org.apache.sqoop.teradata.TeradataConnManager --username user -P --table table --hive-import -m 1 and getting the following error- 18/03/21 14:13:50 INFO teradata.TeradataSqoopImportHelper: Setting source table to : table 18/03/21 14:13:51 INFO common.ConnectorPlugin: load plugins in jar:file:/usr/hdp/current/hadoop/lib/teradata-connector-1.5.2-hadoop2.jar!/teradata.connector.plugins.xml
18/03/21 14:13:52 INFO teradata.TeradataSqoopImportHelper: Teradata import job completed with exit code 1
Exception in thread "main" java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:221)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:209)
at com.teradata.connector.hive.utils.HiveUtils.isHiveOutputTablePartitioned(HiveUtils.java:1140)
at com.teradata.connector.common.tool.ConfigurationMappingUtils.importConfigurationMapping(ConfigurationMappingUtils.java:321)
at org.apache.sqoop.teradata.TeradataSqoopImportHelper.runJob(TeradataSqoopImportHelper.java:364)
at org.apache.sqoop.teradata.TeradataConnManager.importTable(TeradataConnManager.java:505)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507) Any help is highly appreciated and thanks in advance..
... View more
Labels:
03-19-2018
08:40 AM
@Jay Kumar SenSharmaThank you so much for your swift response , I wanted to confirm one thing more. If, for example I reset my admin password for mysql/ranger will that affect any service because I have database of ranger, oozie and ambari in that or in case of ranger i have policies for services there. I think not but just wanted to confirm. Also. if there does not exist any common place / configuration to update all the individual components can i achieve this with a script? using curl commands to call to change password?
... View more
03-19-2018
07:30 AM
1 Kudo
I want to reset my admin password for all the major components of Ambari namely, the ambari webui, mysql, smartsense, ranger etc. After I reset the password of all these places (for the admin user), what places are to be updated after such a reset is done so that my cluster doesn't get affected.. might be those places from where the admin password is being referred? Or the system updates such all those places on its own and we don't need to make any manual changes? I need to check if this won't pose a problem before all the password reset..and none of my services will get affected. thanks in advance.
... View more
02-27-2018
11:57 AM
I faced the same issue and on application of these commands it worked but Even after the hst server/agent are up, smartsense is still not able to collect data. The queries are not populating any data. Why is that happening? @Jay Kumar SenSharma Please can you help.
... View more
09-07-2017
08:18 AM
I fetched details of job in zeppelin notebook query using phoenix interpreter and matched the submit time with the same job id in the Resource Manager UI in Ambari. I found that there is a two hour difference in both. How do i change the timezone in Zeppelin query? please help
... View more
09-07-2017
07:36 AM
@Jay SenSharma Please can you help with how can i access these tables. some of my Tez jobs and all of my Spark jobs are not entering activity.job table. hence it is not giving many jobs in the output. I did left outer join so that i get all jobs but why is it happening?
... View more
09-07-2017
07:33 AM
@Josh Elser Please can you help with this?
... View more
09-07-2017
07:32 AM
I did LEFT OUTER JOIN on this and and it seems to work but i have another issue. The time of the job fetched by the zeppelin smart sense query and that of the same job on RM UI is different by two hours, how do i change that? i did the following-> 1) Go to HBASE -> Configs -> Custom hbase-site. Add the property
phoenix.query.dateFormatTimeZone=GMT+08:00 2) using the "timeZone" option in the Phoenix Thin Driver's but it doesn't seem to work. Is there anything else i can do?
... View more