Member since
10-01-2016
156
Posts
8
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
8188 | 04-04-2019 09:41 PM | |
3142 | 06-04-2018 08:34 AM | |
1470 | 05-23-2018 01:03 PM | |
2974 | 05-21-2018 07:12 AM | |
1830 | 05-08-2018 10:48 AM |
07-16-2018
06:44 AM
I tried sqoop and get following error bu import is successfull ERROR security.InMemoryJAASConfiguration: Unable to add JAAS configuration for client [KafkaClient] as it is missing param [atlas.jaas.KafkaClient.loginModuleName]. Skipping JAAS config for [KafkaClient] and org.apache.atlas.notification.NotificationException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 du e to 30075 ms has passed since batch creation plus linger time
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:239)
at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:212)
at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:114)
at org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:143)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:128)
at org.apache.atlas.sqoop.hook.SqoopHook.publish(SqoopHook.java:190)
at org.apache.atlas.sqoop.hook.SqoopHook.publish(SqoopHook.java:51)
at org.apache.sqoop.mapreduce.PublishJobData.publishJobData(PublishJobData.java:52)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:284)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:748)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:509)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.main(Sqoop.java:243)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 due to 30075 ms has passed since batch creat ion plus linger time
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:65)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:52)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25)
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:230)
... 17 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 due to 30075 ms has passed since batch creation plus linger time
... View more
07-16-2018
06:34 AM
Hi @Sandeep Nemuri thank you. First configs.py has worked. Bu second one has given following: Traceback (most recent call last):
File "/var/lib/ambari-server/resources/scripts/configs.py", line 364, in <module>
sys.exit(main())
File "/var/lib/ambari-server/resources/scripts/configs.py", line 357, in main
return delete_properties(cluster, config_type, action_args, accessor)
File "/var/lib/ambari-server/resources/scripts/configs.py", line 248, in delete_properties
update_config(cluster, config_type, delete_specific_property(config_name), accessor)
File "/var/lib/ambari-server/resources/scripts/configs.py", line 131, in update_config
properties, attributes = config_updater(cluster, config_type, accessor)
File "/var/lib/ambari-server/resources/scripts/configs.py", line 195, in update
properties, attributes = get_current_config(cluster, config_type, accessor)
File "/var/lib/ambari-server/resources/scripts/configs.py", line 123, in get_current_config
config_tag = get_config_tag(cluster, config_type, accessor)
File "/var/lib/ambari-server/resources/scripts/configs.py", line 94, in get_config_tag
response = accessor(DESIRED_CONFIGS_URL.format(cluster))
File "/var/lib/ambari-server/resources/scripts/configs.py", line 89, in do_request
raise Exception('Problem with accessing api. Reason: {0}'.format(exc))
Exception: Problem with accessing api. Reason: HTTP Error 404: Not Found
... View more
07-16-2018
06:23 AM
Thank you @rguruvannagari I just enabled SolrCloud and restarted. I don't see audit error anymore.
... View more
07-16-2018
05:48 AM
Hi @Jonathan Sneep I installed my cluster via Ambari so there is no such directory. I looked at ranger-admin directory all are .py script not .sh
... View more
07-13-2018
12:52 PM
I use HDP 2.6.3. I use Ranger. From Ranger UI once I click Audit I see the following ERROR. Error running solr query, please check solr configs.Errorfrom server at http://hadooptest01.datalonga.com:8886/solr/ranger_audits: Expected mime type application/octet-stream but got text/html.
... View more
Labels:
- Labels:
-
Apache Ranger
-
Apache Solr
07-13-2018
08:43 AM
Hi @Sandeep Nemuri I can't determine the parameters to use configs.py
... View more
07-13-2018
08:42 AM
Hi @svaddi HDP 2.6.3, sqoop import, source RDBMS Sybase, destination Hive table, Kerberos not enabled. Query: sqoop import --connect jdbc:sybase:Tds:XX.XX.XX.XX:2640 --driver com.sybase.jdbc4.jdbc.SybDriver --username XXX --password XXX --target-dir /dw/iq_aktar/XXX --query "select * FROM table and \$CONDITIONS " --m 1 --hive-import --hive-overwrite --hive-table XX.XXX --hive-drop-import-delims
... View more
07-13-2018
07:46 AM
No ranger_audits [zk: hadooptest01:2181(CONNECTED) 0] ls /infra-solr/configs
[atlas_configs]
... View more
07-13-2018
07:22 AM
While grafana service is stopped on Ambari. I find grafana process with. ps aux | grep grafana Then kill the process. Started Grafana from Ambari.
... View more
07-12-2018
02:53 PM
I use HDP 2.6.3. I have a Spark application that runs per five minutes and appends hive tables. In hiveserver2.log I get following error: org.apache.atlas.notification.NotificationException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 due to 30065 ms has passed since last append
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:239)
at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:212)
at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:114)
at org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:143)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:128)
at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:181)
at org.apache.atlas.hive.hook.HiveHook.access$300(HiveHook.java:85)
at org.apache.atlas.hive.hook.HiveHook$3.run(HiveHook.java:225)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.atlas.hive.hook.HiveHook.notifyAsPrivilegedAction(HiveHook.java:234)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:207)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 due to 30065 ms has passed since last append
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:65)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:52)
at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25)
at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:230)
... 17 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ATLAS_HOOK-0 due to 30065 ms has passed since last append I am fed up with AtlasHook errors.
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache Hive