Member since
01-12-2022
16
Posts
2
Kudos Received
0
Solutions
12-06-2022
07:34 PM
Hi,r ecently I have been trying to track my spark application using SparkListener. Now I have a problem: There is a application, the result of its execution is SUCCESSED on Yarn WebUI,but it's actually a failed application. And my listener class can not get the error message,the application log only show the Start events for failed jobs and tasks. Some can help me ?
... View more
Labels:
- Labels:
-
Apache Spark
12-06-2022
07:04 PM
1 Kudo
Thanks,but this api is used to delete the entity,I'm afraid it can't be used to delete a deleted Relationship hardly.
... View more
11-03-2022
02:25 AM
You can try this using the Rest api: # DELETE the Client
curl -k -u admin:admin -H "X-Requested-By: ambari" -X DELETE
"$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/host_components?HostRoles/component_name.in(HDFS_CLIENT)&HostRoles/host_name.in(hostname01,hostname02)"
# INSTAL the Client
curl -v -k -i -u 'admin:admin' -H 'X-Requested-By: ambari' -X POST -d '
{
"RequestInfo":{
"query":"Hosts/host_name.in(hostname01)"
},
"Body":{
"host_components":[
{
"HostRoles":{
"component_name":"HDFS_CLIENT"
}
}
]
}
}' '$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/hosts'
... View more
11-03-2022
02:00 AM
Hi, I'm trying to configure the spark-connector for apache atlas. When I execute this statement "create table dept_three_spark as select * from dept_four_ext_hive;" ,the table created successed ,but I get the following error: And when I visited atlas, a lot of information was missing. I tried to add the number of kafka topic ATLAS_HOOK and ATLAS_ENITIES partitions, but no use. What can I do next to solve this problem? Spark version: 2.3.0 Atlas version: 2.2.0 Kafka version: 2.0.0 HDP version: 3.1.5.0-152 Spark-connector version: 0.1.0.3.1.5.0-152
... View more
Labels:
11-01-2022
12:52 AM
1 Kudo
Hi: Is it currently possible to hard delete (purge) Relationships and RelationshipDefs via the API? I use the api /api/atlas/admin/purge/ to delete a DELETED Relationship but no use. Any help will be useful,Thanks. The Atlas version is 2.2.0
... View more
- Tags:
- Atlas
- hard-delete
Labels:
- Labels:
-
Apache Atlas
07-23-2022
08:51 PM
Hi, Recently, in our production environment, there was an abnormal exit of the Hbase Master. In the log, I found that the ZK connection had some problems. According to the sessionid, I found that the sessionid of one of our applications was the same as the sessionid established by the master at the time in question, and then the application side closed the session, and then the master hung. Is this possible? Why does a session with the same ZooKeeper sessionid appear?
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Zookeeper
07-08-2022
11:22 PM
I know.Need FQDN likes testhost.magg.com,Thanks
... View more
07-08-2022
09:37 PM
Thanks.I open the https for nifi.And it works when Knox and Nifi on same server. But when I deploy NIFI and KNOX separately, I am redirected infinitely when SSO logs in to NIFI and I find Cookies have problems. Is this because of cross-domain? Knox gateway.log has "JWT cookie successfully added." but nifi-user.log have " [<anonymous>] GET https://XXX:9443/nifi-api/flow/current-user"
... View more
07-06-2022
10:58 PM
When I tried to turn on KnoxSSO for NIFI without https, I can access NIFI's Web UI through Knox,but I can also access the NIFI directly through port 10111 which n o jump to Knox. So I tried to enable HTTPS while using Knox, and I got this error "Apache Knox SSO support cannot be enabled if the Login Identity Provider or OpenId Connect or SAML is configured."
... View more
07-06-2022
03:12 AM
Is there a way to use SSO without enabling HTTPS on NIFI?
... View more
Labels:
- Labels:
-
Apache NiFi
04-18-2022
10:06 PM
Hi,every one: I have a question.I found there are many "cmd=get_database" log in hadoop-cmf-hive-HIVEMETASTORE.out file.But I haven't done this much.Is it because there are so many other actions that trigger it?
... View more
Labels:
- Labels:
-
Apache Hive
01-21-2022
12:43 AM
I think the DataNode is restarting and scan blocks? It will return to normal when hen the block scan is complete. You can test it again w hen the following content appears in the log 2022-01-17 22:41:56,564 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add replicas to map for block pool BP-1115033092-xx.xx.xx.xx-1532155849771 on volume /data4/xxx/hdfs/dfs/data/current: 5270395ms
... View more
01-18-2022
12:12 AM
Thanks!It works!
... View more
01-16-2022
05:56 PM
Are you make sure your Datanodes are all started and working properly? When I restarted the cluster earlier,there are also many missingBlocks before the DataNode restarts completely.
... View more
01-16-2022
05:48 PM
Thanks,too sad. So,Is there another way to do that? I want to track the resource status of the container execution
... View more
01-12-2022
11:15 PM
Hello, I use yarn (base on HDP3.1.5.0-152),when I view the nodemanager log,I found the log doesn't note the log likes: INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28532 for container-id container_e51_1637148502450_21618293_01_000024: 4.1 GB of 4.5 GB physical memory used; 6.3 GB of 9.4 GB virtual memory used I don't remember there are some conf in yarn-site.xml can control this. Any one help?
... View more
Labels: