About Meepoljd

Meepoljd · ‎12-06-2022

Hi,r ecently I have been trying to track my spark application using SparkListener. Now I have a problem: There is a application, the result of its execution is SUCCESSED on Yarn WebUI,but it's actually a failed application. And my listener class can not get the error message,the application log only show the Start events for failed jobs and tasks. Some can help me ?

Meepoljd · ‎12-06-2022

Thanks,but this api is used to delete the entity,I'm afraid it can't be used to delete a deleted Relationship hardly.

Meepoljd · ‎11-03-2022

You can try this using the Rest api: # DELETE the Client curl -k -u admin:admin -H "X-Requested-By: ambari" -X DELETE "$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/host_components?HostRoles/component_name.in(HDFS_CLIENT)&HostRoles/host_name.in(hostname01,hostname02)" # INSTAL the Client curl -v -k -i -u 'admin:admin' -H 'X-Requested-By: ambari' -X POST -d ' { "RequestInfo":{ "query":"Hosts/host_name.in(hostname01)" }, "Body":{ "host_components":[ { "HostRoles":{ "component_name":"HDFS_CLIENT" } } ] } }' '$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/hosts'

Meepoljd · ‎11-03-2022

Hi, I'm trying to configure the spark-connector for apache atlas. When I execute this statement "create table dept_three_spark as select * from dept_four_ext_hive;" ,the table created successed ,but I get the following error: And when I visited atlas, a lot of information was missing. I tried to add the number of kafka topic ATLAS_HOOK and ATLAS_ENITIES partitions, but no use. What can I do next to solve this problem? Spark version: 2.3.0 Atlas version: 2.2.0 Kafka version: 2.0.0 HDP version: 3.1.5.0-152 Spark-connector version: 0.1.0.3.1.5.0-152

Meepoljd · ‎11-01-2022

Hi: Is it currently possible to hard delete (purge) Relationships and RelationshipDefs via the API? I use the api /api/atlas/admin/purge/ to delete a DELETED Relationship but no use. Any help will be useful,Thanks. The Atlas version is 2.2.0

Meepoljd · ‎07-23-2022

Hi, Recently, in our production environment, there was an abnormal exit of the Hbase Master. In the log, I found that the ZK connection had some problems. According to the sessionid, I found that the sessionid of one of our applications was the same as the sessionid established by the master at the time in question, and then the application side closed the session, and then the master hung. Is this possible? Why does a session with the same ZooKeeper sessionid appear?

Meepoljd · ‎07-08-2022

I know.Need FQDN likes testhost.magg.com,Thanks

Meepoljd · ‎07-08-2022

Thanks.I open the https for nifi.And it works when Knox and Nifi on same server. But when I deploy NIFI and KNOX separately, I am redirected infinitely when SSO logs in to NIFI and I find Cookies have problems. Is this because of cross-domain? Knox gateway.log has "JWT cookie successfully added." but nifi-user.log have " [<anonymous>] GET https://XXX:9443/nifi-api/flow/current-user"

Meepoljd · ‎07-06-2022

When I tried to turn on KnoxSSO for NIFI without https, I can access NIFI's Web UI through Knox,but I can also access the NIFI directly through port 10111 which n o jump to Knox. So I tried to enable HTTPS while using Knox, and I got this error "Apache Knox SSO support cannot be enabled if the Login Identity Provider or OpenId Connect or SAML is configured."

Meepoljd · ‎07-06-2022

Is there a way to use SSO without enabling HTTPS on NIFI?

Meepoljd · ‎04-18-2022

Hi,every one: I have a question.I found there are many "cmd=get_database" log in hadoop-cmf-hive-HIVEMETASTORE.out file.But I haven't done this much.Is it because there are so many other actions that trigger it?

Meepoljd · ‎01-21-2022

I think the DataNode is restarting and scan blocks? It will return to normal when hen the block scan is complete. You can test it again w hen the following content appears in the log 2022-01-17 22:41:56,564 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add replicas to map for block pool BP-1115033092-xx.xx.xx.xx-1532155849771 on volume /data4/xxx/hdfs/dfs/data/current: 5270395ms

Meepoljd · ‎01-18-2022

Thanks!It works!

Meepoljd · ‎01-16-2022

Are you make sure your Datanodes are all started and working properly? When I restarted the cluster earlier,there are also many missingBlocks before the DataNode restarts completely.

Meepoljd · ‎01-16-2022

Thanks,too sad. So,Is there another way to do that？ I want to track the resource status of the container execution

Meepoljd · ‎01-12-2022

Hello, I use yarn (base on HDP3.1.5.0-152),when I view the nodemanager log,I found the log doesn't note the log likes: INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28532 for container-id container_e51_1637148502450_21618293_01_000024: 4.1 GB of 4.5 GB physical memory used; 6.3 GB of 9.4 GB virtual memory used I don't remember there are some conf in yarn-site.xml can control this. Any one help?

Online	Offline
Last Visited	‎12-06-2022 11:08 PM

Date Registered	‎01-12-2022 11:08 PM
Last Visited	‎12-06-2022 11:08 PM
Total Messages Posted	16
Total Kudos Received	2

Some questions with Spark Listener

Re: How to hard delete the relationShip in Apache ...

Re: In Ambari 2.6.2 how to delete and re-install c...

Some problems occurred while configuring SAC

How to hard delete the relationShip in Apache Atla...

Zookeeper same seesionId cause Master exception?

Re: Whether SSO can be used without enabling HTTPS...

Re: Whether SSO can be used without enabling HTTPS...

Re: Whether SSO can be used without enabling HTTPS...

Whether SSO can be used without enabling HTTPS on ...

Why my metastore log has too many cmd=get_database

Re: DataXceiver error processing WRITE_BLOCK opera...

Re: NodeManger doesn’t log the Memory usage mess...

Re: HDFS blocks missing at every restart of CM clu...

Re: NodeManger doesn’t log the Memory usage mess...

NodeManger doesn’t log the Memory usage message