Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2059 | 06-15-2020 05:23 AM | |
| 17048 | 01-30-2020 08:04 PM | |
| 2220 | 07-07-2019 09:06 PM | |
| 8556 | 01-27-2018 10:17 PM | |
| 4838 | 12-31-2017 10:12 PM |
11-18-2019
01:03 PM
yes I did it ( i forget to write it in my post ( but cant edit the post ) so after I run it as you wrote still with the same problem < HTTP/1.1 405 Method Not Allowed HTTP/1.1 405 Method Not Allowed
... View more
11-18-2019
10:02 AM
we have two SPARK2_THRIFTSERVER on node01/03 in our ambari server
we want to delete both SPARK2_THRIFTSERVER on both nodes
we try the following API , but without success , any idea where we are wrong?
component_name : SPARK2_THRIFTSERVER
the REST API: ( we try to delete the thrift server on one of the nodes )
curl -iv -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://node02:8080/api/v1/clusters/HDP/hosts/node01/SPARK2_THRIFTSERVER
* About to connect() to node02 port 8080 (#0) * Connected to node02 (45.3.23.4) port 8080 (#0) * Server auth using Basic with user 'admin' > DELETE /api/v1/clusters/HDP/hosts/node01/SPARK2_THRIFTSERVER HTTP/1.1 > Authorization: Basic YWRtaW46YWRtaW4= > User-Agent: curl/7.29.0 > Accept: */* > X-Requested-By: ambari > < HTTP/1.1 404 Not Found HTTP/1.1 404 Not Found
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Spark
11-15-2019
05:24 AM
since we have both current folders as: /hadoop/hdfs/namenode/current ( when fsimage exists i /hadoop/hdfs/journal/hdfsha/current/ do you mean to backup both them? second how backup for time prescriptive for example one week or more ?
... View more
11-15-2019
04:37 AM
about option one I guess you not mean to backup the metadata by copy it as scp or rsync maybe you means that there are tool for backup like barman for postgresql so do you know tool for this option? on each name nodes we have the following folders /hadoop/hdfs/namenode/current ( when fsimage exists i /hadoop/hdfs/journal/hdfsha/current/ do you means to backup only these folders , lets say every day ?
... View more
11-15-2019
02:40 AM
ok do you can summarize all options to recover the namenode ( also option out from the box )
... View more
11-15-2019
12:45 AM
hi all
I want to ask this important question
lets say we have the following:
HDP cluster with :
3 masters machine ( active/standby name-node ) , ( active/standby resource manager )
3 datanode machines
- each data-node machine have 4 disks for HDFS ( not include the OS )
3 kafka machines
- each kafka machine have one disk of 10T ( not include the OS )
now we want to install from scratch all cluster include HDP and ambari
but save the data on datanode machines and kafka topics data machine by the following:
we umount the disks on datanode machines and kafka machines
example
on datanode machine ( note - /etc/stab is already configured )
umount /grid/data1
umount /grid/data2
.
.
.
so the second scratch installation we install all the cluster ( by blueprint ) , but without data-node HDFS disks , and kafka topic disk ( scratch installation means fresh new linux OS )
after installation we mount all data-node machines disks and kafka disks machines ( where we are store all topics )
example
on datanode machine ( note - /etc/stab is already configured )
mount /grid/data1
mount /grid/data2
.
.
.
in order to complete the picture , need to restart HDFS and YARN and kafka
so - is this scenario could to work ?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Kafka
11-14-2019
01:01 PM
hi lets say on the faulty namenode we see only the following: under - /hadoop/hdfs/journal/hdfsha/current example: -rw-r--r-- 1 hdfs hadoop 155 Nov 13 00:50 VERSION -rw-r--r-- 1 hdfs hadoop 3 Nov 14 21:24 last-promised-epoch drwxr-xr-x 2 hdfs hadoop 4096 Nov 14 21:24 paxos -rw-r--r-- 1 hdfs hadoop 3 Nov 14 21:24 last-writer-epoch -rw-r--r-- 1 hdfs hadoop 8 Nov 14 23:16 committed-txid as you know usually we have also the files like - edits_0000000000000066195-0000000000000066242 is this scenario change the picture?
... View more
11-13-2019
02:03 PM
you means the files /hadoop/hdfs/namenode/current/fsimage_* should be the same on both nodes exactly ? in this case its easy to copy from the good namenode to the bad namenode and restart the bad namenode let me know if this is the procedure?
... View more
11-13-2019
01:47 PM
sure we can backup both folders in tar file but what is your suggestion on the node without the fsimage files?
... View more
11-13-2019
01:06 PM
hi the second name node have the fsimage files , but from ambari the second namenode not appears as standby/active is just up
... View more