About shivkumar82015

shivkumar82015 · ‎08-04-2016

Please do the following it helped me. 1) Login to Ambari WI HDFS -> Quick Links ->NameNode UI ->Datanodes Check how capacity of HDFS and how much it utilized and left storage if blocks are not replicated equally. Jobs will be running on Data Node only, hence it running on two nodes only. 2) While running the MRapp always try to get for Data Locality during JOB run. 3) Do the Load Balancer on cluster , data will be distributed across the Datanodes. 4) After Balancer completed , try to check how jobs are running. still jobs are facing same , please update . if its help full , your close is appreciated.

shivkumar82015 · ‎08-03-2016

Thank you for your Opinion , this below information be help full on Input Splits InputFormat describes the input-specification for a MapReduce job. The MapReduce framework relies on the InputFormat of the job to: Validate the input-specification of the job. Split-up the input file(s) into logical InputSplit instances, each of which is then assigned to an individual Mapper. Provide the RecordReader implementation used to glean input records from the logical InputSplit for processing by the Mapper. The default behavior of file-based InputFormat implementations, typically sub-classes of FileInputFormat, is to split the input into logical InputSplit instances based on the total size, in bytes, of the input files. However, the FileSystem blocksize of the input files is treated as an upper bound for input splits. A lower bound on the split size can be set via mapreduce.input.fileinputformat.split.minsize. Clearly, logical splits based on input-size is insufficient for many applications since record boundaries must be respected. In such cases, the application should implement a RecordReader, who is responsible for respecting record-boundaries and presents a record-oriented view of the logical InputSplit to the individual task. TextInputFormat is the default InputFormat. The Hadoop job client then submits the job (jar/executable etc.) and configuration to the ResourceManager which then assumes the responsibility of distributing the software/configuration to the slaves( Datanodes), scheduling tasks and monitoring them, providing status and diagnostic information to the job-client. if this is help full coments and accept are appreciated.

shivkumar82015 · ‎08-03-2016

Yes this happens on Slaves Nodes (Datanodes or HDFS nodes only)

shivkumar82015 · ‎08-03-2016

Due to hurry not updated. i have followed the below steps 1) Connected to the Internet 2) Up-graded the openssl,java(1.7 applied latest patch) #yum upgrade openssl #yum upgrade java. 3) Reboot the server 4) Ran Cluster Config Wizard , i am able to register the Host.

shivkumar82015 · ‎08-03-2016

Please find screen shoot

shivkumar82015 · ‎08-03-2016

Now i able to register the host to cluster, will update steps in some time.

shivkumar82015 · ‎08-02-2016

Host Check-up Fine Error log file. ========================== Creating target directory... ========================== Command start time 2016-08-02 22:08:14 Connection to centos1.test.com closed. SSH command execution finished host=centos1.test.com, exitcode=0 Command end time 2016-08-02 22:08:15 ========================== Copying common functions script... ========================== Command start time 2016-08-02 22:08:15 scp /usr/lib/python2.6/site-packages/ambari_commons host=centos1.test.com, exitcode=0 Command end time 2016-08-02 22:08:15 WARNING 2016-08-02 22:09:05,586 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... WARNING 2016-08-02 22:09:05,586 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... INFO 2016-08-02 22:09:09,491 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:09,491 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:14,523 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-08-02 22:09:14,996 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:14,996 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:14,999 DataCleaner.py:39 - Data cleanup thread started INFO 2016-08-02 22:09:15,008 DataCleaner.py:120 - Data cleanup started INFO 2016-08-02 22:09:15,030 DataCleaner.py:122 - Data cleanup finished INFO 2016-08-02 22:09:15,067 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2016-08-02 22:09:15,069 main.py:287 - Connecting to Ambari server at https://centos:8440 (192.168.154.111) INFO 2016-08-02 22:09:15,069 main.py:287 - Connecting to Ambari server at https://centos:8440 (192.168.154.111) INFO 2016-08-02 22:09:15,069 NetUtil.py:59 - Connecting to https://centos:8440/ca ERROR 2016-08-02 22:09:15,148 NetUtil.py:77 - [Errno 1] _ssl.c:492: error:100AE081:elliptic curve routines:EC_GROUP_new_by_curve_name:unknown group ERROR 2016-08-02 22:09:15,148 NetUtil.py:78 - SSLError: Failed to connect. Please check openssl library versions. Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details. WARNING 2016-08-02 22:09:15,148 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... WARNING 2016-08-02 22:09:15,148 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... ', None) ('Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more detail s. WARNING 2016-08-02 22:09:05,586 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... WARNING 2016-08-02 22:09:05,586 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... INFO 2016-08-02 22:09:09,491 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:09,491 main.py:74 - loglevel=logging. INFO INFO 2016-08-02 22:09:14,523 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-08-02 22:09:14,996 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:14,996 main.py:74 - loglevel=logging.INFO INFO 2016-08-02 22:09:14,999 DataCleaner.py:39 - Data cleanup thread started INFO 2016-08-02 22:09:15,008 DataCleaner.py:120 - Data cleanup started INFO 2016-08-02 22:09:15,030 DataCleaner.py:122 - Data cleanup finished INFO 2016-08-02 22:09:15,067 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2016-08-02 22:09:15,069 main.py:287 - Connecting to Ambari server at https://centos:8440 (192.168.154.111) INFO 2016-08-02 22:09:15,069 main.py:287 - Connecting to Ambari server at https://centos:8440 (192.168.154.111) INFO 2016-08-02 22:09:15,069 NetUtil.py:59 - Connecting to https://centos:8440/ca ERROR 2016-08-02 22:09:15,148 NetUtil.py:77 - [Errno 1] _ssl.c:492: error:100AE081:elliptic curve routines:EC_GROUP_new_by_curve_name:unknown group ERROR 2016-08-02 22:09:15,148 NetUtil.py:78 - SSLError: Failed to connect. Please check openssl library versions. Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details. WARNING 2016-08-02 22:09:15,148 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... WARNING 2016-08-02 22:09:15,148 NetUtil.py:105 - Server at https://centos:8440 is not reachable, sleeping for 10 seconds... ', None) Connection to centos1.test.com closed. SSH command execution finished host=centos1.test.com, exitcode=0 Command end time 2016-08-02 22:08:26 Registering with the server... Registration with the server failed.

shivkumar82015 · ‎08-02-2016

Thanks for your reply it worked for me. [root@ces ~]# pssh -v -h "host.txt" -A -i "-O StrictHostKeyChecking=no" -l root "touch sshkeys.txt" Warning: do not enter your password if anyone else has superuser privileges or access to your account. Password: [1] 18:04:48 [SUCCESS] cens [2] 18:04:48 [SUCCESS] cent [3] 18:04:48 [SUCCESS] cen [4] 18:04:48 [SUCCESS] cen4 [root@centos ~]# ls -lrt -rw-r--r-- 1 root root 31 Aug 2 18:01 host.txt -rw-r--r-- 1 root root 0 Aug 2 18:04 sshkeys.txt

shivkumar82015 · ‎08-02-2016

Yep i have given all correct IP for this , while pasting here i changed it.

shivkumar82015 · ‎08-01-2016

[root@centos ~]# pssh -v -t 0 -l root -h clusterhost.txt -x "-t -t -oStrickHostKeyChecking=no" 'touch testing' [1] 17:06:00 [FAILURE] 10.00.01 Exited with error code 255 [2] 17:06:00 [FAILURE] 10.00.02 Exited with error code 255 [3] 17:06:00 [FAILURE] 10.00.03 Exited with error code 255 [4] 17:06:00 [FAILURE] 10.00.04 Exited with error code 255 [root@centos ~]#

Online	Offline
Last Visited	‎09-27-2018 08:11 AM

Member Since	‎07-18-2016 02:20 AM
Last Visited	‎09-27-2018 08:11 AM
Posts	262
Kudos received	12

Cloudera Community

Re: How to find yarn application ID for Hadoop fs ...

Re: HDPCD Certificaiton Environment ssue , not abl...

Re: MR2 Service Check failing after New Installing...

Re: An internal system exception occurred: Stack H...

Re: Delete/update on hadoop partitioned table in...

Re: Load not distributed in the cluster

Re: where an when does the fileinputformat() runs...

Re: where an when does the fileinputformat() runs...

Re: 2.1 Ambari Cluster Install Wizard (1 host Reg...

Re: 2.1 Ambari Cluster Install Wizard (1 host Reg...

Re: 2.1 Ambari Cluster Install Wizard (1 host Reg...

2.1 Ambari Cluster Install Wizard (1 host Registe...

Re: While creting PSSH getting below error ([1] 17...

Re: While creting PSSH getting below error ([1] 17...

While creting PSSH getting below error ([1] 17:06:...