Re: comfirn hosts, ambari agent error,ambari agent...

mathimurugan179 · ‎06-27-2018

Hi Team, I have a table called 'test' and its owner is 'ben', I need to change the ownership of the same table from 'ben' to 'sam' and i don't have login access to sam. How can i achieve this , Is there any possibility to change the ownership or is there any command to change the ownership Regards, Mathivanan

mathimurugan179 · ‎06-08-2018

Hi all, I need to commission a node in my secured cluster, I'm using MIT kerberos in my environment.Once I added, i need to make that newly added node as kerberised. Kindly let me the step to follow or any links to follow. Regards, Mathivanan

mathimurugan179 · ‎04-24-2018

Hi All, I need to take Hive Metastore to other cluster i.e (DR), Is there any Tools or functionality to achieve this scenario.

mathimurugan179 · ‎04-02-2018

Hi all, Is there any sqoop streaming mechanism to check the connectivity between two servers, i.e. source server and target server. If so please explain and do share if you have docs to refer.

mathimurugan179 · ‎04-02-2018

Hi All, I need to check the connection validity between sqoop and SAP HANA for several nodes using shell script. Can any one guide me how to write this shell script for connectivity validation. thank you

mathimurugan179 · ‎03-06-2018

@Aymen Rahal Hi, When registering host go with Manual Installation. and edit /etc/ambari-agent/conf/ambari-agent.ini change hostname=ambari server name Then Retry Failed.

mathimurugan179 · ‎02-09-2018

@Josh Elser Hi , I have set up Kerberos for authenticating HBase Rest API, Below are the steps which i have follewed: I have configured Kerberos in our HDP Environment using Automated method, and i have made some configuration changes in hbase-site.xml and core-site.xml to secure Rest API. "https://community.hortonworks.com/articles/91425/howto-start-and-test-hbase-rest-server-in-a-kerber.html" "https://developer.ibm.com/hadoop/2016/05/12/hbase-rest-gateway-security/" When initiating a kinit from Local machine it accepts and when accessing the HTTP it show as Authentication required 401 error. I have done configuration changes in Firefox "auth:config" and followed this link for HTTP authentication "https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/_configuring_http_authentication_for_HDFS_YARN_MapReduce2_HBase_Oozie_Falcon_and_Storm.html" Please find the snap for the reference, Please Guide How to proceed further. Regards, Mathivanan

mathimurugan179 · ‎01-25-2018

capture.jpg @GEOFFREY SHELTON OKOT Hi All, I have configured Kerberos in our Environment, and i have made some configuration changes in hbase-site.xml to secure Rest API.when initiating Rest API GUI it is asking for Username and Password .I had followed this docs for refence "https://developer.ibm.com/hadoop/2016/05/12/hbase-rest-gateway-security/" Whether this procedure is correct to proceed, if so what would be the username and password. Regards, Mathivanan

mathimurugan179 · ‎12-14-2017

Hi All, I have done upgrade from HDP 2.4.3 to 2.6.1, where my hive jobs are running but i cant get the Full log information of those hive jobs i.e. Map and Reduce logs are missing and i can only see the Query execution time and Application job id and others. It reflects with "log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender". While executing the hive command also i'm getting the same warn. I have tried with setting maxFileSize to 256 in hdfs log4.j configuration, yarn log4.j configuration, Hive log4.j configuration but it reflects the same. Please guide me how to ignore this error and bring back all my log information. Regards, Mathivanan

mathimurugan179 · ‎11-16-2017

Hi Team, I'm in need of the taking a Backup from Hbase Tables, I need to know the options available to take the Hbase table backup based on Column value. I had tried with "hbase org.apache.hadoop.hbase.mapreduce.Export -Dhbase.mapreduce.scan.row.start=0 -Dhbase.mapreduce.scan.row.stop=6" . but it does helped my scenario. For Example: id name 1 user1 2 user2 3 user3 Here I need to take the backup using the condition "id=2" to store the output file. We have Multiple users/client data and need to the take backup of each user's data separately. kindly suggest! Regards, Mathi

mathimurugan179 · ‎11-09-2017

Hi All, We are in need of deploying 85+ nodes in a hortonworks cluster, How many nodes can i go without enterprise Support. Is there any document for reference. Regards, Mathi

mathimurugan179 · ‎11-01-2017

hi Geoffrey Shelton Okot, As you said i had changed the setting in /etc/hosts, but you had mentioned 192.166.66.66 as your local hosts, what do i do in my case. Highlighted is my failed region server node kindly suggest the ip which need to be added with this. PFA Regards, Mathivanan

mathimurugan179 · ‎11-01-2017

@Geoffrey Shelton Okot As you mentioned All permission under /var/lib/hadoop-hdfs/ and HDFS--> configs -->Advanced hdfs site parameter dfs.domain.socket.path is pointing to /var/lib/hadoop-hdfs/dn_socket, are correct by default. Is there any other settings i need to change and any other ways to fix it. Regards, Mathivanan

mathimurugan179 · ‎11-01-2017

new-text-document.txtHi All, My HBase Region server is getting down due to the Connection Refused error. I got ending with error: "Connection refused when trying to connect to '/var/lib/hadoop-hdfs/dn_socket' " and find the attached logs for reference. What kind of error is this, how to resolve this issue. Kindly suggest Regards, Mathivanan

mathimurugan179 · ‎10-26-2017

Hi All, Is this the command help to retrieve a particular rowkey's data to store in hdfs. hbase org.apache.hadoop.hbase.mapreduce.Export -Dhbase.mapreduce.scan.row.start=ABC "mytable" "/export/mytable" Kindly suggest. Regards, Mathi

mathimurugan179 · ‎10-26-2017

Hi All, I need to take backup data from Hbase for Particular row key, is there anyoption to take the data from a particular Rowkey.For example "hbase export utility specifies any condition to retrieve (or) any other backup method will help to solve my problem". Regards, Mathi

mathimurugan179 · ‎10-25-2017

Hi All, We are planning to upgrade our Hadoop cluster, so before that we need to upgrade OS from RHEL 6.8-->RHEL7.x, by doing this OS level upgrade whether there will be any impact for HDP and Ambari level.or Upgrading the OS level will leads to Re-clustering of hadoop. kindly suggest the better way to Upgrade OS and Hadoop cluster and if the OS upgrade happens does it impact Hadoop cluster. Regards, Mathi

mathimurugan179 · ‎10-25-2017

Hi All, I'm planning to upgrade ambari version from 2.2.2 to 2.5.1 and for HDP from 2.4.3 to 2.6.1 is it recommended to upgrade in this same manner, or better to go with lower version upgrade example, ambari 2.2.2 to 2.3.x & to 2.4.x and for HDP 2.4.x to 2.5.x & to 2.6.x. Since im planning to upgrade in production environment i need this clarification.kindly suggest. Regards, Mathi

mathimurugan179 · ‎10-20-2017

Hi All, I'm having the requirement of retrieving a Particular Rowkey from Hbase tables and Store those in HDFS for taking Backup, is there any option to achieve this scenario.I tried with scan 'TABLENAME',{FILTER =>"(PrefixFilter ('ROWKEY'))"} for retrieving, but i don't know how to store those ROWKEY information in HDFS. Is there best way to do this.

mathimurugan179 · ‎08-08-2017

hi Ankit, All my Data nodes are healthy and there is no error log regarding this on HDFS. Is there any other suggestion. Regards, Mathi

mathimurugan179 · ‎08-07-2017

hbase.txtHi Team, My H-Base regions are getting frequently failed.we are running 7node cluster in this we have 5 region servers.with the same error i'm facing any of my regions are going down in a regular basis. Please find the attached Log.Kindly suggest Regards, Mathi

mathimurugan179 · ‎07-14-2017

Hi msumbul, yes i checked my table exists. Also i tried to read the table from the list, but it stated with error as mentioned below. Also i entered my host details in /etc/hosts/ and important is my kafka system is seperate host. Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Thu Jul 13 23:32:28 CDT 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68643: row 'Tenants,,00000000000000' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=aps-hadoop7,16020,1497499074661, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:271) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:203) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:821) at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:193) at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:89) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isTableAvailable(ConnectionManager.java:992) at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1486) at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1494) at example$.main(example.scala:45) at example.main(example.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorI Regards, Mathi

mathimurugan179 · ‎07-13-2017

Hi all, I'm using kafka, spark(Scala) and Hbase. Json data is feeded into the Kafka and it is streamed via Spark using scala code but I cant able to write it in Hbase tables.I had attached the logs for reference. 17/07/13 05:57:00 ERROR AsyncProcess: Failed to get region location org.apache.hadoop.hbase.TableNotFoundException: sample at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1264) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1162) at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:370) at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:321) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:158) at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:120) at org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1$anonfun$12$anonfun$apply$5.apply$mcV$sp(PairRDDFunctions.scala:1131) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1359) at org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1$anonfun$12.apply(PairRDDFunctions.scala:1131) at org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1$anonfun$12.apply(PairRDDFunctions.scala:1102) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) 17/07/13 05:57:00 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2) org.apache.hadoop.hbase.client.RetriesExhaustedWi Kindly suggest.

mathimurugan179 · ‎07-07-2017

Hi, In my case i need to get feed from json and that need to be streamed via spark - scala, so i need to know how to convert json data into case class using scala. For example: {"Id":"1","Name":"Babu","Address":"dsfjskkjfs"} to (1,babu,dsfjskkjfs}

mathimurugan179 · ‎07-05-2017

Hi Sindhu, By setting this property hive.metastore.try.direct.sql=false and hive.metastore.try.direct.sql.ddl=false, does not work.It listed with error which i had attached below, Then i Rolled back those config.Is there any other suggestion to drop hive tables. I'm using MYSQL -5.6.33 and Hive 1.2.1.2.4 hive-error.txt

mathimurugan179 · ‎07-02-2017

Can not drop the Hive tables, if i try it results in Hive server down, then after i restarting the Hive server i can able to access the hive shell. The same issue is posted in below link but no suggestion is given. https://community.hortonworks.com/questions/23690/dropping-hive-table-crashes-hiveserver2.html kindly guide

mathimurugan179 · ‎06-15-2017

Hi Team, In my use case we are going to implement Apache Kafka Plugin, i have gone through the site how to deploy this scenerio (https://phoenix.apache.org/kafka.html), but when trying this command, i need to know what should be specified in phoenix-kafka-<version>-minimal.jar to run this command Command :- HADOOP_CLASSPATH=$(hbase classpath):/path/to/hbase/conf hadoop jar phoenix-kafka-<version>-minimal.jar org.apache.phoenix.kafka.consumer.PhoenixConsumerTool --file /data/kafka-consumer.properties Please suggest what should be specified in this jar, whether path or i need to install jar.if so, please provide the link of jar file.

mathimurugan179 · ‎06-14-2017

hmas.txtregionserver.txtHi nshelke, I had Set this properties as you mentioned , but Hbase-Master and Region server are getting down and there is a backup process running behind this.It too fails. dfs.client.block.write.replace-datanode-on-failure.policy=ALWAYS dfs.client.block.write.replace-datanode-on-failure.best-effort=true Please find the attached Log of Hbase master and Regions server. Kindly suggest.

mathimurugan179 · ‎06-08-2017

capture.pngI'm running a long MR job which holds the capacity of 300GB, while running this job it frequently shows with this error and the job continues to run AttemptID:attempt_1495732157848_30821_m_000009_1 Timed out after 300 secs Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143. I have set this configuration, please guide me whether any configuration need to be set in addition to this. And while running this job my Hbase region server getting failed whether is this the reason? <code>yarn.nodemanager.vmem-check-enabled=false and <code>yarn.nodemanager.vmem-pmem-ratio=10:1

mathimurugan179 · ‎05-26-2017

I try setting Hbase HA , i set up HA by adding Hbase Master through service action.After setting I made my Active Master down for testing and my Standby Hbase master become active when i brought up my down master , both master came to Standby state.Then i Removed one Standby Hbase master.The remaining Hbase Master stays in Standby state.HDP version 2.4.2 and Ambari version 2.2.2.0 How to make it Active Hbase kindly suggest

Online	Offline
Last Visited	‎07-05-2018 11:03 AM

Date Registered	‎04-18-2017 05:09 AM
Last Visited	‎07-05-2018 11:03 AM
Total Messages Posted	39
Total Kudos Received	2

Re: comfirn hosts, ambari agent error,ambari agent...

How to change the ownership of a existing table in...

Adding a node in my cluster which is MIT kerberos ...

Hive Metastore DR

Sqoop Streaming Mechanism to check the Connectivit...

Sqoop with SAP HANA Connectivity Validation using ...

Re: comfirn hosts, ambari agent error,ambari agent...

Authentication Required- HBase Rest API HTTP Acces...

Unable to access HBase - Rest API when enabling K...

log4j:WARN No such property [maxFileSize] in org.a...

How to take Customized Backup in HBase

Maximum how many nods can we deploy our cluster wi...

Re: Hbase Regions Server getting down due to conne...

Re: Hbase Regions Server getting down due to conne...

Hbase Regions Server getting down due to connectio...

Is this Command help to export a particular rowkey...

How to take the Backup data for particular rowkey ...

Impact of OS upgrade from RHEL 6-->RHEL7 for hadoo...

Need suggestion regarding HDP and Ambari upgrade.

How to retrieve a particular Rowkey from Hbase tab...

Re: HBase Regions frequently getting failed, It sh...

HBase Regions frequently getting failed, It show H...

Re: Can't write the data into Hbase tables.

Can't write the data into Hbase tables.

How to convert json data into case class using sc...

Re: Droping Hive tables Crashed Hive Server, Cant ...

Droping Hive tables Crashed Hive Server, Cant drop...

Apache kafka plugin using Phoenix-consumer API

Re: Hbase Region server down frequently

Mapper is failing because of “Container killed by ...

Active HBase Master went into Standby