Member since
04-13-2016
422
Posts
150
Kudos Received
55
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1935 | 05-23-2018 05:29 AM | |
| 4970 | 05-08-2018 03:06 AM | |
| 1685 | 02-09-2018 02:22 AM | |
| 2716 | 01-24-2018 08:37 PM | |
| 6174 | 01-24-2018 05:43 PM |
04-11-2017
04:58 AM
1 Kudo
Hi, I'm trying out Hbase Stargate as a REST server that's bundled with my HBase installation. It's simple to get up and running, but I'm wondering how to view actual row data? We are using Kerberos and Ranger for authentication and authorization. When I perform a GET request in my REST client, I am returned with scrambled values or coded values: curl --negotiate -vi -u : -X GET -H "Accept: text/xml" http://localhost:8085/tablename/value Output: <code><?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CellSet>
<Row key="cm93MQ==">
<Cell column="Y2Y6QXJ0aWNsZUlE" timestamp="1357592601561">MQ==</Cell>
<Cell column="Y2Y6Q2FwRGF0ZQ==" timestamp="1357592790624">dG9kYXk=</Cell>
<Cell column="Y2Y6Q292ZXJhZ2U=" timestamp="1357592715068">U2FtcGxlIENvdmVyYWdl</Cell>
<Cell column="Y2Y6RW5jb2Rpbmc=" timestamp="1357592773388">VVRGLTg=</Cell>
<Cell column="Y2Y6TENvZGU=" timestamp="1357592744929">ZW4=</Cell>
<Cell column="Y2Y6TGFuZw==" timestamp="1357592760758">RW5nbGlzaA==</Cell>
<Cell column="Y2Y6TmF0aXZlTmFtZQ==" timestamp="1357592681884">U2FtcGxlIE5hdGl2ZSBOYW1l</Cell>
<Cell column="Y2Y6UHViRGF0ZQ==" timestamp="1357592786561">dG9kYXk=</Cell>
<Cell column="Y2Y6U291cmNlSUQ=" timestamp="1357592732763">MQ==</Cell>
<Cell column="Y2Y6U291cmNlSUQ=" timestamp="1357592663427">U2FtcGxlIFNvdXJjZSBJRA==</Cell>
<Cell column="Y2Y6U291cmNlVHlwZQ==" timestamp="1357592701595">U2FtcGxlIFNvdXJjZSBUeXBl</Cell>
<Cell column="Y2Y6VGV4dA==" timestamp="1357592802878">U2FtcGxlIFRleHQ=</Cell>
<Cell column="Y2Y6VGl0bGU=" timestamp="1357592629292">U2FtcGxlIFRpdGxl</Cell>
<Cell column="Y2Y6VHJUaXRsZQ==" timestamp="1357592640605">U2FtcGxlIFRyVGl0bGU=</Cell>
<Cell column="Y2Y6VXJs" timestamp="1357592616187">aHR0cDovL3d3dy5nb29nbGUuY29t</Cell>
</Row>
</CellSet> I did reference the WIKI which also shows examples with scrambled results: http://wiki.apache.org/hadoop/Hbase/Stargate#A5, but I cannot find any information on how to decode it. Any help is higly appricated and Thanks in advance...!
... View more
Labels:
- Labels:
-
Apache HBase
03-31-2017
02:52 PM
@Namit Maheshwari "WARN tools.DistCp: Failed to compute snapshot diff on hdfs://hadoop.hortonworks.com:8020/tmp/sbandaru" Above one is part of the error message which is my target location, why it's trying to find the snapshot in target location ?
... View more
03-31-2017
03:52 AM
@Namit Maheshwari
Below is the requested information. [sbandaru@hadoop ~]$ hdfs dfs -ls /user/sbandaru/.snapshot
Found 3 items
drwxr-x--- - sbandaru sbandaru 0 2017-03-30 11:38 /user/sbandaru/.snapshot/afterdistcp
drwxr-x--- - sbandaru sbandaru 0 2016-11-08 19:57 /user/sbandaru/.snapshot/sbandaru
drwxr-x--- - sbandaru sbandaru 0 2016-11-08 19:57 /user/sbandaru/.snapshot/sbandaru2
[sbandaru@hadoop ~]$
[sbandaru@hadoop ~]$ hadoop --loglevel DEBUG distcp -update -diff sbandaru afterdistcp /user/sbandaru hdfs://hadoop.hortonworks.com:8020/tmp/sbandaru
I have created snapshot on /user/sbandaru directory then I'm trying to get difference of old and new snapsort and move that difference to a location /tmp/sbnadaru.
... View more
03-31-2017
02:50 AM
@Namit Maheshwari Thanks for the quick response, I have tried the same way but I'm getting below error message. Any help is highly appreciated. 17/03/30 21:39:38 WARN retry.RetryInvocationHandler: Exception while invoking class org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport over null. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.SnapshotException): Cannot find the snapshot of directory /tmp/sbandaru with name sbandaru
at org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.getSnapshotByName(DirectorySnapshottableFeature.java:285)
at org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.computeDiff(DirectorySnapshottableFeature.java:257)
at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:372)
at org.apache.hadoop.hdfs.server.namenode.FSDirSnapshotOp.getSnapshotDiffReport(FSDirSnapshotOp.java:155)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:7674)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1792)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolServerSideTranslatorPB.java:1149)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2273)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2269)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2267)
17/03/30 21:57:46 WARN tools.DistCp: Failed to compute snapshot diff on hdfs://hadoop.hortonworks.com:8020/tmp/sbandaru
org.apache.hadoop.hdfs.protocol.SnapshotException: Cannot find the snapshot of directory /tmp/sbandaru with name sbandaru
at org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.getSnapshotByName(DirectorySnapshottableFeature.java:285)
at org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.computeDiff(DirectorySnapshottableFeature.java:257)
at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:372)
at org.apache.hadoop.hdfs.server.namenode.FSDirSnapshotOp.getSnapshotDiffReport(FSDirSnapshotOp.java:155)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:7674)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1792)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolServerSideTranslatorPB.java:1149)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
... View more
03-30-2017
09:15 PM
Hi, Can anyone provide me syntax and sample example for checking the difference between two snapshot and move that difference data to target cluster using distcp? AIM: I have two clusters clusterA and ClusterB. I have recently built ClusterB and moving all the data from clusterA to clusterB. Before moving the data I have taken the snapshot on cluster A. During the interval of transferring the data, as the cluster A is still in active state the data got changed. Now I want to move only changed data from cluster A to cluster B. can someone provide me syntax with simple example like how can I get difference and move the changed data. Thanks in advance.
... View more
Labels:
- Labels:
-
Apache Hadoop
03-28-2017
09:23 PM
@Deepesh Here is the https://issues.apache.org/jira/browse/SPARK-19970
... View more
03-24-2017
06:04 PM
@Deepesh@Deepesh Anyways this issue is confirmed as the bug, which is going to get fixed in Spark 2.1.+. Posted , so that someone can refer for future instead of spending their time.
... View more
03-24-2017
06:01 PM
@Deepesh Everything is from Spark-SQL only.
... View more
03-24-2017
03:32 PM
Hi, With the Spark-SQL shell, kicking it off with settings along the lines of: /usr/hdp/current/spark-client/bin/spark-sql --master yarn-client --queue abc --num-executors 5 --executor-memory 10g --executor-cores 10 Running the following command to create the table with minimal configuration: create table [DB].[table_name] stored as orc as select…. Running into errors when attempting to grant select access to others for the table created in SQL Standard Authorization. Looking into it, notice: Spark created table: owner: username@realm Any regular Hive created table: owner: username While using create table t1 as select from another table, the table t1 has owner name with the realm name included. Simple create table t1 without selecting from another table is working fine.
... View more
Labels:
- Labels:
-
Apache Spark
03-24-2017
03:01 PM
@Uvaraj Seerangan Please verify the proxy parameters, follow below link to set them up. http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-views/content/configuring_tez_in_your_cluster.html Hope this helps you.
... View more