How to check the RocksDb data for Ozone using ldb tool?

[#1]

We will be using the Ozone ldb CLI tool to view the RocksDb table, which stores all the information in various tables. Here I will be showing how to check the table data that it has created for the filesystem in Ozone.

[root@ccycloud-1 data]# pwd
/var/lib/hadoop-ozone/scm/data
[root@ccycloud-1 data]# pwd
/var/lib/hadoop-ozone/om/data

We need to traverse to the Ozone metadata directory through Cloudera Manager UI > configuration and search for "metadata".

[root@ccycloud-1 data]#  ozone debug ldb --db=om.db ls
default
fileTable
principalToAccessIdsTable
deletedTable
userTable
s3SecretTable
transactionInfoTable
openKeyTable
snapshotInfoTable
directoryTable
prefixTable
multipartInfoTable
volumeTable
tenantStateTable
deletedDirectoryTable
tenantAccessIdTable
openFileTable
snapshotRenamedTable
dTokenTable
metaTable
keyTable
bucketTable

Under the directory /var/lib/hadoop-ozone/scm/data we need need to ls to list out the tables.

[root@ccycloud-1 data]# ozone debug ldb --db=scm.db ls
default
sequenceId
revokedCertsV2
pipelines
crls
crlSequenceId
meta
containers
validCerts
validSCMCerts
scmTransactionInfos
deletedBlocks
statefulServiceConfig
move
revokedCerts

Use the column_family filter to see each of the tables under scm. db and om. db
Example: Like for userTable under om.db use "--column_family=userTable":

[root@ccycloud-1 data]# ozone debug ldb --db=om.db scan --column_family=userTable
24/01/19 09:19:28 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/01/19 09:19:28 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/01/19 09:19:28 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/01/19 09:19:28 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/01/19 09:19:28 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
{ "hdfs": {
  "unknownFields": {
    "fields": {}
  },
  "bitField0_": 3,
  "volumeNames_": [
    "s3v"
  ],
  "objectID_": -4611686018427388160,
  "updateID_": 18014398509481983,
  "memoizedIsInitialized": 1,
  "memoizedSerializedSize": -1,
  "memoizedHashCode": 0,
  "memoizedSize": -1
} }

Example: Like for container under scm.db use  "--column_family=containers"

[root@ccycloud-1 data]# ozone debug ldb --db=scm.db scan --column_family=containers
24/04/15 11:42:40 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/04/15 11:42:40 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/04/15 11:42:40 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/04/15 11:42:40 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
24/04/15 11:42:40 INFO helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline = true
{ {
  "id": 1
}: {
  "state": "OPEN",
  "stateEnterTime": {
    "seconds": 1713164197,
    "nanos": 579000000
  },
  "pipelineID": {
    "id": "8d897f9a-0d35-42e1-b3a7-e010c56198f0"
  },
  "replicationConfig": {
    "replicationFactor": "THREE"
  },
  "clock": {
    "zone": {
      "totalSeconds": 0
    }
  },
  "usedBytes": 2772,
  "numberOfKeys": 44,
  "lastUsed": {
    "seconds": 1713181360,
    "nanos": 301000000
  },
  "owner": "om81",
  "containerID": {
    "id": 1
  },
  "deleteTransactionId": 267,
  "sequenceId": 514
}, {
  "id": 2
}: {
  "state": "OPEN",
  "stateEnterTime": {
    "seconds": 1713164247,
    "nanos": 612000000
  },
  "pipelineID": {
    "id": "3a045dc0-ad58-4710-8d06-70fa1b8b9f7f"
  },
  "replicationConfig": {
    "replicationFactor": "THREE"
  },
  "clock": {
    "zone": {
      "totalSeconds": 0
    }
  },
  "usedBytes": 2520,
  "numberOfKeys": 40,
  "lastUsed": {
    "seconds": 1713181360,
    "nanos": 307000000
  },
  "owner": "om81",
  "containerID": {
    "id": 2
  },
  "deleteTransactionId": 265,
  "sequenceId": 441
}, {
  "id": 3
}: {
  "state": "OPEN",
  "stateEnterTime": {
    "seconds": 1713164309,
    "nanos": 369000000
  },
  "pipelineID": {
    "id": "8d897f9a-0d35-42e1-b3a7-e010c56198f0"
  },
  "replicationConfig": {
    "replicationFactor": "THREE"
  },
  "clock": {
    "zone": {
      "totalSeconds": 0
    }
  },
  "usedBytes": 2772,
  "numberOfKeys": 44,
  "lastUsed": {
    "seconds": 1713181360,
    "nanos": 308000000
  },
  "owner": "om81",
  "containerID": {
    "id": 3
  },
  "deleteTransactionId": 268,
  "sequenceId": 518
}, {
  "id": 4
}: {
  "state": "OPEN",
  "stateEnterTime": {
    "seconds": 1713164366,
    "nanos": 431000000
  },
  "pipelineID": {
    "id": "3a045dc0-ad58-4710-8d06-70fa1b8b9f7f"
  },
  "replicationConfig": {
    "replicationFactor": "THREE"
  },
  "clock": {
    "zone": {
      "totalSeconds": 0
    }
  },
  "usedBytes": 2520,
  "numberOfKeys": 40,
  "lastUsed": {
    "seconds": 1713181360,
    "nanos": 310000000
  },
  "owner": "om81",
  "containerID": {
    "id": 4
  },
  "deleteTransactionId": 266,
  "sequenceId": 437
} }