How to access fine grained audit logging in CML

[#1]

CML provides some nice high-level session, model, and experiment auditing. But sometimes a situation arises where you would like audit logs at the level of file modification and user logins. This article provides a quick walkthrough of accessing the underlying RDBMS for this information. This page gives details on the tables and information available. This guide presupposes you are using AWS.

Thanks to @fletch_jeff for the help.

Step 1 - Install aws-iam-authenticator

Install according to your OS as per these instructions.

Step 2 - Grab your User ARN

From the command line:

==> aws sts get-caller-identity
{
    "Account": "012345678910", 
    "UserId": "ABBBBBBBBB BXXXXXXXXXX", 
    "Arn": "arn:aws:iam::012345678910:user/cperro"
}

From AWS Console

1. Open the AWS Console.
2. Navigate to IAM.
3. Navigate to Users.
4. Select your user name.
5. Copy the ARN.
   

Step 3 - Add your ARN to your Workspace

1. Navigate to Machine Learning Workspaces in CDP.
2. Click on the Options icon (three vertical dots) for your workspace and select the Manage Remote Access option:
   
3. Paste your ARN and click Grant Access.
4. Click Download Kubeconfig:
   

Step 4 - Query the Audit Database

Note: the --kubeconfig file should be the KubeConfig you downloaded in the previous step

#List the tables in the database - interactive shell for db-0 pod in mlx namespace -> execute the psql command with user "sense"
kubectl --kubeconfig ~/Downloads/perro-small-workspace-kubeconfig.yaml exec -it db-0 -n mlx -- psql -P pager=off -U sense -c "\l”

#Show the last 10 user events
kubectl --kubeconfig ~/Downloads/perro-small-workspace-kubeconfig.yaml exec -it db-0 -n mlx -- psql -P pager=off -U sense -c "SELECT id, user_id, event_name, description, created_at FROM user_events order by created_at desc limit 10;”