Using Cloud Asset Inventory feeds to monitor changes in permissions of storage buckets (part 1)

Posted on Updated on

This is the first of several articles I will be publishing related to Google Cloud Platform (GCP).

Cloud Asset Inventory has a very useful feature to monitor resource and policy changes to your assets. This feature is called Cloud Asset Inventory feeds.

A Cloud Asset Inventory feed is used to publish asset updates to Pub/Sub topics, enabling us to receive real-time notifications of interesting changes in the inventory.

By creating and subscribing to an asset feed (via a Pub/Sub subscription), you’ll receive immediate updates about any changes for the desired asset names or asset types.

In this article we explore how we can use this feature to receive notifications when a specific role is granted to the special identifier allUsers in any GCP Storage Bucket.

Overview

To receive real-time notifications about resource and policy changes, you need to create the following:

  • Pub/Sub topic
  • Pub/Sub subcription
  • Cloud Asset Inventory feed

Process

Create the Pub/Sub topic and subscription

A pre-requisite to create a Cloud Asset feed is to have an existing Pub/Sub topic to publish the changes to.

First, we create a Pub/Sub topic called asset-feed-topic:

gcloud pubsub topics create asset-feed-topic

Now, we create a subscription called asset-subscription to receive the messages from the topic:

gcloud pubsub subscriptions create asset-subscription \
  --topic=asset-feed-topic \
  --ack-deadline=30 \
  --expiration-period=1d \
  --retain-acked-messages

We are now ready to create a configure a Cloud Asset Inventory feed.

Creating a feed

We can create a feed to receive notifications for any change made to storage buckets

For the purposes of this article, we want to get notifications when the role roles/storage.objectViewer is granted to allUsers.

In the following example we assume you already configured the environment variable $PROJECT_ID with the name of your GCP project:

gcloud asset feeds create gs-feed \
  --project=$PROJECT_ID \
  --asset-types="storage.googleapis.com/Bucket" \
  --pubsub-topic="projects/$PROJECT_ID/topics/asset-feed-topic"

This, however is not exactly what we need: Because we want to be notified when the buckets permissions (IAM) are changed, we need to get additional information in the notification.

The content sent in the notification is controlled by the --content-type parameter. In this case, we want to set the content type to iam-policy so we can inspect any changes to the IAM policy in the bucket. You can see all the available content types in the gcloud asset feeds create reference.

gcloud asset feeds create gs-feed \
  --project=$PROJECT_ID \
  --asset-types="storage.googleapis.com/Bucket" \
  --pubsub-topic="projects/$PROJECT_ID/topics/asset-feed-topic" \
  --content-type=iam-policy

If the feed was already created, you can change its content type by updating it:

gcloud asset feeds update gs-feed --project=$PROJECT_ID --content-type=iam-policy

Testing the feed

We will now proceed to:

  • Create a bucket
  • Trigger a notification by adding an IAM policy binding to the bucket
  • Pull the Pub/Sub subscription to get the notification

1. Let’s create a bucket with today’s date in the name:

BUCKET=gs://my-test-bucket-`date +%Y%m%d`
gcloud storage buckets create $BUCKET

2. Trigger a notification by adding an IAM policy binding (permission)

gcloud storage buckets add-iam-policy-binding $BUCKET \
--member=allUsers --role=roles/storage.objectViewer

3. Pull the notification message from the Pub/Sub subscription:

gcloud pubsub subscriptions pull asset-subscription --format="value(message.data)" --limit=10 --auto-ack

Because the feed’s content type was set to iam-policy, the message has the field iamPolicy of type Policy:

{
  "asset": {
    "ancestors": [ <REDACTED> ],
    "assetType": "storage.googleapis.com/Bucket",
    "iamPolicy": {
      "bindings": [
        <REDACTED>,
        {
          "members": [
            "allUsers"
          ],
          "role": "roles/storage.objectViewer"
        }
      ],
      "etag": "BwX355oZR9I="
    },
    "name": "//storage.googleapis.com/PROJECT_NAME",
    "updateTime": "2023-03-27T20:29:38.407378Z"
  },
  "priorAssetState": "PRESENT",
  "window": {
    "startTime": "2023-03-27T20:29:38.407378Z"
  }
}

Conclusions

In this session we learned how to create a Cloud Asset Inventory feed to monitor changes to a Cloud Storage Bucket.

We also modified the feed to send the related IAM information when a permission is changed.

In the next article, we’ll learn how to use conditions to monitor for specific IAM changes (like getting notifications when a specific role is assigned to a specific user).