Google Cloud Platform
Hide
Compute Engine
Write Feedback

Activity Logs

Google Compute Engine provides activity logs through the Google Cloud Logging service. Activity logs contain details of different events that occur in your Compute Engine project—specifically, information about user-initiated actions, such as API requests, and certain system events like live migration and virtual machine restarts. For example, an activity log can provide details such as the start and end time of an API request, the request body details, the authorized user who made the API request, and the request endpoint.

You can download activity logs into a Google Cloud Storage bucket, stream the logs into Google BigQuery, or view them in the Logs Viewer. This document specifically discusses how to enable, access, and read activity logs that have been exported into Google Cloud Storage.

Activity logs do not provide billing or usage information about a project, such as how long a virtual machine has been running or how much it costs. For billing logs, see the billing export feature. For usage logs, see the usage logs.

Contents

Enabling activity logs

Activity logging is enabled by default for all Compute Engine projects. Unlike virtual machine logging, you do not need to install any additional software on your virtual machines to enable activity logs.

Accessing activity logs

You can access a project's activity logs through the Logs Viewer in the Google Developers Console, export your logs into Google Cloud Storage, or stream them directly into Google BigQuery.

  • To use the Logs Viewer, see the Logs Viewer documentation. When in the logs viewer pane, select Compute Engine from the drop-down menu and then select activity_log from the All logs drop-down to see Compute Engine activity logs.
  • To enable exporting your logs into Google Cloud Storage or streaming logs into Google BigQuery, see Preparing to Export Logs.

There can be a delay between when you first enable logs exporting to when your first log is delivered. For details, see Exported logs availability.

Logs directory structure

When you export logs into Google Cloud Storage, the log files are stored in the structure described by the Log entries in Google Cloud Storage documentation.

In summary, log files are stored using the following directory structure:

<bucket>/compute.googleapis.com/activity_log/<year>/<month>/<day>

Each log file is named using the following format:

<start-time>_<end-time>_<shard>.json
  • <start-time> and <end-time> is the start and end type of the event, in UTC. The times are are encoded in the HH:MM:SS format. Start and end ranges are mutually exclusive, so that the end time for HH:00:00 is HH:59:59.
  • <shard> has the form S### or A### with ### representing a non-zero-padded number. Each shard represents either the original log file (collected between start-time and end-time) or an appended addition to the original log file. Shards prefixed by S represent the original file. Shards prefixed by A are addendum information, containing late arriving log information.

For example, the following is an example log file name stored in a Google Cloud Storage bucket named my-bucket:

my-bucket/compute.googleapis.com/activity_log/2014/10/31/20:00:00_20:59:59_S0.json

In BigQuery, activity logs are stored in a set of tables, one table for each log type and day, and the tables are named using the following format:

<dataset>.compute_googleapis_com_activity_log_YYYYMMDD

For information on how to query activity logs in BigQuery, see Log entries in Google BigQuery.

Reading activity logs

Activity logs are structured as described in the LogEntry type documentation.

Compute Engine activity logs have:

  • compute.googleapis.com/activity_log as value of the log field
  • metadata, which describes common information such as timestamp
  • textPayload, which contains the specific contents of the log entry

You can get more details about the common fields provide with every log entry from the LogEntry description, but the payload contents of activity log entries are specific to Compute Engine and are described below.

Payload contents

The contents of a log entry is a string-encoded JSON value stored in the textPayload field. The textPayload can contain any of the following information:

Field Type Description
actor string Email of user or service account performing the operation. This is the same as the user_id.
error string Provides any error details if an error occured during this event. This is omitted if there were no errors. Errors will usually prevent a request from completing successfully. You can use this field to debug a failed request.
event_subtype string Describes the specific subtype as an API method.

For example, a request to insert a new instance appears as compute.instances.insert, and a request to delete an Address resource appears as compute.addresses.delete.

See the API reference for a comprehensive list of API methods.
event_timestamp_us timestamp The timestamp of the logged event in microseconds since standard epoch. This is the same as metadata.timestamp.
event_type string Describes the general event type.

This can be one of:

  • GCE_API_CALL: Indicates a REST API call
  • GCE_OPERATION_DONE: Once an API request completes, whether successful or not, this event type is logged.
info string An optional field with additional information if applicable. This field is omitted if there is no additional information to show.
operation string When a API request is made to update or change any resources, a corresponding operation object is created to track the request to completion. This property describes the operation object for this event, providing information such as the operation ID, the zone or region of the operation, and the operation ID.

Operations can be a zone operation, a region opeation, or a global operation, depending on the resource the operation is modifying.
request JSON Contains the original JSON API request body.

resource JSON Describes the particular resource that is being modified by this event. For example, an virtual machine instance is considered a resource and an example resource property for a virtual machine would look like the following: 
\"resource\":{
      \"type\":\"instance\",
      \"name\":\"my-new-instance\",
      \"id\":\"4762502054210132951\",
      \"zone\":\"us-central1-a\"
    }

A list of resource types is described in detail in the API reference.

Note: If an action affects multiple resources, there may be multiple log entries with the same trace_id.

trace_id number A system-provided trace ID used to group related logs that are triggered by a single action. For example: 
\"trace_id\":\"operation-1422925655708-50e24b2401160-6e523c4d-abfe68f9\"
user_agent string Describes the client that performed this request. For example, if you used the Java client library to make a request, the user agent would be Google-API-Java-Client.
version string The current log format version indicates of the Compute Engine log schema. The current version is 1.0.

Note: The Compute Engine API versioning is separate from the log format versioning.
warning string Provides any warning details if any warnings occured during this event. A warning is informationl and does not affect the request, unlike errors.

Sample log entry

For example, a pair of sample log entries describing an API request to create a virtual machine looks like the following:

{
  "metadata":{
    "severity":"INFO",
    "projectId":"",
    "serviceName":"compute.googleapis.com",
    "region":"",
    "zone":"us-central1-a",
    "userId":"user@example.com",
    "labels":{
      "compute.googleapis.com/resource_zone":"us-central1-a",
      "compute.googleapis.com/resource_name":"my-new-instance",
      "compute.googleapis.com/resource_type":"instance",
      "compute.googleapis.com/resource_id":"0"
    },
    "timestamp":"2015-02-25T22:21:03.252329Z"
  },
  "textPayload":"{
    \"version\":\"1.0\",
    \"trace_id\":\"operation-1234567890987-50ff10cecd583-54eeddf0-2c78e487\",
    \"event_timestamp_us\":1424902863252329,
    \"event_type\":\"GCE_API_CALL\",
    \"event_subtype\":\"compute.instances.insert\",
    \"resource\":{
      \"type\":\"instance\",
      \"name\":\"my-new-instance\",
      \"id\":\"0\",
      \"zone\":\"us-central1-a\"
    },
    \"actor\":{
      \"user\":\"user@example.com\"
    },
    \"ip_address\":\"\",
    \"user_agent\":\"apitools-client/1.0\",
    \"request\":{
      \"url\":    \"https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances?alt=json\",
      \"body\":\"    {
        \"machineType\":      \"https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/machineTypes/n1-standard-1\",
        \"name\":\"my-new-instance\",
        \"disks\":[
          {
            \"initializeParams\":{
              \"sourceImage\": \"https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/backports-debian-7-wheezy-v20150127\"
            },
            \"boot\":true,
            \"mode\":\"READ_WRITE\",
            \"autoDelete\":true,
            \"type\":\"PERSISTENT\"
          }
        ],
        \"scheduling\":{
          \"automaticRestart\":true
        },
        \"canIpForward\":false,
        \"serviceAccounts\":[
          {
            \"scopes\":[
              \"https://www.googleapis.com/auth/devstorage.read_only\",
              \"https://www.googleapis.com/auth/logging.write\"
            ],
            \"email\":\"default\"
          }
        ],
        \"networkInterfaces\":[
          {
            \"accessConfigs\":[
              {
                \"type\":\"ONE_TO_ONE_NAT\",
                \"name\":\"external-nat\"
              }
            ],
            \"network\": \"https://www.googleapis.com/compute/v1/projects/example-project/global/networks/default\"
          }
        ],
        \"metadata\":{

        }
      }    \"
    },
    \"operation\":{
      \"type\":\"operation\",
      \"name\":\"operation-1234567890987-50ff10cecd583-54eeddf0-2c78e487\",
      \"id\":\"1234567890987654321\",
      \"zone\":\"us-central1-a\"
    }
  }",
  "insertId":"2015-02-25|14:21:03.303071-08|10.42.202.69|1358965990",
  "log":"compute.googleapis.com/activity_log"
}
{
  "metadata":{
    "severity":"INFO",
    "projectId":"",
    "serviceName":"compute.googleapis.com",
    "region":"",
    "zone":"us-central1-a",
    "userId":"user@example.com",
    "labels":{
      "compute.googleapis.com/resource_zone":"us-central1-a",
      "compute.googleapis.com/resource_name":"my-new-instance",
      "compute.googleapis.com/resource_type":"instance",
      "compute.googleapis.com/resource_id":"11835120265051257771"
    },
    "timestamp":"2015-02-25T22:21:24.544769Z"
  },
  "textPayload":"{
    \"version\":\"1.0\",
    \"trace_id\":\"operation-1234567890987-50ff10cecd583-54eeddf0-2c78e487\",
    \"event_timestamp_us\":1424902884544769,
    \"event_type\":\"GCE_OPERATION_DONE\",
    \"event_subtype\":\"compute.instances.insert\",
    \"resource\":{
      \"type\":\"instance\",
      \"name\":\"my-new-instance\",
      \"id\":\"11835120265051257771\",
      \"zone\":\"us-central1-a\"
    },
    \"actor\":{
      \"user\":\"user@example.com\"
    },
    \"operation\":{
      \"type\":\"operation\",
      \"name\":\"operation-1234567890987-50ff10cecd583-54eeddf0-2c78e487\",
      \"id\":\"1234567890987654321\",
      \"zone\":\"us-central1-a\"
    }
  }",
  "insertId":"2015-02-25|14:21:24.683031-08|10.60.29.134|1543517397",
  "log":"compute.googleapis.com/activity_log"
}

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies.