メインコンテンツまでスキップ
Sumo Logic Japanese

Google Compute Engine のログの収集

Google Compute Engine 用 Sumo Logic アプリケーションのログ収集を設定する手順。

このページでは、Google Cloud Platform (GCP) サービスからログを取り込むための Sumo パイプラインについて説明し、Google Compute Engine からログを収集する手順を紹介します。

Collection process for GCP services

The key components in the collection process for GCP services are:  Google Stackdriver, Google Cloud Pub/Sub, and Sumo’s Google Cloud Platform (GCP) source running on a hosted collector. 

The integration works like this: Google Stackdriver collects logs from GCP services. Once you’ve configured the pipeline shown below, the logs collected by Stackdriver will be published to a Google Pub/Sub topic. A Sumo GCP source on a hosted collector subscribed to that topic ingests the logs into Sumo.

monitor-gcp-services.png 

The configuration process is as follows. 

  1. Configure a GCP source on a hosted collector. You'll obtain the HTTP URL for the source, and then use Google Cloud Console to register the URL as a validated domain.  
  2. Create a topic in Google Pub/Sub and subscribe the GCP source URL to that topic.
  3. Create an export of GCP logs from Stackdriver. Exporting involves writing a filter that selects the log entries you want to export, and choosing a Pub/Sub as the destination. The filter and destination are held in an object called a sink. 

See the sections below for instructions.

Stackdriver からの Google Compute Engine ログのエクスポートの作成

  1. [Logging (ログ)] に移動して [Exports (エクスポート)] をクリックします。
    gcp6.png
  2. [Create Export (エクスポートの作成)] をクリックします。
    gcp7.png
  3. GCP サービスを選択してログを絞り込みます。sink を作成するお勧めの GCP サービスは "GCE VM Instance" であり、これによってサービスのログが Sumo Logic に送信されます。右側の [Edit Export (エクスポートの編集)] ウィンドウで次の操作を実行します。

    1. [Sink Name (Sink 名)] を設定します。たとえば、"gce-vm-instance" です。
    2. [Sink Service (Sink サービス)] として [Cloud Pub/Sub] を選択します。
    3. [Sink Destination (Sink ターゲット)] を作成したばかりの Pub/Sub トピックに設定します。たとえば、"pub-sub-logs" です。
    4. [Create Sink (Sink の作成)] をクリックします。
      gcp8.png

ログ メッセージのサンプル

{
 "message":{
   "data":{
     "insertId":"55E9891F381C2.A6AC1EA.F3043722",
     "logName":"projects/wk-dev/logs/cloudaudit.googleapis.com%2Factivity",
     "operation":{
       "first":true,
       "id":"operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
       "producer":"compute.googleapis.com"
     },
     "protoPayload":{
       "@type":"type.googleapis.com/google.cloud.audit.AuditLog",
       "authenticationInfo":{
         "principalEmail":"service-287993422434@dataflow-service-producer-prod.iam.gserviceaccount.com"
       },
       "authorizationInfo":[{
         "granted":true,
         "permission":"compute.instances.delete"
       }],
       "methodName":"v1.compute.instances.delete",
       "requestMetadata":{
         "callerSuppliedUserAgent":"Managed Infrastructure Mixer Client"
       },
       "resourceName":"projects/287993422434/zones/us-central1-f/instances/permissionlogs-yuanwang-1-11221246-d0b6-harness-p548",
       "response":{
         "@type":"compute.googleapis.com/operation",
         "id":"6917821783428586027",
         "insertTime":"2017-11-22T12:57:40.084-08:00",
         "name":"operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
         "operationType":"delete",
         "progress":"0",
         "selfLink":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f/operations/operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
         "status":"PENDING",
         "targetId":"7642006033207418043",
         "targetLink":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f/instances/permissionlogs-yuanwang-1-11221246-d0b6-harness-p548",
         "zone":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f"
       },
       "serviceName":"compute.googleapis.com"
     },
     "receiveTimestamp":"2017-11-22T20:57:41.0202444Z",
     "resource":{
       "labels":{
         "instance_id":"7642006033207418043",
         "project_id":"wk-dev",
         "zone":"us-central1-f"
       },
       "type":"gce_instance"
     },
     "severity":"NOTICE",
     "timestamp":"2017-11-22T20:57:39.896Z"
   },
   "attributes":{
     "logging.googleapis.com/timestamp":"2017-11-22T20:57:39.896Z"
   },
   "message_id":"174545382671298",
   "messageId":"174545382671298",
   "publish_time":"2017-11-22T20:57:42.118Z",
   "publishTime":"2017-11-22T20:57:42.118Z"
 },
 "subscription":"projects/wk-dev/subscriptions/sumo-test"
}

クエリのサンプル

上位 10 人のユーザ

_collector="HTTP Source for GCP Pub/Sub" logName resource timestamp
| json "message.data.resource.type" as type 
| parse regex "\s+\"logName\":\"(?<log_name>\S+)\"" 
| where type = "gce_instance" and log_name matches "projects/*/logs/cloudaudit.googleapis.com%2Factivity"
| parse regex "\s+\"resourceName\":\"projects/\S+/zones/(?<zone>\S+)/instances/(?<instance>\S+)\""
| json "message.data.resource.labels" as labels
| json field=labels "project_id" as project
| json "message.data.protoPayload.authenticationInfo.principalEmail" as user
| count as requests by user
| sort by requests
| limit 10

 

  • この記事は役に立ちましたか?