WizTelemetry Logging

Overview

WizTelemetry Logging is an extension component in the KubeSphere Observability Platform for log collection, processing, and storage.

When to Use

• Installing or configuring the WizTelemetry Logging extension
• Understanding log collection architecture (container logs + disk log collection)
• Using the log query API to query logs

Components

| Component | Description | Default Enabled | |-----------|-------------|-----------------| | vector-logging | Container log collection (collects stdout/stderr from Docker/Containerd) | true | | logsidecar-injector | Disk log collection (collects logs from files inside containers) | false |

Dependencies

• WizTelemetry Platform Service (whizard-telemetry): Required
• WizTelemetry Data Pipeline (vector): Required
• OpenSearch (opensearch): Required

Installation

Prerequisites

REQUIRED: Complete all steps in order before generating InstallPlan.

Step 1: Get Available Clusters and Confirm Target

⚠️ CRITICAL: DO NOT proceed until target clusters are determined.

Step 1.1: Get available clusters

bash
kubectl get clusters -o jsonpath='{.items[*].metadata.name}'

Step 1.2: Determine target clusters

• If user explicitly specified target clusters in the request → Use those clusters directly, proceed to Step 2
• If user did NOT specify target clusters → Ask user to confirm which clusters to deploy to, then proceed to Step 2

Ask user (if not specified):

Available clusters: host, dev
Which clusters do you want to deploy WizTelemetry Logging to?

Step 2: Get Latest Version (if not provided by user)

MUST do this to get the latest version:

bash
kubectl get extensionversions -l kubesphere.io/extension-ref=whizard-logging -o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V | tail -1

This outputs the latest version (e.g., 1.4.0). Note this down - you'll use it in the InstallPlan.

Install WizTelemetry Logging

⚠️ IMPORTANT: Complete prerequisite steps BEFORE this step.

Based on your selections:

• Target clusters: User-confirmed cluster names

⚠️ CRITICAL: InstallPlan metadata.name MUST be whizard-logging. DO NOT use any other name.

⚠️ CRITICAL: config field is YAML format. You MUST:

• Use the config structure exactly as shown in the template
• DO NOT add configuration fields that are not shown in the template
• DO NOT modify the structure or hierarchy

⚠️ CRITICAL: All placeholders MUST be replaced with actual values. DO NOT leave them as placeholders.

Template

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>  # From Step 2
  enabled: true
  upgradeStrategy: Manual
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Replace placeholders:

• <VERSION>: From Step 2 (e.g., 1.4.0)
• <TARGET_CLUSTERS>: User-confirmed cluster names

Note: OpenSearch sink configuration (endpoints, auth) is provided by the vector extension. Make sure vector is installed and configured with OpenSearch before installing logging.

Enable Disk Log Collection

To enable disk log collection, add logsidecar-injector to the config:

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>  # From Step 2
  enabled: true
  upgradeStrategy: Manual
  config: |
    logsidecar-injector:
      enabled: true
    vector-logging:
      filter:
        extraLabelSelector: "app.kubernetes.io/name!=kube-events-exporter"
      calico:
        enabled: true
      systemd:
        docker:
          enabled: true
        kubelet:
          enabled: true
      sinks:
        opensearch:
          enabled: true
          index:
            prefix: "{{ .cluster }}-logs"
            timestring: "%Y.%m.%d"
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Configuration Parameters

Logsidecar Injector Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | logsidecar-injector.enabled | bool | false | Enable disk log collection | | logsidecar-injector.sidecar.sidecarType | string | vector | Sidecar type | | logsidecar-injector.resources.limits.cpu | string | 100m | CPU limit | | logsidecar-injector.resources.limits.memory | string | 100Mi | Memory limit | | logsidecar-injector.resources.requests.cpu | string | 10m | CPU request | | logsidecar-injector.resources.requests.memory | string | 10Mi | Memory request |

Vector Logging Parameters

Filter Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | vector-logging.filter.extraLabelSelector | string | "app.kubernetes.io/name!=kube-events-exporter" | Extra label selector | | vector-logging.filter.extraNamespaceLabelSelector | string | "" | Extra namespace label selector | | vector-logging.filter.includeNamespaces | list | ] | List of namespaces to collect | | vector-logging.filter.excludeNamespaces | list | ] | List of namespaces to exclude |

Calico Log Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | vector-logging.calico.enabled | bool | true | Enable Calico log collection | | vector-logging.calico.logPath | list | "/var/log/calico/cni/cni.log"] | Calico log paths |

Systemd Log Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | vector-logging.systemd.docker.enabled | bool | true | Enable Docker systemd log collection | | vector-logging.systemd.kubelet.enabled | bool | true | Enable Kubelet systemd log collection | | vector-logging.systemd.directory | string | /var/log/journal | Systemd journal directory |

OpenSearch Sink Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | vector-logging.sinks.opensearch.enabled | bool | true | Enable OpenSearch sink | | vector-logging.sinks.opensearch.index.prefix | string | "{{ .cluster }}-logs" | Index prefix | | vector-logging.sinks.opensearch.index.timestring | string | "%Y.%m.%d" | Index time format |

ISM Policy Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | vector-logging.ism_policy.enable | bool | false | Enable Index State Management policy | | vector-logging.ism_policy.min_index_age | string | "7d" | Minimum index retention period |

Log Query API

Query Logs

bash
curl -X GET "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/logging.kubesphere.io/v1alpha2/logs?operation=query&log_query=error&size=10&cluster=host&sort=desc" \
  -H "X-Remote-User: admin"

Query Parameters:

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | operation | string | query | Operation type: query/statistics/histogram/export | | namespaces | string | | Comma-separated list of namespaces | | namespace_query | string | | Fuzzy match namespace names | | workloads | string | | Comma-separated list of workloads | | workload_query | string | | Fuzzy match workload names | | pods | string | | Comma-separated list of pods | | pod_query | string | | Fuzzy match pod names | | containers | string | | Comma-separated list of containers | | container_query | string | | Fuzzy match container names | | log_query | string | | Log content keywords (case-insensitive) | | interval | string | 15m | Time interval for histogram (e.g., 15m, 1h, 1d) | | start_time | string | | Start time (seconds since epoch) | | end_time | string | | End time (seconds since epoch) | | sort | string | desc | Sort order: asc/desc | | from | int | 0 | Offset | | size | int | 10 | Number of results | | cluster | string | host | Cluster name | | exportLineLimit | int | | Max lines for export |

Extension Operations

Check Extension Status

bash
kubectl get installplan whizard-logging
kubectl get extensionversions -l kubesphere.io/extension-ref=whizard-logging

Uninstall Extension

Uninstall from all clusters:

bash
kubectl delete installplan whizard-logging

Uninstall from specific cluster:

To remove WizTelemetry Logging from a specific cluster, update the InstallPlan by removing that cluster from clusterScheduling.placement.clusters:

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>
  enabled: true
  upgradeStrategy: Manual
  clusterScheduling:
    placement:
      clusters:
        - <REMAINING_CLUSTERS>  # Remove the cluster you want to uninstall from