WizTelemetry Data Pipeline (Vector)

Overview

WizTelemetry Data Pipeline is an extension based on vector (https://vector.dev/) that provides the ability to collect, transform, and route observability data. It is a core dependency for other WizTelemetry extensions like Logging, Auditing, Events, and Notification.

When to Use

• Installing or configuring the WizTelemetry Data Pipeline extension
• Setting up data collection for logs, auditing, events, and notifications
• Configuring Vector sinks (OpenSearch)
• Managing Vector agent components

Installation

Prerequisites

REQUIRED: Complete all steps in order before generating InstallPlan.

Step 1: Get Available Clusters and Confirm Target

⚠️ CRITICAL: DO NOT proceed until target clusters are determined.

Step 1.1: Get available clusters

bash
kubectl get clusters -o jsonpath='{.items[*].metadata.name}'

Step 1.2: Determine target clusters

• If user explicitly specified target clusters in the request → Use those clusters directly, proceed to Step 2
• If user did NOT specify target clusters → Ask user to confirm which clusters to deploy to, then proceed to Step 2

Ask user (if not specified):

Available clusters: host, dev
Which clusters do you want to deploy Vector to?

Step 2: Get OpenSearch Endpoint and Credentials (MUST DO)

• If user already provided OpenSearch endpoint and credentials in the request → Use those directly, proceed to Step 3
• If user did NOT provide → You MUST ask user for OpenSearch endpoint and credentials

Ask user for (if not provided):

1. OpenSearch endpoint URL (required)

• Example: http://<node-ip>:30920 or https://opensearch.example.com:9200

1. OpenSearch credentials (required)

• Username (default: admin)
• Password

DO NOT proceed to Step 3 until user provides both endpoint and credentials.

Step 3: Get Latest Vector Version (if not provided by user)

MUST do this to get the latest version:

bash
kubectl get extensionversions -l kubesphere.io/extension-ref=vector -o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V | tail -1

This outputs the latest version (e.g., 1.1.4). Note this down - you'll use it in the InstallPlan.

Install Vector Extension

⚠️ IMPORTANT: Complete prerequisite steps (1-3) BEFORE this step.

⚠️ CRITICAL: InstallPlan metadata.name MUST be vector. DO NOT use any other name.

Based on your selections:

• Target clusters: Use the user-confirmed cluster names
• OpenSearch endpoint: User-provided endpoint
• OpenSearch credentials: User-provided username and password

⚠️ CRITICAL: config field is YAML format. You MUST:

• Use the config structure exactly as shown in the template
• DO NOT add configuration fields that are not shown in the template
• DO NOT modify the structure or hierarchy

⚠️ CRITICAL: All placeholders MUST be replaced with actual values. DO NOT leave them as placeholders.

Template

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: vector
spec:
  extension:
    name: vector
    version: <VECTOR_VERSION>  # From Step 3
  enabled: true
  upgradeStrategy: Manual
  config: |
    agent:
      sinks:
        opensearch:
          auth:
            strategy: basic
            user: <OPENSEARCH_USER>
            password: <OPENSEARCH_PASSWORD>
          endpoints:
            - <OPENSEARCH_ENDPOINT>
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Replace placeholders:

• <VECTOR_VERSION>: From Step 2 (e.g., 1.1.4)
• <OPENSEARCH_ENDPOINT>: User-provided endpoint (e.g., http://<node-ip>:30920)
• <OPENSEARCH_USER>: User-provided username (default: admin)
• <OPENSEARCH_PASSWORD>: User-provided password
• <TARGET_CLUSTERS>: User-confirmed cluster names

⚠️ DO NOT generate InstallPlan until all placeholders have real values.

Wait for Deployment

After applying InstallPlan, you MUST wait for deployment to complete:

bash
# Wait for Vector pods to be ready (on each cluster)
kubectl wait --for=condition=Ready pods -n kubesphere-logging-system -l app.kubernetes.io/instance=vector --timeout=300s

# Verify deployment status
kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/instance=vector

Show deployment summary to user:

• Which clusters Vector was deployed to
• OpenSearch endpoint used
• Pod status (Ready/Total)

Enable Metrics Export

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: vector
spec:
  extension:
    name: vector
    version: <VECTOR_VERSION>  # From Step 2
  enabled: true
  upgradeStrategy: Manual
  config: |
    agent:
      sinks:
        opensearch:
          auth:
            strategy: basic
            user: <OPENSEARCH_USER>
            password: <OPENSEARCH_PASSWORD>
          endpoints:
            - <OPENSEARCH_ENDPOINT>
      exportMetrics:
        enabled: true
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Configuration Parameters

Agent Parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | agent.role | string | "Agent" | Role identifier | | agent.image.tag | string | "0.53.0-debian" | Vector image tag | | agent.resources.requests.cpu | string | "100m" | CPU request | | agent.resources.requests.memory | string | "100Mi" | Memory request | | agent.resources.limits.cpu | string | "2000m" | CPU limit | | agent.resources.limits.memory | string | "2000Mi" | Memory limit | | agent.service.ports | list | see values.yaml | Service ports | | agent.exportMetrics.enabled | bool | false | Enable metrics export |

Agent Sinks OpenSearch Parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | agent.sinks.opensearch.endpoints | list | Yes | OpenSearch endpoint URLs | | agent.sinks.opensearch.auth.strategy | string | Yes | Authentication strategy (set to basic) | | agent.sinks.opensearch.auth.user | string | Yes | Username for authentication | | agent.sinks.opensearch.auth.password | string | Yes | Password for authentication | | agent.sinks.opensearch.tls.verify | bool | No | Enable TLS verification (default: false) |

Example:

yaml
agent:
  sinks:
    opensearch:
      endpoints:
        - http://<node-ip>:30920
      auth:
        strategy: basic
        user: admin
        password: admin
      tls:
        verify: false

Docker Root Directory Configuration

If Docker root directory is not /var/lib:

yaml
agent:
  extraVolumes:
    - name: docker-root
      hostPath:
        path: /path/to/docker
        type: ''
  extraVolumeMounts:
    - name: docker-root
      mountPath: /path/to/docker

Extension Operations

Check Extension Status

bash
# View extension installation status
kubectl get installplan vector

# View extension version
kubectl get extensionversions -l kubesphere.io/extension-ref=vector

Check Pod Status

bash
# View all Vector pods
kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/name=vector

# View agent pods
kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/name=vector,app.kubernetes.io/component=agent

View Logs

bash
# View agent logs
kubectl logs -n kubesphere-logging-system -l app.kubernetes.io/name=vector,app.kubernetes.io/component=agent --tail=100

Update Configuration

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: vector
spec:
  extension:
    name: vector
    version: <VECTOR_VERSION>
  enabled: true
  upgradeStrategy: Manual
  config: |
    agent:
      sinks:
        opensearch:
          auth:
            strategy: basic
            user: <OPENSEARCH_USER>
            password: <OPENSEARCH_PASSWORD>
          endpoints:
            - <OPENSEARCH_ENDPOINT>
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Uninstall Extension

Uninstall from all clusters:

bash
kubectl delete installplan vector

Uninstall from specific cluster:

To remove Vector from a specific cluster, update the InstallPlan by removing that cluster from clusterScheduling.placement.clusters:

yaml
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: vector
spec:
  extension:
    name: vector
    version: <VECTOR_VERSION>
  enabled: true
  upgradeStrategy: Manual
  config: |
    agent:
      sinks:
        opensearch:
          auth:
            strategy: basic
            user: <OPENSEARCH_USER>
            password: <OPENSEARCH_PASSWORD>
          endpoints:
            - <OPENSEARCH_ENDPOINT>
  clusterScheduling:
    placement:
      clusters:
        - <REMAINING_CLUSTERS>  # Remove the cluster you want to uninstall from

Important Notes

1. Dependency: Vector is a core dependency for WizTelemetry extensions. Install it first before installing Logging, Auditing, Events, or Notification.
2. OpenSearch Required: User must provide OpenSearch endpoint and credentials.
3. Multicluster: The extension uses installationMode: Multicluster:

• agent (tag: agent) is deployed to all selected member clusters

1. Agent Scheduling: Agent pods have affinity to avoid edge nodes and tolerate all taints.
2. Cross-cluster Access: Ensure OpenSearch endpoint is accessible from all Vector clusters.

Troubleshooting

Check Vector Configuration

bash
# View Vector configmap
kubectl get configmap -n kubesphere-logging-system -l app.kubernetes.io/name=vector

# View specific config
kubectl get configmap -n kubesphere-logging-system vector-config -o yaml

Verify Sinks

bash
# Check if sinks are configured correctly
kubectl get secret -n kubesphere-logging-system vector-sinks -o yaml

Common Issues

| Issue | Solution | |-------|----------| | Pods not starting | Check if OpenSearch is accessible | | Data not flowing | Verify sink configuration and network connectivity | | Agent not on member cluster | Check multicluster installation settings | | Out of memory | Increase resource limits in configuration |