Runtime Behavior Probe

Runtime Behavior Probe

Overview

Use this skill to investigate real runtime behavior, not to restate code or documentation. Start by planning the investigation, then execute a case matrix, record observed behavior, and report both the findings and the method used to obtain them.

Core Rules

• Treat this skill as manual-only. Do not rely on implicit invocation.
• Invoking this skill authorizes planning only. Every runtime probe requires explicit user approval after the exact probe has been proposed. Do not infer execution approval from the skill invocation or a general request to investigate runtime behavior.
• Before requesting approval, disclose the source identity, exact command, transitively executed material, known filesystem, environment, network, and host-service capabilities, expected side effects, and control for the proposed probe. Mark unknown capabilities as unknown rather than assuming that they are unavailable.
• Wait for an affirmative response before executing the probe. Approval is bound to the disclosed source, command, executed material, and capability scope. Obtain new approval before changing any of those fields, adding another probe, or expanding the approved matrix.
• A baseline success or smoke case is often the right entry point, but do not stop there when the real question involves edge cases, drift, or failure behavior.
• Plan before running anything. Write the case matrix first, then fill it in with observed results. The matrix can live in a scratch note, a temporary file, or the probe script header.
• Default to proposing local or read-only probes. Consider a live service only when it is clearly relevant, then apply the lightweight gates below before requesting approval.
• Size the probe to the decision. Start with the smallest matrix that can disqualify or validate the current hypothesis, then expand only when uncertainty remains.
• Before a live probe, apply three lightweight gates:
  • Destination gate. Use only a live destination that is clearly allowed for the task.
  • Intent gate. Run the live probe only when the user explicitly wants runtime verification on that integration, or explicitly approves it after you propose the probe.
  • Data gate. If the probe will read environment variables, mutate remote state, incur material cost, or exercise non-public or user data, name the exact variable names or data class and get explicit approval first.
• Classify each case as read-only, mutating, or costly before execution. For mutating or costly cases, or for any live case that will read environment variables, define cleanup or rollback before running the probe.
• Use temporary files or a temporary directory for one-off probe scripts.
• Keep temporary artifacts until the final response is drafted. Then delete them by default unless the user asked to keep them or they are needed for follow-up. Even when artifacts are deleted, keep a short run summary of the command shape, runtime context, and artifact status in the report.
• Before executing a live probe that will read environment variables, tell the user the exact variable names you plan to use and why, then wait for explicit approval. Examples include OPENAI_API_KEY and other expected default names for the system under test.
• When the environment-variable approval gate is required and the request_user_input tool is available, use that tool instead of a plain-text approval question. Ask one concise question with mutually exclusive choices such as Allow once (Recommended) and Do not allow, omit autoResolutionMs, and make the approval single-probe and limited to the exact named variables and destination. If the tool is unavailable, fall back to a concise plain-text approval question and do not proceed until the user explicitly approves.
• Never print secrets, even when they come from standard environment variables that this skill may use.
• For OpenAI API or OpenAI platform probes in this repository, use $openai-knowledge early to confirm contract-sensitive details such as supported parameters, field names, and limits. Use runtime probing to validate or challenge the documented behavior, not to skip the documentation pass entirely. If the docs MCP is unavailable, fall back to the official OpenAI docs and say that you used the fallback in the report.
• For benchmark or comparison probes, make parity explicit before execution. Record what is held constant, what variable is under test, which response-shape constraints keep the comparison fair, and any usage or token counters that matter for interpreting latency or cost.
• For OpenAI hosted tool probes, remove setup ambiguity before attributing a negative result to runtime behavior:
  • Force the tool path with the matching tool_choice when the question depends on tool invocation.
  • Treat container_auto and container_reference as separate cases, not interchangeable setup details.
  • Clear unsupported model or tool options first so they do not invalidate the probe.

Workflow

1. Restate the investigation target in operational terms. Name the runtime surface, the key uncertainty, and the highest-risk behaviors to test.
2. Do a short preflight. Check the relevant code or docs first, decide whether the question needs local or live validation, and note any repo, baseline, or release boundary that matters.
3. Define the decision signal before building the matrix. For a suspected defect, name the exact user-visible symptom, the command or probe that can distinguish it from correct behavior, the expected failing observation, and a known-good control. Confirm that the signal exercises the real producer and caller path rather than only an adjacent helper. Prefer a fast, deterministic local loop when one can answer the question. If no credible signal can be built, state the missing access or artifact and do not substitute a nearby behavior as proof.
4. Create a validation matrix before executing probes. Cover both baseline behavior and the most relevant failure or drift cases. The matrix can live in a scratch note, a temporary file, or a structured header inside the probe script.
5. For each case, choose an execution mode up front: