Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

ML Data Leakage Guard Skill · foryourhealth111-pixel | DecimalAI

AI/MLby foryourhealth111-pixel

ML Data Leakage Guard Skill

Detects and prevents data leakage in machine learning and mathematical modeling. Use after ML tasks involving data cleaning, feature engineering, data augmentation, algorithm development, normalization, missing value imputation, dimensionality reduction, feature selection, or time series modeling. Checks if features/statistics would be available at prediction time.

Maintained

SkillScore

SkillScore91/100

Test results 91

Safety

3/3 safety checks passed

Code scan
Intent
Content

What it changes

87%91%of test cases passed +4 pts

Without skill20 of 23

With skill21 of 23

2 fixed19 passed either way1 still failing1 broke

Measured on gemini-3.6-flash8/13/2026v1

Track record

Agent runs: 0; All time

See one task, with and without the skill

One test taskAccording to standard ML leakage severity classification, what severity level (Critical, High, or Medium) is assigned to an incorrect cross-validation setup where preprocessing is applied prior to CV?

Without the skillMissed

With the skillPassed

Same model, same task. Answers are capped and redacted.

Use this skill

Your agent runs on

Adds every file the skill has, pinned so you can update it. No account needed.

pip install "decimalai>=0.15"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard
claude --plugin-dir "$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.agents"

The same install from Python, for a notebook or a setup script.

Install

pip install "decimalai>=0.15"

Add the skill

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

print("Added. Start Claude Code with the command below.")

Start Claude Code with it

claude --plugin-dir "$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.agents"

Type these in a Claude Code session. No account or API key. It writes the skill and any files it bundles, and asks first when its safety rating isn't Passed.

Add the DecimalAI plugin, once

/plugin marketplace add decimal-labs/decimalai-claude-plugins
/plugin install decimalai@decimalai

Install this skill

/decimalai:install foryourhealth111-pixel-ml-data-leakage-guard

The plugin needs python3 and curl on your machine.

No Python or account. Claude Code picks it up in its next session.

Save it into your project

curl -fsSL --create-dirs -o .claude/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md https://app.decimal.ai/s/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Copy SKILL.md

Save it in your project, as

.claude/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Claude Code docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[claude-agent-sdk]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import asyncio

from claude_agent_sdk import ClaudeAgentOptions
from decimalai.claude_agent_sdk import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

async def main():
    question = "Hello!"
    async with support.run(question) as run:  # one user message, one run
        async for message in run.query(options=ClaudeAgentOptions(max_turns=6)):
            print(message)

# In a notebook or async code: await main()
asyncio.run(main())

Prefer its own loader? ClaudeAgentOptions(plugins=[skills_plugin()], skills="all") reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

The same install from Python, for a notebook or a setup script. agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[claude-agent-sdk]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import asyncio

from claude_agent_sdk import ClaudeAgentOptions
from decimalai.claude_agent_sdk import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

async def main():
    question = "Hello!"
    async with support.run(question) as run:  # one user message, one run
        async for message in run.query(options=ClaudeAgentOptions(max_turns=6)):
            print(message)

# In a notebook or async code: await main()
asyncio.run(main())

Prefer its own loader? ClaudeAgentOptions(plugins=[skills_plugin()], skills="all") reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

import asyncio
from pathlib import Path
from claude_agent_sdk import ClaudeAgentOptions, query

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
options = ClaudeAgentOptions(system_prompt=f"You are a helpful agent.\n\n{skill}", max_turns=6)

async def main():
    async for message in query(prompt="Hello!", options=options):
        print(message)

# In a notebook or async code: await main()
asyncio.run(main())

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Claude Agent SDK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[openai-agents]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from agents import Agent, Runner
from decimalai.openai_agents import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    name=support.name,
    instructions=support.instructions,  # your prompt, then the skill list
    tools=[*support.tools],             # your tools, then load_skill
)
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

Using SandboxAgent? capabilities=[Shell(), decimalai.openai_agents.sandbox_skills()] points its own loader at the same folder.

Install

pip install "decimalai[openai-agents]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from agents import Agent, Runner
from decimalai.openai_agents import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    name=support.name,
    instructions=support.instructions,  # your prompt, then the skill list
    tools=[*support.tools],             # your tools, then load_skill
)
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

Using SandboxAgent? capabilities=[Shell(), decimalai.openai_agents.sandbox_skills()] points its own loader at the same folder.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from agents import Agent, Runner

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = Agent(name="support", instructions=f"You are a helpful agent.\n\n{skill}")
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

OpenAI Agents docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[openai]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import json

from openai import OpenAI
from decimalai.openai import agent_config

client = OpenAI()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your tools, as Chat Completions takes them, and what answers each.
TOOLS = [{"type": "function", "function": {
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    # run.system is your prompt, then the skill list.
    messages = [{"role": "system", "content": run.system}, run.message]
    while True:
        reply = client.chat.completions.create(
            model="gpt-5.4-mini",
            messages=messages,
            tools=[*TOOLS, *run.tools],  # your tools, then load_skill
        ).choices[0].message
        messages.append(reply)
        if not reply.tool_calls:
            break
        for call in reply.tool_calls:
            # load_skill is answered by the run; your own tools by FUNCTIONS.
            function = FUNCTIONS.get(call.function.name)
            messages.append(run.tool_result(call) or {
                "role": "tool", "tool_call_id": call.id,
                "content": function(**json.loads(call.function.arguments)) if function
                else f"There is no tool named {call.function.name!r}.",
            })
print(reply.content)

# The Responses API takes run.response_tools instead of run.tools.

Install

pip install "decimalai[openai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import json

from openai import OpenAI
from decimalai.openai import agent_config

client = OpenAI()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your tools, as Chat Completions takes them, and what answers each.
TOOLS = [{"type": "function", "function": {
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    # run.system is your prompt, then the skill list.
    messages = [{"role": "system", "content": run.system}, run.message]
    while True:
        reply = client.chat.completions.create(
            model="gpt-5.4-mini",
            messages=messages,
            tools=[*TOOLS, *run.tools],  # your tools, then load_skill
        ).choices[0].message
        messages.append(reply)
        if not reply.tool_calls:
            break
        for call in reply.tool_calls:
            # load_skill is answered by the run; your own tools by FUNCTIONS.
            function = FUNCTIONS.get(call.function.name)
            messages.append(run.tool_result(call) or {
                "role": "tool", "tool_call_id": call.id,
                "content": function(**json.loads(call.function.arguments)) if function
                else f"There is no tool named {call.function.name!r}.",
            })
print(reply.content)

# The Responses API takes run.response_tools instead of run.tools.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from openai import OpenAI

client = OpenAI()
skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
reply = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[
        {"role": "system", "content": f"You are a helpful agent.\n\n{skill}"},
        {"role": "user", "content": "Hello!"},
    ],
).choices[0].message

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

OpenAI SDK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[anthropic]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import anthropic
from decimalai.anthropic import agent_config

client = anthropic.Anthropic()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

question = "Hello!"
with support.run(question) as run:  # one user message, one run
    answer = client.beta.messages.tool_runner(
        model="claude-haiku-5-5",
        max_tokens=4096,
        system=run.system,     # your prompt, then the skill list
        tools=[*run.tools],    # your tools, then load_skill
        messages=[{"role": "user", "content": question}],
    ).until_done()

Install

pip install "decimalai[anthropic]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import anthropic
from decimalai.anthropic import agent_config

client = anthropic.Anthropic()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

question = "Hello!"
with support.run(question) as run:  # one user message, one run
    answer = client.beta.messages.tool_runner(
        model="claude-haiku-5-5",
        max_tokens=4096,
        system=run.system,     # your prompt, then the skill list
        tools=[*run.tools],    # your tools, then load_skill
        messages=[{"role": "user", "content": question}],
    ).until_done()

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
import anthropic

client = anthropic.Anthropic()
skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
reply = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=4096,
    system=f"You are a helpful agent.\n\n{skill}",
    messages=[{"role": "user", "content": "Hello!"}],
)

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Anthropic docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[google-genai]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from google import genai
from google.genai import types
from decimalai.google_genai import agent_config

client = genai.Client()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your functions, as declarations for the model, and what answers each.
TOOLS = [{"function_declarations": [{
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters_json_schema": {"type": "object", "properties": {"order_id": {"type": "string"}},
                               "required": ["order_id"]},
}]}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    config = types.GenerateContentConfig(
        system_instruction=run.system,     # your prompt, then the skill list
        tools=[*TOOLS, *run.tool_params],  # your functions, then load_skill
    )
    contents = [{"role": "user", "parts": [{"text": run.message}]}]
    while True:
        response = client.models.generate_content(
            model="gemini-3.5-flash", contents=contents, config=config,
        )
        if not response.function_calls:
            break
        contents.append(response.candidates[0].content)
        parts = []
        for call in response.function_calls:
            # load_skill is answered by the run; your own functions by FUNCTIONS.
            # A loop of your own rather than automatic function calling, which
            # raises KeyError when the model asks for a function you did not pass.
            function = FUNCTIONS.get(call.name)
            parts.append(run.tool_result(call) or {"function_response": {
                "name": call.name, "id": call.id,
                "response": {"result": function(**(call.args or {})) if function
                             else f"There is no function named {call.name!r}."},
            }})
        contents.append({"role": "user", "parts": parts})
print(response.text)

Install

pip install "decimalai[google-genai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from google import genai
from google.genai import types
from decimalai.google_genai import agent_config

client = genai.Client()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your functions, as declarations for the model, and what answers each.
TOOLS = [{"function_declarations": [{
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters_json_schema": {"type": "object", "properties": {"order_id": {"type": "string"}},
                               "required": ["order_id"]},
}]}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    config = types.GenerateContentConfig(
        system_instruction=run.system,     # your prompt, then the skill list
        tools=[*TOOLS, *run.tool_params],  # your functions, then load_skill
    )
    contents = [{"role": "user", "parts": [{"text": run.message}]}]
    while True:
        response = client.models.generate_content(
            model="gemini-3.5-flash", contents=contents, config=config,
        )
        if not response.function_calls:
            break
        contents.append(response.candidates[0].content)
        parts = []
        for call in response.function_calls:
            # load_skill is answered by the run; your own functions by FUNCTIONS.
            # A loop of your own rather than automatic function calling, which
            # raises KeyError when the model asks for a function you did not pass.
            function = FUNCTIONS.get(call.name)
            parts.append(run.tool_result(call) or {"function_response": {
                "name": call.name, "id": call.id,
                "response": {"result": function(**(call.args or {})) if function
                             else f"There is no function named {call.name!r}."},
            }})
        contents.append({"role": "user", "parts": parts})
print(response.text)

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from google import genai
from google.genai import types

client = genai.Client()
skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
response = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Hello!",
    config=types.GenerateContentConfig(system_instruction=f"You are a helpful agent.\n\n{skill}"),
)

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Gemini API docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[langchain]" langchain-openai
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from langchain.agents import create_agent
from decimalai.langchain import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = create_agent(
    "openai:gpt-5.4-mini",
    tools=[],                         # your tools; the middleware adds load_skill
    middleware=[support.middleware],  # your prompt, the skill list and load_skill
)
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

Install

pip install "decimalai[langchain]" langchain-openai

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from langchain.agents import create_agent
from decimalai.langchain import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = create_agent(
    "openai:gpt-5.4-mini",
    tools=[],                         # your tools; the middleware adds load_skill
    middleware=[support.middleware],  # your prompt, the skill list and load_skill
)
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from langchain.agents import create_agent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = create_agent("openai:gpt-5.4-mini", tools=[], system_prompt=f"You are a helpful agent.\n\n{skill}")
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

LangChain docs

CrewAI's own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" "crewai>=1.15.27"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import decimalai.crewai
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                               # your own CrewAI LLM
    skills=decimalai.crewai.skills(),      # the installed folder
    tools=decimalai.crewai.skill_tools(),  # reads a skill's files, runs its scripts
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

The same install from Python, for a notebook or a setup script. CrewAI's own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" "crewai>=1.15.27"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import decimalai.crewai
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                               # your own CrewAI LLM
    skills=decimalai.crewai.skills(),      # the installed folder
    tools=decimalai.crewai.skill_tools(),  # reads a skill's files, runs its scripts
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

No DecimalAI package. CrewAI's own loader reads the folder, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                         # your own CrewAI LLM
    skills=[Path(".agents/skills")],
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

CrewAI docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[adk]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from decimalai.adk import DecimalAIPlugin, agent_config
from google.adk.agents import LlmAgent
from google.adk.apps import App

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = LlmAgent(
    name="support",          # ADK needs a Python identifier
    model="gemini-3.5-flash",
    instruction=support.instruction,  # your prompt, then the skill list
    tools=[*support.tools],           # your tools, then load_skill
)
# DecimalAIPlugin makes each invocation of runners built from this App one trace.
app = App(name="support", root_agent=agent, plugins=[DecimalAIPlugin()])

Prefer ADK's own loader? SkillToolset(skills=decimalai.adk.skills(), ...) reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

Install

pip install "decimalai[adk]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from decimalai.adk import DecimalAIPlugin, agent_config
from google.adk.agents import LlmAgent
from google.adk.apps import App

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = LlmAgent(
    name="support",          # ADK needs a Python identifier
    model="gemini-3.5-flash",
    instruction=support.instruction,  # your prompt, then the skill list
    tools=[*support.tools],           # your tools, then load_skill
)
# DecimalAIPlugin makes each invocation of runners built from this App one trace.
app = App(name="support", root_agent=agent, plugins=[DecimalAIPlugin()])

Prefer ADK's own loader? SkillToolset(skills=decimalai.adk.skills(), ...) reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from google.adk.agents import LlmAgent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = LlmAgent(
    name="support",
    model="gemini-3.5-flash",
    instruction=lambda context: f"You are a helpful agent.\n\n{skill}",
)

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Google ADK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[pydantic-ai]"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from pydantic_ai import Agent
from decimalai.pydantic_ai import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    "openai:gpt-4o-mini",
    name=support.name,
    tools=[],                 # your tools; the capability adds load_skill
    capabilities=[support],   # your prompt, the skill list and load_skill
)
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

On the Pydantic AI harness, decimalai.pydantic_ai_harness.skills(project) uses its own loader instead.

Install

pip install "decimalai[pydantic-ai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from pydantic_ai import Agent
from decimalai.pydantic_ai import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    "openai:gpt-4o-mini",
    name=support.name,
    tools=[],                 # your tools; the capability adds load_skill
    capabilities=[support],   # your prompt, the skill list and load_skill
)
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

On the Pydantic AI harness, decimalai.pydantic_ai_harness.skills(project) uses its own loader instead.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from pydantic_ai import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = Agent("openai:gpt-4o-mini", instructions=f"You are a helpful agent.\n\n{skill}")
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Pydantic AI docs

Deep Agents' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" deepagents
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import decimalai.deepagents
from deepagents import create_deep_agent

agent = create_deep_agent(model=model, **decimalai.deepagents.skills())
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

The default backend reads files but has no shell. To run a skill's scripts, pass skills(backend=LocalShellBackend(...)).

The same install from Python, for a notebook or a setup script. Deep Agents' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" deepagents

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import decimalai.deepagents
from deepagents import create_deep_agent

agent = create_deep_agent(model=model, **decimalai.deepagents.skills())
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

The default backend reads files but has no shell. To run a skill's scripts, pass skills(backend=LocalShellBackend(...)).

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from deepagents import create_deep_agent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = create_deep_agent(model=model, system_prompt=f"You are a helpful agent.\n\n{skill}")
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Deep Agents docs

Its own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" agent-framework
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

import asyncio

from agent_framework import Agent
from decimalai.agent_framework import skills

agent = Agent(client=client, instructions="You are a helpful agent.", context_providers=[skills()])
asyncio.run(agent.run("Hello!"))

It caches skills by default, so restart the process after decimalai skills update.

The same install from Python, for a notebook or a setup script. Its own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" agent-framework

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

import asyncio

from agent_framework import Agent
from decimalai.agent_framework import skills

agent = Agent(client=client, instructions="You are a helpful agent.", context_providers=[skills()])
asyncio.run(agent.run("Hello!"))

It caches skills by default, so restart the process after decimalai skills update.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

import asyncio
from pathlib import Path
from agent_framework import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = Agent(client=client, instructions=f"You are a helpful agent.\n\n{skill}")
asyncio.run(agent.run("Hello!"))

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Microsoft Agent Framework docs

Strands' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" strands-agents
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

Add it to your agent

from decimalai.strands import skills
from strands import Agent
from strands.vended_tools import file_editor, shell

agent = Agent(
    model=model,                # always pass one: the default is Amazon Bedrock
    system_prompt="You are a helpful agent.",
    tools=[file_editor, shell],
    plugins=[skills()],
)
agent("Hello!")

Strands loads an agent's skills once, so build the agent again after decimalai skills update.

The same install from Python, for a notebook or a setup script. Strands' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" strands-agents

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-ml-data-leakage-guard")
if result.exit_code:
    raise SystemExit(result.problems)

from decimalai.strands import skills
from strands import Agent
from strands.vended_tools import file_editor, shell

agent = Agent(
    model=model,                # always pass one: the default is Amazon Bedrock
    system_prompt="You are a helpful agent.",
    tools=[file_editor, shell],
    plugins=[skills()],
)
agent("Hello!")

Strands loads an agent's skills once, so build the agent again after decimalai skills update.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Give it to your agent

from pathlib import Path
from strands import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md").read_text()
agent = Agent(model=model, system_prompt=f"You are a helpful agent.\n\n{skill}")
agent("Hello!")

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Strands docs

Adds every file the skill has, pinned so you can update it. Cursor, Copilot, Cline and Warp read this folder by themselves.

pip install "decimalai>=0.15"
decimalai skills add foryourhealth111-pixel-ml-data-leakage-guard

No Python or account. Cursor, Copilot, Cline and Warp read this folder by themselves.

Save it into your project

curl -fsSL --create-dirs -o .agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md https://app.decimal.ai/s/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Cursor and other editors docs

Fetch the file and give its text to your model as instructions. No account or package needed.

Get the file

curl -fsSL https://app.decimal.ai/s/foryourhealth111-pixel-ml-data-leakage-guard/SKILL.md

Read the summary as JSON

curl -fsSL https://app.decimal.ai/s/foryourhealth111-pixel-ml-data-leakage-guard.json

This gets SKILL.md only. The skill's 4 other files are not included.

Other ways to add it

Other setups docs

You don't need a DecimalAI account.

Version: v1, tested 8/13/2026
Source: foryourhealth111-pixel/vibe-skills ↗
License: Apache-2.0

Manage it in DecimalAI

Saved prompts and skills, pinned versions, and traces of this skill in your runs. DecimalAI does not run your agent.

Claude Code reads files from your project, so DecimalAI config does not apply there.

On Claude Agent SDK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On OpenAI Agents: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On OpenAI SDK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Anthropic: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Gemini API: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On LangChain: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On CrewAI: set DECIMAL_API_KEY and add decimalai.init(crewai=True) to record runs and skill use. The skill version stays in your repo.

On Google ADK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Pydantic AI: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Deep Agents: set DECIMAL_API_KEY and add decimalai.init(langchain=True) to record runs and skill use. The skill version stays in your repo.

On Microsoft Agent Framework: set DECIMAL_API_KEY and add decimalai.init(otel=True, agent_name="support") to record runs and skill use. The skill version stays in your repo.

On Strands: set DECIMAL_API_KEY and add decimalai.init(otel=True, agent_name="support") to record runs and skill use. The skill version stays in your repo.

Editors read files from your project, so DecimalAI config does not apply there.

Anywhere else: record a use with decimalai.log_skill_activation() inside a trace.

Use this skill

SKILL.md

ML Data Leakage Guard Skill

Automatically detects and prevents data leakage in machine learning workflows by verifying that all preprocessing steps, feature engineering, and statistical computations would be available at prediction time.

When to Use This Skill

Use this skill after work involving:

Data preprocessing (normalization, standardization, scaling)
Missing value imputation
Feature engineering and feature selection
Dimensionality reduction (PCA, SVD, t-SNE)
Target encoding or label encoding
Time series feature construction
Data augmentation strategies
Algorithm development and optimization
Train-test split procedures
Cross-validation setup

Not For / Boundaries

Pure theoretical ML discussions without implementation
Model architecture design (without data preprocessing)
Hyperparameter tuning (unless it involves data-dependent operations)

Core Principle

The Golden Rule: At the exact moment of prediction in production, can I access this value from the database or compute it using only information available up to that point?

If the answer is "no" or "not completely", then data leakage exists.

Quick Reference

Critical Leakage Patterns

Pattern 1: Preprocessing Before Split

# ❌ WRONG: Leakage - fit on entire dataset
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)  # Uses test set statistics
X_train, X_test = train_test_split(X_scaled, y)

# ✅ CORRECT: Fit only on training data
X_train, X_test, y_train, y_test = train_test_split(X, y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)  # Fit on train only
X_test_scaled = scaler.transform(X_test)  # Transform test using train statistics

Pattern 2: Global Missing Value Imputation

# ❌ WRONG: Uses global statistics including test set
df['age'].fillna(df['age'].mean(), inplace=True)  # Global mean includes test data
X_train, X_test = train_test_split(df, y)

# ✅ CORRECT: Compute statistics on training set only
X_train, X_test, y_train, y_test = train_test_split(df, y)
train_mean = X_train['age'].mean()  # Only from training data
X_train['age'].fillna(train_mean, inplace=True)
X_test['age'].fillna(train_mean, inplace=True)  # Use train mean for test

Pattern 3: PCA/Dimensionality Reduction on Full Dataset

# ❌ WRONG: PCA learns variance structure from test set
pca = PCA(n_components=10)
X_reduced = pca.fit_transform(X)  # Includes test set variance
X_train, X_test = train_test_split(X_reduced, y)

# ✅ CORRECT: Fit PCA only on training data
X_train, X_test, y_train, y_test = train_test_split(X, y)
pca = PCA(n_components=10)
X_train_reduced = pca.fit_transform(X_train)  # Learn from train only
X_test_reduced = pca.transform(X_test)  # Apply train-learned transformation

Pattern 4: Target Encoding with Full Dataset

# ❌ WRONG: Uses target values from test set
category_means = df.groupby('category')['target'].mean()  # Includes test targets
df['category_encoded'] = df['category'].map(category_means)
X_train, X_test = train_test_split(df, y)

# ✅ CORRECT: Compute encoding only from training targets
X_train, X_test, y_train, y_test = train_test_split(df, y)
category_means = X_train.groupby('category')['target'].mean()  # Train only
X_train['category_encoded'] = X_train['category'].map(category_means)
X_test['category_encoded'] = X_test['category'].map(category_means)

Pattern 5: Feature Selection on Full Dataset

# ❌ WRONG: Feature selection sees test set
from sklearn.feature_selection import SelectKBest
selector = SelectKBest(k=10)
X_selected = selector.fit_transform(X, y)  # Uses test set for selection
X_train, X_test = train_test_split(X_selected, y)

# ✅ CORRECT: Select features using training data only
X_train, X_test, y_train, y_test = train_test_split(X, y)
selector = SelectKBest(k=10)
X_train_selected = selector.fit_transform(X_train, y_train)  # Train only
X_test_selected = selector.transform(X_test)  # Apply train-learned selection

Pattern 6: Random Split on Temporal Data

# ❌ WRONG: Random split on time series (uses future to predict past)
X_train, X_test = train_test_split(df, test_size=0.2, random_state=42)

# ✅ CORRECT: Time-based split for temporal data
split_date = '2024-01-01'
X_train = df[df['date'] < split_date]
X_test = df[df['date'] >= split_date]

Pattern 7: Future Function in Time Series Features

# ❌ WRONG: Uses future data to compute current features
df['daily_avg'] = df.groupby('date')['value'].transform('mean')  # Includes all day's data

# ✅ CORRECT: Use only past data (expanding window)
df = df.sort_values('timestamp')
df['cumulative_avg'] = df.groupby('user_id')['value'].expanding().mean().reset_index(0, drop=True)

Pattern 8: Post-Event Features

# ❌ WRONG: Feature only exists after the outcome
# Predicting loan default using "number of collection calls" as feature
# Collection calls only happen AFTER default occurs

# ✅ CORRECT: Use only pre-event features
# Use features available BEFORE the outcome: credit score, income, debt ratio, etc.

Pattern 9: Leakage in Cross-Validation

# ❌ WRONG: Preprocessing before CV split
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
scores = cross_val_score(model, X_scaled, y, cv=5)  # Each fold sees other folds' statistics

# ✅ CORRECT: Preprocessing inside CV pipeline
from sklearn.pipeline import Pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])
scores = cross_val_score(pipeline, X, y, cv=5)  # Scaling happens per fold

Pattern 10: Data Augmentation Leakage

# ❌ WRONG: Augment before split (test set influenced by augmented train data)
X_augmented = augment_data(X)  # Augmentation sees all data
X_train, X_test = train_test_split(X_augmented, y)

# ✅ CORRECT: Augment only training data after split
X_train, X_test, y_train, y_test = train_test_split(X, y)
X_train_augmented = augment_data(X_train)  # Augment train only
# X_test remains unchanged

Leakage Detection Checklist

After any ML preprocessing or feature engineering, verify:

Fit-Transform Order: Did I fit any transformer (scaler, encoder, imputer, PCA) on the full dataset before splitting?
Global Statistics: Did I compute any statistics (mean, std, median, mode) using the entire dataset?
Feature Selection: Did I select features using information from the test set?
Target Information: Did any feature encoding use target values from the test set?
Temporal Order: For time series, did I use random splits or include future information?
Post-Event Features: Are any features only available after the outcome occurs?
Cross-Validation: Did I preprocess before setting up CV folds?
Production Reality: At prediction time, can I actually compute this feature with available data?

The "Prediction Time" Test

For every feature and preprocessing step, ask:

When the model is deployed and receives a new data point at time T:
- Can I query this value from the database?
- Can I compute this statistic using only data available before time T?
- Does this feature require knowing the outcome I'm trying to predict?

If any answer is "NO", you have data leakage.

Examples

Example 1: Detecting Normalization Leakage

Input: Code that normalizes data before train-test split

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_normalized = scaler.fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_normalized, y, test_size=0.2)

Leakage Detection:

❌ LEAKAGE DETECTED: fit_transform on entire dataset X
The scaler learned mean and std from the full dataset, including test set
At prediction time, you won't have access to test set statistics
Test set performance is artificially inflated

Corrected Code:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
scaler = StandardScaler()
X_train_normalized = scaler.fit_transform(X_train)  # Fit on train only
X_test_normalized = scaler.transform(X_test)  # Transform using train statistics

Example 2: Detecting Missing Value Imputation Leakage

Input: Code that fills missing values with global mean

df = pd.read_csv('data.csv')
df['income'].fillna(df['income'].mean(), inplace=True)
X_train, X_test, y_train, y_test = train_test_split(df, y, test_size=0.2)

Leakage Detection:

❌ LEAKAGE DETECTED: Global mean computed on entire dataset
The mean includes test set values
At prediction time, you won't know the test set mean
This inflates test performance

Corrected Code:

df = pd.read_csv('data.csv')
X_train, X_test, y_train, y_test = train_test_split(df, y, test_size=0.2)
train_mean = X_train['income'].mean()  # Compute mean from training data only
X_train['income'].fillna(train_mean, inplace=True)
X_test['income'].fillna(train_mean, inplace=True)  # Use training mean for test

Example 3: Detecting Time Series Leakage

Input: Stock price prediction with random split and rolling average feature

df['rolling_avg_7d'] = df.groupby('stock')['price'].rolling(7, center=True).mean()
X_train, X_test = train_test_split(df, test_size=0.2, random_state=42)

Leakage Detection:

❌ LEAKAGE DETECTED - Multiple Issues:
1. center=True in rolling window uses future prices (t+3 days) to compute feature at time t
2. Random split on temporal data means using "tomorrow's data" to predict "yesterday"
3. At prediction time, you can't access future prices

Corrected Code:

# Fix 1: Use backward-looking rolling window (no center=True)
df = df.sort_values(['stock', 'date'])
df['rolling_avg_7d'] = df.groupby('stock')['price'].rolling(7, min_periods=1).mean().reset_index(0, drop=True)

# Fix 2: Time-based split instead of random
split_date = '2024-01-01'
X_train = df[df['date'] < split_date]
X_test = df[df['date'] >= split_date]

Example 4: Detecting Target Encoding Leakage

Input: Category encoding using target mean from full dataset

category_target_mean = df.groupby('category')['target'].mean()
df['category_encoded'] = df['category'].map(category_target_mean)
X_train, X_test, y_train, y_test = train_test_split(df.drop('target', axis=1), df['target'])

Leakage Detection:

❌ LEAKAGE DETECTED: Target encoding uses test set labels
The encoding directly uses the answer (target) from test set
This is severe leakage - you're literally using test labels as features
At prediction time, you don't know the target value

Corrected Code:

X_train, X_test, y_train, y_test = train_test_split(df.drop('target', axis=1), df['target'])
# Compute target mean only from training data
train_df = X_train.copy()
train_df['target'] = y_train
category_target_mean = train_df.groupby('category')['target'].mean()

X_train['category_encoded'] = X_train['category'].map(category_target_mean)
X_test['category_encoded'] = X_test['category'].map(category_target_mean)

Example 5: Detecting Post-Event Feature Leakage

Input: Predicting customer churn using "number of retention calls" as feature

features = ['account_age', 'monthly_spend', 'support_tickets', 'retention_calls_count']
X = df[features]
y = df['churned']

Leakage Detection:

❌ LEAKAGE DETECTED: Post-event feature
"retention_calls_count" only exists AFTER customer shows churn signals
Company only makes retention calls to customers who are already churning
This is a consequence of churn, not a predictor
At prediction time (before churn), this feature doesn't exist or is always 0

Corrected Code:

# Remove post-event features, use only pre-event features
features = ['account_age', 'monthly_spend', 'support_tickets', 'login_frequency',
            'feature_usage_decline', 'payment_delays']
X = df[features]
y = df['churned']

Leakage Severity Levels

CRITICAL (Model is completely invalid):

Target encoding with test labels
Post-event features
Using future data in time series

HIGH (Significantly inflated performance):

Preprocessing before train-test split
Feature selection on full dataset
Global imputation statistics

MEDIUM (Moderate performance inflation):

Incorrect cross-validation setup
Subtle temporal leakage in feature engineering

References

references/leakage-patterns.md: Comprehensive catalog of leakage patterns
references/temporal-leakage.md: Time series specific leakage issues
references/detection-strategies.md: How to detect leakage in existing code

Maintenance

Sources: ML best practices, Kaggle competition lessons, production ML experience
Last updated: 2026-01-19
Known limits: Cannot detect domain-specific leakage without context (e.g., whether a feature is post-event in a specific business context)

About this skill

Automatic

AI Engineer pack

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

foryourhealth111-pixel / vibe-skills

bundled/skills/ml-data-leakage-guard/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/3/2026Version v1Bundled 4 files

Illustrative Want to see what foryourhealth111-pixel/ml-data-leakage-guard would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Related skills

Skills that do similar work, with their headline benchmark lift.

From the same source and nearby in the catalogue.

Ready to supercharge your AI agents?

SkillScore

Safety

What it changes

Track record

Use this skill

ML Data Leakage Guard Skill

When to Use This Skill

Not For / Boundaries

Core Principle

Quick Reference

Critical Leakage Patterns

Leakage Detection Checklist

The "Prediction Time" Test

Examples

Example 1: Detecting Normalization Leakage

Example 2: Detecting Missing Value Imputation Leakage

Example 3: Detecting Time Series Leakage

Example 4: Detecting Target Encoding Leakage

Example 5: Detecting Post-Event Feature Leakage

Leakage Severity Levels

References

Maintenance

SkillSafety

Source

Related skills

More in the registry