Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Scientific Data Preprocessing Skill · foryourhealth111-pixel | DecimalAI

Databy foryourhealth111-pixel

Scientific Data Preprocessing Skill

⚠️ CRITICAL USER EXPERIENCE-BASED SKILL - ALWAYS CONSULT BEFORE DATA PREPROCESSING ⚠️ Prevents catastrophic errors (88.9% error rate in V1.0 case study) through multi-level feature analysis, data leakage detection, and semantic validation. MANDATORY for: data preprocessing, feature engineering, standardization, normalization, interpolation, missing value handling, feature selection, or ANY data transformation task. Covers grouped time-series, cross-sectional, panel data. Detects: time travel lea

Never made a case worse

SkillScore

SkillScore95/100

Test results 95

Safety

3/3 safety checks passed

Code scan
Intent
Content

What it changes

77%95%of test cases passed +18 pts

Without skill17 of 22

With skill21 of 22

4 fixed17 passed either way1 still failing0 broke

Measured on gemini-3.6-flash8/13/2026v1

Track record

Agent runs: 0; All time

See one task, with and without the skill

One test taskWe have a sports performance dataset with binary indicator columns `is_ace` and `is_winner`, alongside continuous columns `distance_run` and `speed_mph`. We plan to pass all numeric columns through `StandardScaler` to normalize them before training. Write a Python pandas preprocessing script for these numeric features.

Without the skillMissed

With the skillPassed

Same model, same task. Answers are capped and redacted.

Use this skill

Your agent runs on

Adds every file the skill has, pinned so you can update it. No account needed.

pip install "decimalai>=0.15"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing
claude --plugin-dir "$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.agents"

The same install from Python, for a notebook or a setup script.

Install

pip install "decimalai>=0.15"

Add the skill

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

print("Added. Start Claude Code with the command below.")

Start Claude Code with it

claude --plugin-dir "$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.agents"

Type these in a Claude Code session. No account or API key. It writes the skill and any files it bundles, and asks first when its safety rating isn't Passed.

Add the DecimalAI plugin, once

/plugin marketplace add decimal-labs/decimalai-claude-plugins
/plugin install decimalai@decimalai

Install this skill

/decimalai:install foryourhealth111-pixel-scientific-data-preprocessing

The plugin needs python3 and curl on your machine.

No Python or account. Claude Code picks it up in its next session.

Save it into your project

curl -fsSL --create-dirs -o .claude/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md https://app.decimal.ai/s/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Copy SKILL.md

Save it in your project, as

.claude/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Claude Code docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[claude-agent-sdk]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import asyncio

from claude_agent_sdk import ClaudeAgentOptions
from decimalai.claude_agent_sdk import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

async def main():
    question = "Hello!"
    async with support.run(question) as run:  # one user message, one run
        async for message in run.query(options=ClaudeAgentOptions(max_turns=6)):
            print(message)

# In a notebook or async code: await main()
asyncio.run(main())

Prefer its own loader? ClaudeAgentOptions(plugins=[skills_plugin()], skills="all") reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

The same install from Python, for a notebook or a setup script. agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[claude-agent-sdk]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import asyncio

from claude_agent_sdk import ClaudeAgentOptions
from decimalai.claude_agent_sdk import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

async def main():
    question = "Hello!"
    async with support.run(question) as run:  # one user message, one run
        async for message in run.query(options=ClaudeAgentOptions(max_turns=6)):
            print(message)

# In a notebook or async code: await main()
asyncio.run(main())

Prefer its own loader? ClaudeAgentOptions(plugins=[skills_plugin()], skills="all") reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

import asyncio
from pathlib import Path
from claude_agent_sdk import ClaudeAgentOptions, query

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
options = ClaudeAgentOptions(system_prompt=f"You are a helpful agent.\n\n{skill}", max_turns=6)

async def main():
    async for message in query(prompt="Hello!", options=options):
        print(message)

# In a notebook or async code: await main()
asyncio.run(main())

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Claude Agent SDK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[openai-agents]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from agents import Agent, Runner
from decimalai.openai_agents import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    name=support.name,
    instructions=support.instructions,  # your prompt, then the skill list
    tools=[*support.tools],             # your tools, then load_skill
)
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

Using SandboxAgent? capabilities=[Shell(), decimalai.openai_agents.sandbox_skills()] points its own loader at the same folder.

Install

pip install "decimalai[openai-agents]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from agents import Agent, Runner
from decimalai.openai_agents import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    name=support.name,
    instructions=support.instructions,  # your prompt, then the skill list
    tools=[*support.tools],             # your tools, then load_skill
)
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

Using SandboxAgent? capabilities=[Shell(), decimalai.openai_agents.sandbox_skills()] points its own loader at the same folder.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from agents import Agent, Runner

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = Agent(name="support", instructions=f"You are a helpful agent.\n\n{skill}")
# In a notebook or async code: result = await Runner.run(agent, "Hello!")
result = Runner.run_sync(agent, "Hello!")

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

OpenAI Agents docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[openai]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import json

from openai import OpenAI
from decimalai.openai import agent_config

client = OpenAI()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your tools, as Chat Completions takes them, and what answers each.
TOOLS = [{"type": "function", "function": {
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    # run.system is your prompt, then the skill list.
    messages = [{"role": "system", "content": run.system}, run.message]
    while True:
        reply = client.chat.completions.create(
            model="gpt-5.4-mini",
            messages=messages,
            tools=[*TOOLS, *run.tools],  # your tools, then load_skill
        ).choices[0].message
        messages.append(reply)
        if not reply.tool_calls:
            break
        for call in reply.tool_calls:
            # load_skill is answered by the run; your own tools by FUNCTIONS.
            function = FUNCTIONS.get(call.function.name)
            messages.append(run.tool_result(call) or {
                "role": "tool", "tool_call_id": call.id,
                "content": function(**json.loads(call.function.arguments)) if function
                else f"There is no tool named {call.function.name!r}.",
            })
print(reply.content)

# The Responses API takes run.response_tools instead of run.tools.

Install

pip install "decimalai[openai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import json

from openai import OpenAI
from decimalai.openai import agent_config

client = OpenAI()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your tools, as Chat Completions takes them, and what answers each.
TOOLS = [{"type": "function", "function": {
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    # run.system is your prompt, then the skill list.
    messages = [{"role": "system", "content": run.system}, run.message]
    while True:
        reply = client.chat.completions.create(
            model="gpt-5.4-mini",
            messages=messages,
            tools=[*TOOLS, *run.tools],  # your tools, then load_skill
        ).choices[0].message
        messages.append(reply)
        if not reply.tool_calls:
            break
        for call in reply.tool_calls:
            # load_skill is answered by the run; your own tools by FUNCTIONS.
            function = FUNCTIONS.get(call.function.name)
            messages.append(run.tool_result(call) or {
                "role": "tool", "tool_call_id": call.id,
                "content": function(**json.loads(call.function.arguments)) if function
                else f"There is no tool named {call.function.name!r}.",
            })
print(reply.content)

# The Responses API takes run.response_tools instead of run.tools.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from openai import OpenAI

client = OpenAI()
skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
reply = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[
        {"role": "system", "content": f"You are a helpful agent.\n\n{skill}"},
        {"role": "user", "content": "Hello!"},
    ],
).choices[0].message

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

OpenAI SDK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[anthropic]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import anthropic
from decimalai.anthropic import agent_config

client = anthropic.Anthropic()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

question = "Hello!"
with support.run(question) as run:  # one user message, one run
    answer = client.beta.messages.tool_runner(
        model="claude-haiku-5-5",
        max_tokens=4096,
        system=run.system,     # your prompt, then the skill list
        tools=[*run.tools],    # your tools, then load_skill
        messages=[{"role": "user", "content": question}],
    ).until_done()

Install

pip install "decimalai[anthropic]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import anthropic
from decimalai.anthropic import agent_config

client = anthropic.Anthropic()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

question = "Hello!"
with support.run(question) as run:  # one user message, one run
    answer = client.beta.messages.tool_runner(
        model="claude-haiku-5-5",
        max_tokens=4096,
        system=run.system,     # your prompt, then the skill list
        tools=[*run.tools],    # your tools, then load_skill
        messages=[{"role": "user", "content": question}],
    ).until_done()

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
import anthropic

client = anthropic.Anthropic()
skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
reply = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=4096,
    system=f"You are a helpful agent.\n\n{skill}",
    messages=[{"role": "user", "content": "Hello!"}],
)

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Anthropic docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[google-genai]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from google import genai
from google.genai import types
from decimalai.google_genai import agent_config

client = genai.Client()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your functions, as declarations for the model, and what answers each.
TOOLS = [{"function_declarations": [{
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters_json_schema": {"type": "object", "properties": {"order_id": {"type": "string"}},
                               "required": ["order_id"]},
}]}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    config = types.GenerateContentConfig(
        system_instruction=run.system,     # your prompt, then the skill list
        tools=[*TOOLS, *run.tool_params],  # your functions, then load_skill
    )
    contents = [{"role": "user", "parts": [{"text": run.message}]}]
    while True:
        response = client.models.generate_content(
            model="gemini-3.5-flash", contents=contents, config=config,
        )
        if not response.function_calls:
            break
        contents.append(response.candidates[0].content)
        parts = []
        for call in response.function_calls:
            # load_skill is answered by the run; your own functions by FUNCTIONS.
            # A loop of your own rather than automatic function calling, which
            # raises KeyError when the model asks for a function you did not pass.
            function = FUNCTIONS.get(call.name)
            parts.append(run.tool_result(call) or {"function_response": {
                "name": call.name, "id": call.id,
                "response": {"result": function(**(call.args or {})) if function
                             else f"There is no function named {call.name!r}."},
            }})
        contents.append({"role": "user", "parts": parts})
print(response.text)

Install

pip install "decimalai[google-genai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from google import genai
from google.genai import types
from decimalai.google_genai import agent_config

client = genai.Client()
# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")


def lookup_order(order_id: str) -> str:
    return f"Order {order_id} shipped yesterday."


# Your functions, as declarations for the model, and what answers each.
TOOLS = [{"function_declarations": [{
    "name": "lookup_order",
    "description": "Look up the status of an order.",
    "parameters_json_schema": {"type": "object", "properties": {"order_id": {"type": "string"}},
                               "required": ["order_id"]},
}]}]
FUNCTIONS = {"lookup_order": lookup_order}

question = "Where is order A1001?"
with support.run(question) as run:  # one user message, one run
    config = types.GenerateContentConfig(
        system_instruction=run.system,     # your prompt, then the skill list
        tools=[*TOOLS, *run.tool_params],  # your functions, then load_skill
    )
    contents = [{"role": "user", "parts": [{"text": run.message}]}]
    while True:
        response = client.models.generate_content(
            model="gemini-3.5-flash", contents=contents, config=config,
        )
        if not response.function_calls:
            break
        contents.append(response.candidates[0].content)
        parts = []
        for call in response.function_calls:
            # load_skill is answered by the run; your own functions by FUNCTIONS.
            # A loop of your own rather than automatic function calling, which
            # raises KeyError when the model asks for a function you did not pass.
            function = FUNCTIONS.get(call.name)
            parts.append(run.tool_result(call) or {"function_response": {
                "name": call.name, "id": call.id,
                "response": {"result": function(**(call.args or {})) if function
                             else f"There is no function named {call.name!r}."},
            }})
        contents.append({"role": "user", "parts": parts})
print(response.text)

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from google import genai
from google.genai import types

client = genai.Client()
skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
response = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Hello!",
    config=types.GenerateContentConfig(system_instruction=f"You are a helpful agent.\n\n{skill}"),
)

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Gemini API docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[langchain]" langchain-openai
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from langchain.agents import create_agent
from decimalai.langchain import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = create_agent(
    "openai:gpt-5.4-mini",
    tools=[],                         # your tools; the middleware adds load_skill
    middleware=[support.middleware],  # your prompt, the skill list and load_skill
)
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

Install

pip install "decimalai[langchain]" langchain-openai

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from langchain.agents import create_agent
from decimalai.langchain import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = create_agent(
    "openai:gpt-5.4-mini",
    tools=[],                         # your tools; the middleware adds load_skill
    middleware=[support.middleware],  # your prompt, the skill list and load_skill
)
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from langchain.agents import create_agent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = create_agent("openai:gpt-5.4-mini", tools=[], system_prompt=f"You are a helpful agent.\n\n{skill}")
result = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

LangChain docs

CrewAI's own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" "crewai>=1.15.27"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import decimalai.crewai
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                               # your own CrewAI LLM
    skills=decimalai.crewai.skills(),      # the installed folder
    tools=decimalai.crewai.skill_tools(),  # reads a skill's files, runs its scripts
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

The same install from Python, for a notebook or a setup script. CrewAI's own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" "crewai>=1.15.27"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import decimalai.crewai
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                               # your own CrewAI LLM
    skills=decimalai.crewai.skills(),      # the installed folder
    tools=decimalai.crewai.skill_tools(),  # reads a skill's files, runs its scripts
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

No DecimalAI package. CrewAI's own loader reads the folder, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from crewai import Agent, Crew, Task

agent = Agent(
    role="Support", goal="Help the customer", backstory="A support agent.",
    llm=llm,                         # your own CrewAI LLM
    skills=[Path(".agents/skills")],
)
task = Task(description="Hello!", expected_output="an answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

CrewAI docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[adk]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from decimalai.adk import DecimalAIPlugin, agent_config
from google.adk.agents import LlmAgent
from google.adk.apps import App

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = LlmAgent(
    name="support",          # ADK needs a Python identifier
    model="gemini-3.5-flash",
    instruction=support.instruction,  # your prompt, then the skill list
    tools=[*support.tools],           # your tools, then load_skill
)
# DecimalAIPlugin makes each invocation of runners built from this App one trace.
app = App(name="support", root_agent=agent, plugins=[DecimalAIPlugin()])

Prefer ADK's own loader? SkillToolset(skills=decimalai.adk.skills(), ...) reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

Install

pip install "decimalai[adk]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from decimalai.adk import DecimalAIPlugin, agent_config
from google.adk.agents import LlmAgent
from google.adk.apps import App

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = LlmAgent(
    name="support",          # ADK needs a Python identifier
    model="gemini-3.5-flash",
    instruction=support.instruction,  # your prompt, then the skill list
    tools=[*support.tools],           # your tools, then load_skill
)
# DecimalAIPlugin makes each invocation of runners built from this App one trace.
app = App(name="support", root_agent=agent, plugins=[DecimalAIPlugin()])

Prefer ADK's own loader? SkillToolset(skills=decimalai.adk.skills(), ...) reads the same folder. Use one or the other: a skill delivered both ways reaches the prompt twice.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from google.adk.agents import LlmAgent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = LlmAgent(
    name="support",
    model="gemini-3.5-flash",
    instruction=lambda context: f"You are a helpful agent.\n\n{skill}",
)

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Google ADK docs

agent_config() hands your agent the skill. With no DecimalAI key it reads your project's files and sends nothing. Needs SDK 0.27 or newer.

Install

pip install "decimalai[pydantic-ai]"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from pydantic_ai import Agent
from decimalai.pydantic_ai import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    "openai:gpt-4o-mini",
    name=support.name,
    tools=[],                 # your tools; the capability adds load_skill
    capabilities=[support],   # your prompt, the skill list and load_skill
)
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

On the Pydantic AI harness, decimalai.pydantic_ai_harness.skills(project) uses its own loader instead.

Install

pip install "decimalai[pydantic-ai]"

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from pydantic_ai import Agent
from decimalai.pydantic_ai import agent_config

# With no DECIMAL_API_KEY this reads the skills your project lists, from disk.
support = agent_config("support", fallback="You are a helpful agent.")

agent = Agent(
    "openai:gpt-4o-mini",
    name=support.name,
    tools=[],                 # your tools; the capability adds load_skill
    capabilities=[support],   # your prompt, the skill list and load_skill
)
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

On the Pydantic AI harness, decimalai.pydantic_ai_harness.skills(project) uses its own loader instead.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from pydantic_ai import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = Agent("openai:gpt-4o-mini", instructions=f"You are a helpful agent.\n\n{skill}")
# In a notebook or async code: result = await agent.run("Hello!")
result = agent.run_sync("Hello!")

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Pydantic AI docs

Deep Agents' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" deepagents
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import decimalai.deepagents
from deepagents import create_deep_agent

agent = create_deep_agent(model=model, **decimalai.deepagents.skills())
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

The default backend reads files but has no shell. To run a skill's scripts, pass skills(backend=LocalShellBackend(...)).

The same install from Python, for a notebook or a setup script. Deep Agents' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" deepagents

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import decimalai.deepagents
from deepagents import create_deep_agent

agent = create_deep_agent(model=model, **decimalai.deepagents.skills())
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

The default backend reads files but has no shell. To run a skill's scripts, pass skills(backend=LocalShellBackend(...)).

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from deepagents import create_deep_agent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = create_deep_agent(model=model, system_prompt=f"You are a helpful agent.\n\n{skill}")
agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Deep Agents docs

Its own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" agent-framework
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

import asyncio

from agent_framework import Agent
from decimalai.agent_framework import skills

agent = Agent(client=client, instructions="You are a helpful agent.", context_providers=[skills()])
asyncio.run(agent.run("Hello!"))

It caches skills by default, so restart the process after decimalai skills update.

The same install from Python, for a notebook or a setup script. Its own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" agent-framework

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

import asyncio

from agent_framework import Agent
from decimalai.agent_framework import skills

agent = Agent(client=client, instructions="You are a helpful agent.", context_providers=[skills()])
asyncio.run(agent.run("Hello!"))

It caches skills by default, so restart the process after decimalai skills update.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

import asyncio
from pathlib import Path
from agent_framework import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = Agent(client=client, instructions=f"You are a helpful agent.\n\n{skill}")
asyncio.run(agent.run("Hello!"))

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Microsoft Agent Framework docs

Strands' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" strands-agents
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

Add it to your agent

from decimalai.strands import skills
from strands import Agent
from strands.vended_tools import file_editor, shell

agent = Agent(
    model=model,                # always pass one: the default is Amazon Bedrock
    system_prompt="You are a helpful agent.",
    tools=[file_editor, shell],
    plugins=[skills()],
)
agent("Hello!")

Strands loads an agent's skills once, so build the agent again after decimalai skills update.

The same install from Python, for a notebook or a setup script. Strands' own skill loader reads the installed folder. No DecimalAI account needed.

Install

pip install "decimalai>=0.16" strands-agents

Add it to your agent

from decimalai import skillpkg

# Writes .agents/skills and .decimal/skills.toml. Safe to run twice.
result = skillpkg.add("foryourhealth111-pixel-scientific-data-preprocessing")
if result.exit_code:
    raise SystemExit(result.problems)

from decimalai.strands import skills
from strands import Agent
from strands.vended_tools import file_editor, shell

agent = Agent(
    model=model,                # always pass one: the default is Amazon Bedrock
    system_prompt="You are a helpful agent.",
    tools=[file_editor, shell],
    plugins=[skills()],
)
agent("Hello!")

Strands loads an agent's skills once, so build the agent again after decimalai skills update.

No DecimalAI package. The whole skill goes in every prompt, and nothing pins or updates it.

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Give it to your agent

from pathlib import Path
from strands import Agent

skill = Path(".agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md").read_text()
agent = Agent(model=model, system_prompt=f"You are a helpful agent.\n\n{skill}")
agent("Hello!")

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Strands docs

Adds every file the skill has, pinned so you can update it. Cursor, Copilot, Cline and Warp read this folder by themselves.

pip install "decimalai>=0.15"
decimalai skills add foryourhealth111-pixel-scientific-data-preprocessing

No Python or account. Cursor, Copilot, Cline and Warp read this folder by themselves.

Save it into your project

curl -fsSL --create-dirs -o .agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md https://app.decimal.ai/s/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Copy SKILL.md

Save it in your project, as

.agents/skills/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Cursor and other editors docs

Fetch the file and give its text to your model as instructions. No account or package needed.

Get the file

curl -fsSL https://app.decimal.ai/s/foryourhealth111-pixel-scientific-data-preprocessing/SKILL.md

Read the summary as JSON

curl -fsSL https://app.decimal.ai/s/foryourhealth111-pixel-scientific-data-preprocessing.json

This gets SKILL.md only. The skill's 5 other files are not included.

Other ways to add it

Other setups docs

You don't need a DecimalAI account.

Version: v1, tested 8/13/2026
Source: foryourhealth111-pixel/vibe-skills ↗
License: Apache-2.0

Manage it in DecimalAI

Saved prompts and skills, pinned versions, and traces of this skill in your runs. DecimalAI does not run your agent.

Claude Code reads files from your project, so DecimalAI config does not apply there.

On Claude Agent SDK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On OpenAI Agents: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On OpenAI SDK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Anthropic: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Gemini API: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On LangChain: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On CrewAI: set DECIMAL_API_KEY and add decimalai.init(crewai=True) to record runs and skill use. The skill version stays in your repo.

On Google ADK: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Pydantic AI: save the skill to an agent here, set DECIMAL_API_KEY, and the same code uses it with no command to run. Changes you save reach the next run with no redeploy.

On Deep Agents: set DECIMAL_API_KEY and add decimalai.init(langchain=True) to record runs and skill use. The skill version stays in your repo.

On Microsoft Agent Framework: set DECIMAL_API_KEY and add decimalai.init(otel=True, agent_name="support") to record runs and skill use. The skill version stays in your repo.

On Strands: set DECIMAL_API_KEY and add decimalai.init(otel=True, agent_name="support") to record runs and skill use. The skill version stays in your repo.

Editors read files from your project, so DecimalAI config does not apply there.

Anywhere else: record a use with decimalai.log_skill_activation() inside a trace.

Use this skill

SKILL.md

Scientific Data Preprocessing Skill

⚠️ CRITICAL: USER'S HARD-WON EXPERIENCE - MANDATORY CONSULTATION ⚠️

This skill encapsulates painful lessons learned from real preprocessing disasters (88.9% error rate documented). ALWAYS use this skill for planning, reflection, and validation when ANY data preprocessing is involved.

Why this skill is mandatory:

Based on actual project failures (V1.0, V2.0 case studies)
Prevents data leakage that causes production disasters
Catches semantic errors AI agents commonly make
Saves weeks of debugging and model retraining

When to invoke (DO NOT SKIP):

✅ Before starting ANY data preprocessing task
✅ During preprocessing for reflection and validation
✅ After preprocessing for comprehensive audit
✅ When reviewing AI-generated preprocessing code

Core Mission

Prevent catastrophic preprocessing errors in grouped time-series data by applying multi-level feature analysis and respecting data structure boundaries.

When to Use This Skill

MANDATORY consultation - trigger immediately when:

Data Preprocessing Tasks (ALWAYS)

Any data cleaning, transformation, or preparation work
Loading and preparing data for modeling
Creating training/test splits
Handling missing values (imputation, deletion)
Feature scaling/normalization/standardization
Encoding categorical variables
Feature engineering or construction
Feature selection or dimensionality reduction

Data Structure Types (ALWAYS)

Preprocesssing time-series data with natural groupings (matches, sessions, patients, experiments)
Sports analytics (tennis, basketball, etc.)
Medical/clinical data with patient groupings
Panel data or longitudinal studies
Any grouped/hierarchical data structure

Quality Assurance (ALWAYS)

Auditing existing preprocessing for data leakage or semantic errors
Reviewing AI-generated preprocessing code for common pitfalls
Validating preprocessing before model training
Debugging unexpected model performance

Critical Checkpoints (NEVER SKIP)

✅ BEFORE: Planning preprocessing strategy
✅ DURING: Reflecting on decisions and checking for errors
✅ AFTER: Comprehensive validation and audit

Trigger keywords that MUST invoke this skill:

"preprocess", "preprocessing", "data cleaning", "data preparation"
"standardize", "normalize", "scale", "transform"
"impute", "fill missing", "handle NaN"
"encode", "one-hot", "categorical"
"feature engineering", "feature selection", "feature construction"
"train test split", "cross validation split"
"interpolate", "smooth", "aggregate"

Not For / Boundaries

This skill does NOT:

Handle purely cross-sectional data (ungrouped, single timepoint)
Make domain-specific feature engineering decisions (you decide business logic)
Choose ML models (focuses on preprocessing only)
Handle distributed/big data infrastructure (assumes data fits in memory)

Required inputs before proceeding:

Confirmation that data has groups (e.g., match_id, patient_id, session_id)
Understanding of whether goal is within-group (relative) or cross-group (absolute) comparison
Domain constraints on data ranges/units

Quick Reference

Multi-Level Feature Analysis Framework

Level 1: Data Type

# Check data types
df.dtypes  # int64, float64, object, etc.

Level 2: Feature Type Classification

# Binary (0/1)
binary_features = [col for col in df.columns if df[col].nunique() == 2]

# Categorical (finite discrete values)
categorical_features = [col for col in df.select_dtypes(include='object').columns]

# Continuous (infinite possible values)
continuous_features = [col for col in df.select_dtypes(include=['float64', 'int64']).columns
                       if df[col].nunique() > 10]

Level 3: Data Structure

# Check for grouping
print(f"Number of groups: {df['group_id'].nunique()}")
print(f"Avg points per group: {df.groupby('group_id').size().mean():.1f}")

# Check for time-series
df_sorted = df.sort_values(['group_id', 'timestamp'])

Level 4: Physical Meaning

# Validate physical ranges
assert df['speed_mph'].max() < 200, "Speed exceeds physical limit"
assert df['distance_meters'].min() >= 0, "Negative distance impossible"

Critical Processing Decision Tree

# Decision: Within-group or global processing?
def choose_processing_scope(data, feature, goal):
    """
    goal = 'relative' → within-group (e.g., "this point was intense FOR THIS MATCH")
    goal = 'absolute' → global (e.g., "this was an intense point OVERALL")
    """
    if goal == 'relative':
        return 'within_group'
    elif goal == 'absolute':
        return 'global'
    else:
        raise ValueError("Goal must be 'relative' or 'absolute'")

Pattern 1: Within-Group Interpolation (CORRECT)

from scipy.interpolate import CubicSpline
import numpy as np

# ✅ CORRECT: Interpolate within each group
for group_id in df['match_id'].unique():
    mask = df['match_id'] == group_id
    group_data = df.loc[mask, 'speed_mph'].copy()

    # Get valid (non-NaN) indices
    valid_idx = group_data.notna()
    valid_positions = np.where(valid_idx)[0]
    valid_values = group_data[valid_idx].values

    if len(valid_positions) >= 4:
        cs = CubicSpline(valid_positions, valid_values)
        missing_positions = np.where(~valid_idx)[0]
        df.loc[mask & ~valid_idx, 'speed_mph'] = cs(missing_positions)

Pattern 2: Global Interpolation (WRONG - Don't Do This)

# ❌ WRONG: Cross-group interpolation
# This interpolates between match A's last point and match B's first point!
cs = CubicSpline(
    np.where(df['speed_mph'].notna())[0],  # ❌ All indices globally
    df['speed_mph'].dropna().values
)
df.loc[df['speed_mph'].isna(), 'speed_mph'] = cs(
    np.where(df['speed_mph'].isna())[0]
)

Pattern 3: Within-Group Standardization (for Relative Analysis)

from sklearn.preprocessing import StandardScaler

# ✅ CORRECT: Standardize within each match
for match_id in df['match_id'].unique():
    mask = df['match_id'] == match_id
    scaler = StandardScaler()

    df.loc[mask, 'distance_run_std_within'] = scaler.fit_transform(
        df.loc[mask, [['distance_run']]
    )

# Interpretation: z=+2 means "2 std above average FOR THIS MATCH"

Pattern 4: Global Standardization (for Absolute Comparison)

# ✅ CORRECT: Global standardization (when appropriate)
scaler = StandardScaler()
df['distance_run_std_global'] = scaler.fit_transform(df[['distance_run']])

# Interpretation: z=+2 means "2 std above average ACROSS ALL MATCHES"

Pattern 5: Feature Type Processing Rules

# Binary variables (0/1) - KEEP AS-IS
binary_cols = ['is_ace', 'is_winner', 'is_error']
# ❌ NEVER standardize these! They have semantic meaning as 0/1

# Categorical variables - ONE-HOT ENCODE
df_encoded = pd.get_dummies(df, columns=['server', 'serve_number'], dtype=int)

# Continuous variables - STANDARDIZE (within-group or global)
continuous_cols = ['distance_run', 'rally_count', 'speed_mph']
# ✅ Apply pattern 3 or 4 based on goal

Pattern 6: Sliding Window Features (for Momentum)

# ✅ CORRECT: Sliding window for momentum analysis
window = 10

df['win_rate_last10'] = df.groupby('match_id')['point_won'].transform(
    lambda x: x.rolling(window, min_periods=1).mean()
)

# ❌ WRONG: Cumulative features (loses temporal locality)
df['cumulative_points_won'] = df.groupby('match_id')['point_won'].cumsum()
# This just increases monotonically and correlates with point_number

Pattern 7: Data Quality Validation

def validate_data_quality(df, feature, expected_range):
    """Validate before processing"""
    # Check range
    assert df[feature].min() >= expected_range[0], f"{feature} below minimum"
    assert df[feature].max() <= expected_range[1], f"{feature} above maximum"

    # Check for anomalies
    mean = df[feature].mean()
    std = df[feature].std()

    if std > mean:
        print(f"⚠️ WARNING: {feature} has std > mean (highly skewed or errors)")

    # Check missing pattern
    missing_by_group = df.groupby('match_id')[feature].apply(lambda x: x.isna().sum())
    if missing_by_group.max() > len(df) / df['match_id'].nunique() * 0.5:
        print(f"⚠️ WARNING: {feature} has >50% missing in some groups")

# Example
validate_data_quality(df, 'speed_mph', expected_range=(50, 165))

Pattern 8: Detect Processing Scope Automatically

def detect_processing_scope(df, group_col, feature_col):
    """
    Recommend within-group vs global based on variance structure
    """
    # Calculate variance components
    within_group_var = df.groupby(group_col)[feature_col].var().mean()
    global_var = df[feature_col].var()

    # Intraclass correlation
    between_group_var = global_var - within_group_var
    icc = between_group_var / global_var

    if icc > 0.5:
        return 'within_group', f"High between-group variance (ICC={icc:.2f})"
    else:
        return 'global', f"Low between-group variance (ICC={icc:.2f})"

scope, reason = detect_processing_scope(df, 'match_id', 'distance_run')
print(f"Recommended: {scope} - {reason}")

Pattern 9: Data Leakage Detection

def detect_data_leakage(df, target_col, feature_cols, id_cols):
    """
    Critical checks for data leakage and AI common pitfalls
    """
    issues = []

    # 1. ID Leakage: High cardinality variables as features
    for col in feature_cols:
        if col in id_cols:
            issues.append(f"❌ FATAL: {col} is an ID - NEVER use as feature")
            continue

        # Check if looks like ID (>50% unique)
        uniqueness = df[col].nunique() / len(df)
        if uniqueness > 0.5:
            issues.append(f"⚠️ {col}: {uniqueness*100:.1f}% unique - possible ID leakage")

    # 2. Causal Inversion: Perfect correlation with target
    for col in feature_cols:
        if col == target_col:
            continue
        if df[col].dtype in ['int64', 'float64']:
            corr = abs(df[[col, target_col]].corr().iloc[0, 1])
            if corr > 0.95:
                issues.append(f"❌ FATAL: {col} correlation={corr:.3f} - likely consequence of target!")

    # 3. Meaningless Numeric: Codes treated as numbers
    for col in feature_cols:
        if df[col].dtype in ['int64', 'float64']:
            # Pattern: High values, many uniques, looks like code
            if df[col].min() > 1000 and df[col].nunique() > 100:
                issues.append(f"⚠️ {col}: Looks like code (zipcode/ID) - should be categorical")

    # 4. Time Travel: Check if standardization used global statistics
    # (Requires knowing if train/test split was done first)

    # Print report
    if issues:
        print("="*60)
        print("DATA LEAKAGE AUDIT")
        print("="*60)
        for issue in issues:
            print(issue)
        print("="*60)
    else:
        print("✅ No obvious leakage detected")

    return issues

# Example usage
issues = detect_data_leakage(
    df,
    target_col='point_won',
    feature_cols=['speed_mph', 'user_id', 'distance_run'],
    id_cols=['match_id', 'user_id']
)

Pattern 10: Distribution-Aware Scaling

from scipy.stats import skew, kurtosis
from sklearn.preprocessing import StandardScaler, RobustScaler

def smart_scaler_selection(df, col):
    """
    Choose scaler based on distribution characteristics
    """
    data = df[col].dropna()

    # Check distribution
    skewness = skew(data)
    kurt = kurtosis(data)

    print(f"{col}: skewness={skewness:.2f}, kurtosis={kurt:.2f}")

    if abs(skewness) < 0.5 and abs(kurt) < 3:
        # Roughly normal
        print("  → StandardScaler (data is roughly normal)")
        return StandardScaler(), None

    elif skewness > 1:
        # Right-skewed (long tail)
        print("  → Log transform + StandardScaler (right-skewed)")
        return StandardScaler(), 'log'

    else:
        # Heavy outliers or non-normal
        print("  → RobustScaler (heavy outliers)")
        return RobustScaler(), None

# Example usage
for col in continuous_features:
    scaler, transform = smart_scaler_selection(df, col)

    if transform == 'log':
        df[f'{col}_log'] = np.log1p(df[col])
        df[f'{col}_scaled'] = scaler.fit_transform(df[[f'{col}_log']])
    else:
        df[f'{col}_scaled'] = scaler.fit_transform(df[[col]])

Examples

Example 1: Tennis Match Preprocessing (Complete Pipeline)

Input:

CSV with 7,284 rows, 31 matches
Features: speed_mph, distance_run, rally_count, is_ace, server
Goal: Analyze momentum (relative intensity within each match)

Steps:

import pandas as pd
from sklearn.preprocessing import StandardScaler

# 1. Load and inspect
df = pd.read_csv('tennis_data.csv')
print(f"Matches: {df['match_id'].nunique()}")
print(f"Features: {df.dtypes}")

# 2. Classify features
binary_features = ['is_ace', 'is_winner', 'is_break_point']
categorical_features = ['server', 'serve_number']
continuous_features = ['distance_run', 'speed_mph', 'rally_count']

# 3. Validate data quality
for feat in continuous_features:
    print(f"\n{feat}:")
    print(df[feat].describe())
    # Check for impossible values
    if feat == 'speed_mph':
        assert df[feat].max() < 170, "Speed exceeds world record!"

# 4. Handle missing values (within-group)
for match_id in df['match_id'].unique():
    mask = df['match_id'] == match_id
    for feat in continuous_features:
        if df.loc[mask, feat].isna().any():
            # Simple linear interpolation within match
            df.loc[mask, feat] = df.loc[mask, feat].interpolate(method='linear')

# 5. One-hot encode categorical
df = pd.get_dummies(df, columns=categorical_features, dtype=int)

# 6. Standardize continuous features WITHIN each match
for feat in continuous_features:
    df[f'{feat}_std'] = np.nan
    for match_id in df['match_id'].unique():
        mask = df['match_id'] == match_id
        scaler = StandardScaler()
        df.loc[mask, f'{feat}_std'] = scaler.fit_transform(
            df.loc[mask, [[feat]]
        )

# 7. Create sliding window features
window = 10
df['win_rate_last10'] = df.groupby('match_id')['point_won'].transform(
    lambda x: x.rolling(window, min_periods=1).mean()
)

# 8. KEEP binary features as 0/1 (don't transform!)
# binary_features are already correct

print("\n✅ Preprocessing complete!")
print(f"Final shape: {df.shape}")
print(f"Standardized features: {[f for f in df.columns if f.endswith('_std')]}")

Expected output:

Binary features remain 0/1
Categorical features one-hot encoded (e.g., server_1, server_2)
Continuous features have both original and _std versions
_std features have mean≈0, std≈1 WITHIN each match
Sliding window features capture local momentum
No missing values

Example 2: Detecting Cross-Group Contamination

Input:

Preprocessed data where you suspect cross-group standardization

Steps:

# Check if standardization was done correctly
def check_within_group_standardization(df, group_col, feature_std_col):
    """
    Verify that standardized feature has mean≈0, std≈1 within each group
    """
    results = df.groupby(group_col)[feature_std_col].agg(['mean', 'std'])

    # Within-group standardization: each group should have mean≈0, std≈1
    if (results['mean'].abs() < 0.1).all() and (results['std'].between(0.9, 1.1)).all():
        print("✅ CORRECT: Within-group standardization detected")
        return True

    # Global standardization: groups will have varying means and stds
    else:
        print("❌ WRONG: Global standardization detected!")
        print("Group means:", results['mean'].values[:5])
        print("Group stds:", results['std'].values[:5])
        return False

check_within_group_standardization(df, 'match_id', 'distance_run_std')

Expected output:

CORRECT: All group means ≈ 0, all group stds ≈ 1
WRONG: Group means vary widely, indicating global standardization

Example 3: Fixing Cumulative Feature Error

Input:

Existing pipeline using cumulative sums for momentum

Steps:

# ❌ WRONG approach (existing code)
df['cumulative_wins'] = df.groupby('match_id')['point_won'].cumsum()

# Problem: This just counts total wins up to this point
# Doesn't capture recent momentum!

# ✅ CORRECT approach (fix)
# Replace cumulative with sliding window
window = 10
df['recent_win_rate'] = df.groupby('match_id')['point_won'].transform(
    lambda x: x.rolling(window, min_periods=1).mean()
)

# Compare
print("Cumulative (wrong):", df['cumulative_wins'].values[50:60])
print("Sliding window (correct):", df['recent_win_rate'].values[50:60])

# Cumulative: [25, 26, 26, 27, 28, ...] - monotonic
# Sliding window: [0.6, 0.7, 0.5, 0.6, ...] - fluctuates with momentum

Expected output:

Cumulative features removed
Sliding window features show local variations
Momentum analysis now captures short-term trends

References

references/index.md: Navigation and overview
references/error-case-studies.md: Real-world preprocessing disasters from tennis data
references/decision-trees.md: Full decision trees for all preprocessing choices
references/validation-checklist.md: Pre-processing validation checklist
references/ai-common-pitfalls.md: AI-specific errors (data leakage, semantic fallacies, distribution blindness)

Maintenance

⚠️ CRITICAL NOTICE: USER'S PERSONAL EXPERIENCE-BASED SKILL ⚠️

This skill is NOT theoretical - it's based on real project failures:

V1.0 disaster: 88.9% error rate, weeks of wasted work
V2.0 issues: Cross-group contamination, unreliable results
V3.0 success: All errors fixed, production-ready

Why this matters to you (Claude):

These are the EXACT errors AI agents commonly make
User has already paid the price for these mistakes
Ignoring this skill = repeating documented failures
Following this skill = learning from experience without pain

Authority level: HIGHEST

Based on user's hard-won lessons from actual project
Validated through multiple iterations (V1.0 → V2.0 → V3.0)
Every error documented with impact metrics
Every fix validated with comprehensive testing

Sources:

Primary: User's personal project (2024 MCM Problem C - Tennis Momentum Analysis)
Secondary: Statistical best practices for grouped data
Tertiary: Common AI preprocessing errors observed across domains

Mandatory consultation:

⚠️ ALWAYS consult before, during, and after any data preprocessing
⚠️ NEVER skip validation steps outlined in this skill
⚠️ When in doubt, err on the side of caution (use this skill)

Last updated: 2026-01-18 (V1.1)

Known limits:

Assumes data fits in memory (not for big data infrastructure)
Focused on numeric/categorical features (text/image preprocessing partially covered)
Does not prescribe domain-specific feature engineering (user decides business logic)
Requires basic understanding of statistics (mean, std, correlation)

About this skill

Automatic

Data Engineer pack

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v8 — 10/22/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

foryourhealth111-pixel / vibe-skills

bundled/skills/scientific-data-preprocessing/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/3/2026Version v1Bundled 5 files

Illustrative Want to see what foryourhealth111-pixel/scientific-data-preprocessing would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Related skills

Skills that do similar work, with their headline benchmark lift.

From the same source and nearby in the catalogue.

Ready to supercharge your AI agents?

SkillScore

Safety

What it changes

Track record

Use this skill

Scientific Data Preprocessing Skill

Core Mission

When to Use This Skill

Data Preprocessing Tasks (ALWAYS)

Data Structure Types (ALWAYS)

Quality Assurance (ALWAYS)

Critical Checkpoints (NEVER SKIP)

Not For / Boundaries

Quick Reference

Multi-Level Feature Analysis Framework

Critical Processing Decision Tree

Pattern 1: Within-Group Interpolation (CORRECT)

Pattern 2: Global Interpolation (WRONG - Don't Do This)

Pattern 3: Within-Group Standardization (for Relative Analysis)

Pattern 4: Global Standardization (for Absolute Comparison)

Pattern 5: Feature Type Processing Rules

Pattern 6: Sliding Window Features (for Momentum)

Pattern 7: Data Quality Validation

Pattern 8: Detect Processing Scope Automatically

Pattern 9: Data Leakage Detection

Pattern 10: Distribution-Aware Scaling

Examples

Example 1: Tennis Match Preprocessing (Complete Pipeline)

Example 2: Detecting Cross-Group Contamination

Example 3: Fixing Cumulative Feature Error

References

Maintenance

SkillSafety

Source

Related skills

More in the registry