Inno Prepare Resources

Inputs

Read from instance.json. Path values are absolute when the project is created by Dr. Claw; use as-is. If relative (e.g. hand-edited), resolve with path.join(project_path, value).

| Parameter | Required | Description | |------------------------|----------|-------------| | instance | Yes | Path to the instance JSON file (absolute in Dr. Claw). Use as-is to read the file. File contains source_papers, task1/task2, etc. | | task_level | Yes | Which task field to read from the instance — "task1" (Plan) or "task2" (Idea) | | Ideation.references | Yes | Path to Ideation references dir (absolute in Dr. Claw) — for downloaded papers and prepare logs | | Experiment.code_references | Yes | Path (absolute in Dr. Claw) — for cloned repos | | Experiment.datasets | Yes | Path (absolute in Dr. Claw) — for dataset files | | category | Yes | Research category tag (e.g. nlp_qa, gnn, recommendation). Used to locate the built-in dataset metaprompt | | references | Yes | A pre-formatted string listing all source papers from the instance | | context_variables | Yes | Shared context dictionary; this step will write date_limit into it | | ideas | No | Full innovative-idea / plan text. Provide only in Plan mode — when present the Prepare Agent query includes the ideas for more targeted repo selection | | dataset_description | No | Pre-built dataset description from the orchestrator (for custom / user-provided datasets). When provided, skip the metaprompt import in Step 3 |

Outputs

| Output | Description | |------------------------|-------------| | prepare_res | Full text response from the Prepare Agent (contains selected reference repositories and reasoning) | | download_res | Result log from downloading arXiv paper sources to local disk | | dataset_description | Composed prompt string describing the datasets, baselines, comparisons, and evaluation metrics | | data_module | The imported metaprompt module object (Idea mode). In Plan mode this is not returned | | context_variables | Updated with date_limit (str, YYYY-MM-DD) |

Cache file outputs

Every intermediate result must be persisted as a JSON file under Ideation/references/logs/. The directory layout follows:

Ideation/references/logs/
├── load_instance.json                  ← written by orchestrator
├── github_search.json
├── download_arxiv_source_by_title.json
└── prepare_agent.json

> Ideation/references/logs/load_instance.json is written by the orchestrator before this skill runs — do not overwrite it.

Tool cache format (tools/*.json)

Each tool output file records the function call arguments and result:

json
{
  "name": "<tool_name>",
  "args": { ... },
  "result": <result_value>
}

github_search.json — written after Step 2:

json
{
  "name": "github_search",
  "args": {
    "metadata": {
      "source_papers": [ ... ],
      "task_instructions": "...",
      "date_limit": "YYYY-MM-DD"
    }
  },
  "result": "<concatenated github_result string>"
}

download_arxiv_source_by_title.json — written after Step 6:

json
{
  "name": "download_arxiv_source_by_title",
  "args": {
    "paper_list": ["paper title 1", "paper title 2"],
    "references_path": "<instance.Ideation.references if absolute, else path.join(project_path, instance.Ideation.references)>"
  },
  "result": "<download result log string>"
}

Agent cache format (agents/*.json)

Each agent output file records the final context variables (no conversation messages):

json
{
  "context_variables": {
    "references_path": "<use instance.Ideation.references as-is if absolute, else path.join(project_path, ...)>",
    "code_references_path": "<use instance.Experiment.code_references as-is if absolute, else path.join(project_path, ...)>",
    "datasets_path": "<use instance.Experiment.datasets as-is if absolute, else path.join(project_path, ...)>",
    "date_limit": "YYYY-MM-DD",
    "prepare_result": {
      "reference_codebases": ["repo1", "repo2"],
      "reference_paths": ["Experiment/code_references/repo1", "Experiment/code_references/repo2"],
      "reference_papers": ["paper title 1", "paper title 2"]
    }
  }
}

prepare_agent.json — written after Step 4–5. Contains the final context_variables with prepare_result holding reference_codebases, reference_paths, and reference_papers.

Step-by-step Instructions

Step 1 — Load the evaluation instance

Call load_instance(instance.instance, task_level) — when created by Dr. Claw, instance.instance is already absolute; otherwise resolve with path.join(project_path, instance.instance).

This reads the instance JSON and returns an EvalMetadata object containing:

• source_papers — list of dicts, each with reference, rank, type, justification, usage
• task_instructions — the task description text (from the field named by task_level)
• date_limit — the publication date of the target paper (fetched from arXiv via the instance url); defaults to "2024-01-01" if metadata cannot be retrieved

Write date_limit into context_variables["date_limit"].

> Note: Ideation/references/logs/load_instance.json should already exist — it was written by the orchestrator. If not, write it now following the tool cache format.

> Graceful handling: If the instance JSON was constructed by the orchestrator and has no url or an empty source_papers list, use a sensible default date_limit and continue — do not raise an error.

Step 2 — Search GitHub for related repositories

Call github_search(metadata).

Iterate over every entry in metadata["source_papers"]. For each paper, use its reference (title) as the search query and call the GitHub Search Repositories API:

GET https://api.github.com/search/repositories?q=<paper_title>&per_page=10&page=1

From each result item, extract:

• name: {owner}/{repo}
• description: repository description
• link: html_url

Format each paper's results as a human-readable block:

Here are some of the repositories I found on GitHub:
    Name: owner/repo
    Description: ...
    Link: https://github.com/owner/repo

Concatenate all papers' results into a single github_result string, using a ****************************** separator between papers.

Rate-limit handling: wait ~2 seconds between consecutive GitHub API calls to avoid HTTP 403 throttling. The last paper does not need a delay.

> Fallback when source_papers is empty: use keywords extracted from task_instructions as the search query instead, and perform a single GitHub search call to find relevant repositories.

Save → Ideation/references/logs/github_search.json

Step 3 — Build the dataset description

Choose one of the following strategies (in priority order):

1. Orchestrator override — if dataset_description was passed as input, use it directly.
2. Built-in metaprompt — if category maps to an existing metaprompt module, import it and compose the description from its five fields:

• TASK — what the task is about
• DATASET — dataset files, structure, and loading instructions
• BASELINE — representative baseline methods
• COMPARISON — performance comparison table
• EVALUATION — evaluation metrics and scoring functions
• REF — additional references and notes

The composed description follows this template:

You should select SEVERAL datasets as experimental datasets from the following description: {DATASET}

We have already selected the following baselines for these datasets: {BASELINE}

The performance comparison of these datasets: {COMPARISON}

And the evaluation metrics are: {EVALUATION}

{REF}

1. Manual / minimal — if neither of the above is available, construct a minimal description from what is known about the task (e.g. from task_instructions), or ask the user for additional information.

Also retain the data_module object (the imported metaprompt module) for later use in Idea mode.

Step 4 — Query the Prepare Agent

Build the query depending on mode:

• Idea mode (no ideas provided):

You are given a list of papers, searching results of the papers on GitHub. List of papers: {references}

Searching results of the papers on GitHub: {github_result}

Your task is to choose at least 5 repositories as the reference codebases. Note that this time there is no innovative ideas, you should choose the most valuable repositories as the reference codebases.

• Plan mode (ideas provided):

You are given a list of papers, searching results of the papers on GitHub, and innovative ideas according to the papers. List of papers: {references}

Searching results of the papers on GitHub: {github_result}

innovative ideas: {ideas}

Your task is to choose at least 5 repositories as the reference codebases.

Send the query to the Prepare Agent and record the full response as prepare_res.

Save → Ideation/references/logs/prepare_agent.json (final context_variables only, no messages)

Step 5 — Extract reference paper list

Parse prepare_res to extract a JSON object containing "reference_papers" (a list of paper title strings the agent selected).

Use bracket-matching JSON extraction — find the first complete {…} in the text, parse it, and read reference_papers.

Fallback: if reference_papers is empty (e.g. the agent found no GitHub repos and returned nothing), fall back to the original source_papers titles from the instance metadata so that paper download can still proceed.

Step 6 — Download arXiv paper sources

Call download_arxiv_source_by_title(paper_list, references_path) where references_path is instance.Ideation.references (absolute in Dr. Claw) or path.join(project_path, instance.Ideation.references) if relative.

This searches arXiv for each paper title, downloads the LaTeX / source archive, and extracts it into Ideation/references/papers/. Record the result log as download_res.

Save → Ideation/references/logs/download_arxiv_source_by_title.json

Checklist

• ] load_instance called; date_limit written to context_variables (default used if unavailable)
• ] github_search completed; result saved → Ideation/references/logs/github_search.json
• ] dataset_description built (from orchestrator override, built-in metaprompt, or manual construction)
• ] Prepare Agent queried; conversation saved → Ideation/references/logs/prepare_agent.json
• ] reference_papers extracted from Prepare Agent output; fallback to source papers if empty
• ] arXiv paper sources downloaded to Ideation/references/papers/; result saved → Ideation/references/logs/download_arxiv_source_by_title.json
• ] All cache files written under Ideation/references/logs/