---
name: xuzhougeng/fair-esm2
source: https://app.decimal.ai/s/xuzhougeng-fair-esm2@1/SKILL.md
source_sha256: 9cc073588443
---

# fair-esm2 — ESM-2 (Meta AI)

ESM-2 code and weights are MIT (Meta AI, github.com/facebookresearch/esm).

> **Package disambiguation.** `pip install fair-esm` gives you `import esm`
> with `esm.pretrained.*` (ESM-1/2). Biohub's github.com/Biohub/esm fork
> (MIT) gives you `from esm.models.esmfold2 import ESMFold2InputBuilder` —
> see the **`esmfold2`** skill. Both share the `esm` namespace but are
> different libraries. This skill covers **fair-esm** (the Meta package).

## Prerequisites

| Requirement | Minimum | Recommended |
| ----------- | ------- | ----------- |
| Python      | 3.8+    | 3.11        |
| CUDA        | 11.7+   | 12.x        |
| GPU VRAM    | 8 GB (8M), 16 GB (650M) | 24 GB+ (650M / 3B) |

## How to run

### Embeddings

```python
import torch, esm

model, alphabet = esm.pretrained.esm2_t33_650M_UR50D()
model = model.eval().cuda()
bc = alphabet.get_batch_converter()

_, _, toks = bc([("ubq", "MQIFVKTLTGKTITLEVEPSDTIENVK")])
with torch.no_grad():
    out = model(toks.cuda(), repr_layers=[33])
emb = out["representations"][33]      # (1, L+2, 1280) — includes BOS/EOS
seq_emb = emb[0, 1:-1].mean(0)        # per-sequence mean
```

### Masked-LM scoring

```python
with torch.no_grad():
    out = model(toks.cuda(), repr_layers=[33])
logits = out["logits"][0, 1:-1]       # (L, |vocab|)
# WT marginal log-likelihood; for mutation scoring, mask the position and
# compare logit[mut] − logit[wt].
```

### Contact prediction

```python
with torch.no_grad():
    out = model(toks.cuda(), repr_layers=[33], return_contacts=True)
contacts = out["contacts"][0]         # (L, L)
```

## Models

| Name                       | Layers | Dim  | Params | Use                        |
| -------------------------- | ------ | ---- | ------ | -------------------------- |
| `esm2_t6_8M_UR50D`         | 6      | 320  | 8 M    | Fast smoke / tiny embeddings |
| `esm2_t33_650M_UR50D`      | 33     | 1280 | 650 M  | Default embedding model    |
| `esm2_t36_3B_UR50D`        | 36     | 2560 | 3 B    | Best embeddings, 24 GB+    |

## Output format

`out["representations"][layer]` is `(B, L+2, D)`; slice `[ :, 1:-1, : ]` to
drop BOS/EOS. `out["contacts"]` (when `return_contacts=True`) is `(B, L, L)`.


## Remote compute

Needs ≥16 GB VRAM (650M model) and either pre-cached `.pt` checkpoints or
egress to `dl.fbaipublicfiles.com`. Use a selected and probed `ssh:<alias>`
context and load `remote-compute-ssh`. Confirm the `fair-esm` environment and
torch-hub cache, then submit a self-contained runner with `run_in_context`:

```json
{
  "context_id": "ssh:gpu-box",
  "title": "ESM-2 embeddings for 200 sequences",
  "command": "source ~/miniforge3/etc/profile.d/conda.sh && conda activate fair-esm && TORCH_HOME=/srv/torch-cache python embed_esm2.py --input seqs.fasta --output /home/me/wisp-results/esm2/embeddings.pt",
  "timeout_secs": 1800,
  "input_paths": ["runs/embed_esm2.py", "data/seqs.fasta"],
  "output_specs": [
    {
      "glob": "ssh://gpu-box/home/me/wisp-results/esm2/embeddings.pt",
      "kind": "pytorch",
      "residency": "remote"
    }
  ]
}
```

Replace context, environment, cache, and output paths with discovered values.
For a large input already on the server, use its absolute path instead of
staging it. Call `monitor_run` once to wait, `get_run` once for a snapshot, or
`cancel_run` to stop.


## Troubleshooting

| Symptom                                       | Cause                              | Fix                                   |
| --------------------------------------------- | ---------------------------------- | ------------------------------------- |
| `ModuleNotFoundError: No module named 'esm.models'` | You want Biohub's `esm` fork, not `fair-esm` | See `esmfold2` skill; this skill uses `esm.pretrained.*` |
| Slow first call                               | Downloading weights via torch.hub  | Set `TORCH_HOME` to a cached location |

---

**Next**: feed embeddings to a classifier. For structure prediction, use
`esmfold2`.