Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/transformer-architecture-guide
Guide to Transformer architectures for NLP and computer vision
Passed 23 of 27 test cases with the skill, and 23 of 27 without it.
1 fixed22 passed either way3 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
According to power-law scaling laws established by Kaplan et al. (2020), what power-law exponents govern loss scaling with model parameters N and dataset size D?
| Checked | Without skill | With skill |
|---|---|---|
| States parameters power law exponent is approximately -0.076 | Missed | Passed |
| States dataset size power law exponent is approximately -0.095 | Missed | Passed |
The instructions your model reads when the skill loads.
Understand, implement, and adapt Transformer architectures for NLP, computer vision, and multimodal research, from the original attention mechanism to modern variants.
The Transformer (Vaswani et al., 2017, "Attention Is All You Need") replaced recurrence and convolution with self-attention as the primary sequence modeling mechanism.
| Component | Function | Key Parameters |
|---|---|---|
| Multi-Head Self-Attention | Computes attention weights across all positions | d_model, n_heads, d_k, d_v |
| Feed-Forward Network | Position-wise nonlinear transformation | d_model, d_ff |
| Positional Encoding | Injects sequence order information | Sinusoidal or learned |
| Layer Normalization | Stabilizes training | Pre-norm or post-norm |
| Residual Connections | Enables gradient flow in deep networks | Add before or after norm |
import torch
import torch.nn as nn
import torch.nn.functional as F
import math
class MultiHeadAttention(nn.Module):
def __init__(self, d_model=512, n_heads=8):
super().__init__()
self.d_model = d_model
self.n_heads = n_heads
self.d_k = d_model // n_heads
self.W_q = nn.Linear(d_model, d_model)
self.W_k = nn.Linear(d_model, d_model)
self.W_v = nn.Linear(d_model, d_model)
self.W_o = nn.Linear(d_model, d_model)
def forward(self, Q, K, V, mask=None):
batch_size = Q.size(0)
test cases passed, vs 23 / 27 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.