---
name: mkurman/sglang
source: https://app.decimal.ai/s/mkurman-sglang@1/SKILL.md
source_sha256: e09d20df7e2c
---

## Overview

SGLang is a fast LLM inference and serving framework with structured generation (constrained decoding), RadixAttention for prefix caching, and an OpenAI-compatible API. Particularly strong for reasoning models (DeepSeek R1, QwQ) and guided generation with JSON schema, grammar, and regex constraints.

## Installation

```bash
uv pip install sglang[all]
```

## Quick Start

```python
import sglang as sgl

@sgl.function
def multi_turn(s, question):
    s += sgl.system("You are a helpful assistant.")
    s += sgl.user(question)
    s += sgl.assistant()

state = multi_turn.run(question="What is the derivative of x^2?")
print(state["answer"])
```

## Server Mode

```bash
python -m sglang.launch_server --model-path Qwen/Qwen2.5-1.5B-Instruct --port 30000
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="none")
```

## Structured Generation

```python
@sgl.function
def json_gen(s):
    s += "Generate a person's info in JSON."
    s += sgl.gen("json_output", max_tokens=128, 
                 schema='{"type": "object", "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}}')

state = json_gen.run()
print(state["json_output"])
```

## References
- [SGLang docs](https://sgl-project.github.io/)
- [SGLang GitHub](https://github.com/sgl-project/sglang)