Running Workloads on Hugging Face Jobs

Detailed Guide

Read the detailed guide before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.

When to Use This Skill

Use this skill when users want to:

• Run Python workloads on cloud infrastructure
• Execute jobs without local GPU/TPU setup
• Process data at scale
• Run batch inference or experiments
• Schedule recurring tasks
• Use GPUs/TPUs for any workload
• Persist results to the Hugging Face Hub

Prerequisites Checklist

Before starting any job, verify:

✅ Account & Authentication

• Hugging Face Account with Pro, Team, or Enterprise plan (Jobs require paid plan)
• Authenticated login: Check with hf_whoami()
• HF_TOKEN for Hub Access ⚠️ CRITICAL - Required for any Hub operations (push models/datasets, download private repos, etc.)
• Token must have appropriate permissions (read for downloads, write for uploads)

✅ Token Usage (See Token Usage section for details)

When tokens are required:

• Pushing models/datasets to Hub
• Accessing private repositories
• Using Hub APIs in scripts
• Any authenticated Hub operations

How to provide tokens:

python
# hf_jobs MCP tool — $HF_TOKEN is auto-replaced with real token:
{"secrets": {"HF_TOKEN": "$HF_TOKEN"}}

# HfApi().run_uv_job() — MUST pass actual token:
from huggingface_hub import get_token
secrets={"HF_TOKEN": get_token()}

⚠️ CRITICAL: The $HF_TOKEN placeholder is ONLY auto-replaced by the hf_jobs MCP tool. When using HfApi().run_uv_job(), you MUST pass the real token via get_token(). Passing the literal string "$HF_TOKEN" results in a 9-character invalid token and 401 errors.

Token Usage Guide

Understanding Tokens

What are HF Tokens?

• Authentication credentials for Hugging Face Hub
• Required for authenticated operations (push, private repos, API access)
• Stored securely on your machine after hf auth login

Token Types:

• Read Token - Can download models/datasets, read private repos
• Write Token - Can push models/datasets, create repos, modify content
• Organization Token - Can act on behalf of an organization

When Tokens Are Required

Always Required:

• Pushing models/datasets to Hub
• Accessing private repositories
• Creating new repositories
• Modifying existing repositories
• Using Hub APIs programmatically

Not Required:

• Downloading public models/datasets
• Running jobs that don't interact with Hub
• Reading public repository information

How to Provide Tokens to Jobs

Method 1: Automatic Token (Recommended)

python
hf_jobs("uv", {
    "script": "your_script.py",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # ✅ Automatic replacement
})

How it works:

• $HF_TOKEN is a placeholder that gets replaced with your actual token
• Uses the token from your logged-in session (hf auth login)
• Most secure and convenient method
• Token is encrypted server-side when passed as a secret

Benefits:

• No token exposure in code
• Uses your current login session
• Automatically updated if you re-login
• Works seamlessly with MCP tools

Method 2: Explicit Token (Not Recommended)

python
hf_jobs("uv", {
    "script": "your_script.py",
    "secrets": {"HF_TOKEN": "hf_abc123..."}  # ⚠️ Hardcoded token
})

When to use:

• Only if automatic token doesn't work
• Testing with a specific token
• Organization tokens (use with caution)

Security concerns:

• Token visible in code/logs
• Must manually update if token rotates
• Risk of token exposure

Method 3: Environment Variable (Less Secure)

python
hf_jobs("uv", {
    "script": "your_script.py",
    "env": {"HF_TOKEN": "hf_abc123..."}  # ⚠️ Less secure than secrets
})

Difference from secrets:

• env variables are visible in job logs
• secrets are encrypted server-side
• Always prefer secrets for tokens

Using Tokens in Scripts

In your Python script, tokens are available as environment variables:

python
# /// script
# dependencies = ["huggingface-hub"]
# ///

import os
from huggingface_hub import HfApi

# Token is automatically available if passed via secrets
token = os.environ.get("HF_TOKEN")

# Use with Hub API
api = HfApi(token=token)

# Or let huggingface_hub auto-detect
api = HfApi()  # Automatically uses HF_TOKEN env var

Best practices:

• Don't hardcode tokens in scripts
• Use os.environ.get("HF_TOKEN") to access
• Let huggingface_hub auto-detect when possible
• Verify token exists before Hub operations

Token Verification

Check if you're logged in:

python
from huggingface_hub import whoami
user_info = whoami()  # Returns your username if authenticated

Verify token in job:

python
import os
assert "HF_TOKEN" in os.environ, "HF_TOKEN not found!"
token = os.environ["HF_TOKEN"]
print(f"Token starts with: {token[:7]}...")  # Should start with "hf_"

Common Token Issues

Error: 401 Unauthorized

• Cause: Token missing or invalid
• Fix: Add secrets={"HF_TOKEN": "$HF_TOKEN"} to job config
• Verify: Check hf_whoami() works locally

Error: 403 Forbidden

• Cause: Token lacks required permissions
• Fix: Ensure token has write permissions for push operations
• Check: Token type at https://huggingface.co/settings/tokens

Error: Token not found in environment

• Cause: secrets not passed or wrong key name
• Fix: Use secrets={"HF_TOKEN": "$HF_TOKEN"} (not env)
• Verify: Script checks os.environ.get("HF_TOKEN")

Error: Repository access denied

• Cause: Token doesn't have access to private repo
• Fix: Use token from account with access
• Check: Verify repo visibility and your permissions

Token Security Best Practices

1. Never commit tokens - Use $HF_TOKEN placeholder or environment variables
2. Use secrets, not env - Secrets are encrypted server-side
3. Rotate tokens regularly - Generate new tokens periodically
4. Use minimal permissions - Create tokens with only needed permissions
5. Don't share tokens - Each user should use their own token
6. Monitor token usage - Check token activity in Hub settings

Complete Token Example

python
# Example: Push results to Hub
hf_jobs("uv", {
    "script": """
# /// script
# dependencies = ["huggingface-hub", "datasets"]
# ///

import os
from huggingface_hub import HfApi
from datasets import Dataset

# Verify token is available
assert "HF_TOKEN" in os.environ, "HF_TOKEN required!"

# Use token for Hub operations
api = HfApi(token=os.environ["HF_TOKEN"])

# Create and push dataset
data = {"text": ["Hello", "World"]}
dataset = Dataset.from_dict(data)
dataset.push_to_hub("username/my-dataset", token=os.environ["HF_TOKEN"])

print("✅ Dataset pushed successfully!")
""",
    "flavor": "cpu-basic",
    "timeout": "30m",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # ✅ Token provided securely
})

Limitations

• Use this skill only when the task clearly matches the scope described above.
• Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
• Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.