Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jpoindexter/multimodal-voice-and-haptics
Reference-grade guide to designing beyond the screen — combining touch/visual/voice/audio/haptic modalities, conversational and LLM-agent UX, voice UI principles and error recovery, sound design, iOS/Android haptics, and TV/automotive/wearable/AR contexts.
Passed 22 of 24 test cases with the skill, and 19 of 24 without it.
and 2 more in the full report
3 fixed19 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are designing the user interface for a new AI-powered text chatbot that helps users troubleshoot software issues. Since the AI takes a few seconds to generate long responses, we want to make sure the experience feels responsive and user-friendly. Give me a bulleted list of the specific UI elements and behaviors we need to build into the chat window to handle latency and guide the user.
| Checked | Without skill | With skill |
|---|---|---|
| Recommends streaming tokens as they generate. | Passed | Passed |
| Recommends providing suggested replies or starter prompts. | Missed | Passed |
| Recommends showing inline citations or sources. | Missed | Passed |
| Recommends allowing users to edit and resend a prior message. | Missed | Passed |
The instructions your model reads when the skill loads.
Screens are one channel. Real products speak, vibrate, chime, and listen — in cars, on wrists, across rooms, hands-free. This skill covers designing for every output and input channel a human has, and how to combine them without overwhelming the user. The governing principle: pick the modality that matches the user's context and the task's nature, then layer modalities so each does what it's best at.
Every modality has a job it does well and jobs it does badly. Design by matching, not defaulting.
| Modality | Best for | Bad at | Context fit |
|---|---|---|---|
| Visual | Browsing, comparison, precision, dense info, scanning, persistent reference | Hands-busy, eyes-busy, no-screen | Desk, phone-in-hand, TV |
| Voice input | Fast input, hands-busy, eyes-busy, long dictation, search by name, "fire-and-forget" commands | Privacy, noisy rooms, precise editing, browsing options | Car, kitchen, accessibility, walking |
| Voice/audio output | Eyes-free confirmation, short answers, ambient notification, alerts | Long lists, comparison, precise data, privacy | Car, smart speaker, screen reader |
| Haptic | Confirmation, alert, texture/boundary feedback, silent notification | Conveying content, complex info | Wearable, phone-in-pocket, controller |
| Touch/gesture | Direct manipulation, precision, spatial control | Eyes-free, hands-busy, distance | Phone, tablet, touchscreen kiosk |
Rule: combine modalities so each does its strength. A smart-display timer shows remaining minutes (visual: precise), confirms "Timer set" (voice: eyes-free), and chimes when done (audio: ambient). Don't make one channel do everything.
Eyes-free vs hands-free are different constraints. Driving is eyes-busy (can glance briefly, can't read). Cooking is hands-busy (can look, can't touch). Walking-with-coffee is both. Identify which the user has before choosing.
Voice has no buttons, no menus, no visual affordances. The user cannot see what they can say. This makes discoverability the central VUI problem — and it shapes every other decision.
Good dialogue obeys four maxims. Violate one and the system feels robotic or rude.
| Maxim | Means | VUI application |
|---|---|---|
| Quantity | Say as much as needed, no more | Don't read 10 results; offer the top one + "want more?" |
| Quality | Be truthful; don't claim what you can't verify | "I'm not sure" beats a confident wrong answer |
| Relevance | Stay on the user's goal | Answer the question asked, not an adjacent one |
| Manner | Be brief, orderly, unambiguous | Short sentences, one idea each, no jargon |
Conversation alternates. The system speaks, then yields the floor clearly (a tone, a falling intonation, silence). Barge-in = let the user interrupt and speak over the prompt; experienced users know what to say and shouldn't be forced to wait. Always allow it. Detect end-of-speech with an endpointer; don't cut users off mid-sentence (tune the silence timeout — too short clips slow speakers, too long feels laggy).
test cases passed, vs 19 / 24 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.