--- name: rampstackco/vendor-evaluation source: https://app.decimal.ai/s/rampstackco-vendor-evaluation@2/SKILL.md source_sha256: a963dcab9325 --- # Vendor Evaluation Pick the right tool or service, negotiate fair terms, and avoid the lock-in traps. Stack-agnostic. Applies to SaaS, infrastructure providers, agencies, and any external dependency. --- ## When to use - Selecting a tool or vendor for a new need - Evaluating alternatives to a current vendor - Build vs buy analysis - Renewal coming up: should we stay or switch? - Running a formal RFP or RFI - Comparing finalists in a vendor selection - Negotiating a contract - Assessing vendor risk (financial, security, dependency) ## When NOT to use - General cost reduction (use `cost-optimization`) - Specific contract legal terms (those go to legal) - Performance issues with an existing vendor (try fixing before switching) - Hiring an agency for a one-off project (lighter framework needed) --- ## Required inputs - The need (what problem are you solving, what would success look like) - Constraints (budget, timeline, integration requirements) - Stakeholders (users, IT, security, finance, legal) - Existing context (what's already used, what's been tried) - Compliance requirements --- ## The framework: 5 phases A structured vendor evaluation. Skip phases at your peril. ### Phase 1: Define the need Before looking at vendors, define what you actually need. - What problem are you solving? - What's the user / use case? - What does "success" look like in 6 months? In 2 years? - What's the budget (range, not just ceiling)? - What's the timeline? - Are there must-have integrations or constraints? The temptation: skip this and start demoing. Vendors are happy to show off; you end up choosing what looks shiny rather than what fits. ### Phase 2: Build vs buy Before evaluating vendors, decide whether you should build instead. **Build when:** - It's core to the business (differentiating) - The need is so specific no vendor matches - The economics work at your scale - You have the team to maintain it - Vendor lock-in would be unacceptable **Buy when:** - It's table stakes (not differentiating) - The need is well-served by existing products - The economics favor it - The team should focus elsewhere - The vendor's specialization beats your generalism Most teams over-build. The rule of thumb: buy unless there's a strong reason to build. Then question even that strong reason. ### Phase 3: Generate the shortlist Cast a wider net than feels comfortable, then narrow. Sources: - Internal team's existing knowledge - Industry analyst reports (Gartner, Forrester, etc.) - Peer recommendations (other companies similar to yours) - Reviews (G2, Capterra; with caveats about review quality) - Adjacent vendors you already use (often have the feature you need) - Open-source alternatives Cast wide first. Aim for 5-8 candidates. Then narrow to 2-4 finalists for deep evaluation. ### Phase 4: Score the finalists Use a scorecard. Without one, you'll be swayed by demo theatrics or who has the friendliest sales rep. Scorecard dimensions (weight by your situation): **Functional fit (40%):** Does it do what you need? Edge cases handled? UX quality. Workflow fit. **Technical fit (15%):** Integration with your stack. API quality and completeness. Data export and portability. Performance at your scale. Self-hosted, hybrid, or SaaS-only. **Operational fit (10%):** Onboarding effort. Training and adoption. Documentation quality. Support quality (test by submitting a ticket). SLAs. **Security and compliance (10%):** SOC 2, ISO 27001, HIPAA, etc., as applicable. Data residency. Encryption at rest and in transit. Access controls and audit logs. Penetration test results (ask). Subprocessors. **Vendor health (10%):** Years in business. Funding and runway (or revenue if private). Customer base size and similar customers. Public references. Roadmap visibility. **Cost (10%):** License or subscription cost. Implementation and onboarding cost. Training cost. Integration cost. Opportunity cost (in-house resource time). Switching cost (in case of failure). **Lock-in risk (5%):** Data export quality. Standard formats vs proprietary. Migration paths to alternatives. Open standards alignment. Contract escape clauses. Score each finalist 1-5 on each dimension. Multiply by weight. Sum. The score isn't gospel. It surfaces the tradeoffs. ### Phase 5: Negotiate Most enterprise contracts are negotiable. Most aren't negotiated. **What's negotiable:** - Price (multi-year, volume, prepayment, end-of-quarter timing) - Terms (payment schedule, renewal terms) - Success commitments (training, onboarding, support) - SLAs (uptime, response time, credits) - Termination clauses (auto-renewal, notice period, data export) - Liability caps and indemnity (legal will care about these) - Subprocessors and data handling (security/legal cares) **Common negotiation moves:** - Multi-year discount (3-year for a 15-30% discount is common) - Volume tiers (commit to higher usage for a per-unit discount) - Annual prepayment for a discount - Free or discounted onboarding - Pilot pricing (trial period at reduced rate) - "Most favored customer" clauses (if your size warrants) **What to avoid:** - Multi-year lock with no escape clause for material breach - Auto-renewal with short notice window (under 60 days; want longer) - "All right, no further negotiation" stance from the start - Signing without legal review --- ## Workflow ### Step 1: Define the need Write a one-page brief: what we need, why, success criteria, constraints, stakeholders. ### Step 2: Build vs buy Honestly answer the build/buy question. Document the rationale. ### Step 3: Generate shortlist Wider net first, narrowed via desk research: - Read summaries - Skim reviews - Look at customer logos - Skim docs - Skim API specs Eliminate obvious misfits. Land on 2-4 finalists. ### Step 4: Run demos and trials For each finalist: - Demo with the use case (don't take their default demo; bring yours) - Trial period if possible - Reference calls with similar customers - Pilot with real data if feasible Don't be charmed by the polished demo. Try it with your real workflow. ### Step 5: Run security and compliance review Critical for any vendor handling sensitive data: - Request SOC 2 / ISO 27001 reports - Review their security questionnaire (most have one ready) - Verify data handling matches your requirements - Identify subprocessors This can take weeks for enterprise vendors. Start early. ### Step 6: Score Apply the scorecard. Do this collaboratively with stakeholders. The scoring conversation matters more than the final number. It surfaces disagreement (one person scored UX 5, another scored 2: why?). ### Step 7: Negotiate With the apparent winner: - Open negotiation by asking for terms (not just price) - Be willing to walk - Run negotiations with #2 in parallel where appropriate (gives you leverage) ### Step 8: Plan the rollout Contract signing is the start, not the end. Plan: - Onboarding owner - Training plan - Migration plan if replacing an incumbent - Success criteria at 30, 90, 180 days - Renewal calendar ### Step 9: Document Record: - The decision and the rationale - Alternatives considered - Scorecard results - Negotiated terms - Renewal date and notice deadlines - Owner of the relationship This is gold for the next renewal or the next similar evaluation. --- ## Failure patterns **Skipping the needs definition.** Demoing first. Buying what's shiny. Realizing 6 months in that the actual need wasn't met. **Single-source decisions.** Talking to one vendor; deciding. No comparison. Probably overpaying or under-fitting. **Charisma-driven decisions.** Buying based on the sales rep's likability. The product is what you'll use for years; the rep won't be there. **Reference calls that the vendor curated.** Of course their references love them. Find references the vendor didn't suggest. **Glossing over security.** Security review skipped because of timeline pressure. Then a breach. Slow down or accept the risk explicitly. **Demos that don't match the use case.** Their default demo, not yours. Always do a use-case demo. **Trial that doesn't simulate real usage.** A trial with synthetic data tells you the product works in synthetic conditions. Use real (or close to real) data. **Negotiating only on price.** Terms, SLAs, and exit clauses matter more for long-term satisfaction than 5% price. **Auto-renewal without notice tracking.** Renewal happens; rate goes up 15%. No one was watching. Track renewals; review with notice. **Lock-in without exit plan.** Tightly integrating into a vendor's proprietary surface. When you want to leave, you can't. Plan exit at the start. **Multi-year contract for an unproven vendor.** Save the multi-year for vendors you trust. New vendor: shorter term, evaluate after. **No internal champion.** Tool selected; no one drives adoption. Tool sits unused. Identify the champion before signing. **Negotiating after a verbal commitment.** "Yes, we want to buy" means they have less reason to negotiate. Keep options open until terms are settled. **Ignoring red flags in security review.** Vendor's security responses are evasive or incomplete. Treat as a no. **Comparing apples to oranges.** Vendors price differently (per user, per usage, flat). Build a comparable cost model at your scale. --- ## Output format A vendor evaluation document includes: - **Need brief:** problem, success criteria, constraints - **Build vs buy decision:** with rationale - **Shortlist:** 2-4 finalists with brief description - **Scorecard:** filled out per finalist, or state the gap per the data-availability rule - **Demo and trial notes:** what was learned - **Security and compliance summary:** findings per finalist - **Reference call notes:** what customers said - **Recommendation:** which vendor, with rationale - **Negotiated terms:** what was agreed - **Rollout plan:** onboarding, training, migration - **Renewal calendar:** with notice deadline --- ## If required data is unavailable This skill's output depends on data, measurements, or tool results it cannot generate on its own. When a required input, tool, or data source is unavailable or unverifiable, the sanctioned output is the deliverable with the gap stated: what was needed, what was actually obtained or verified, and which parts of the output are affected. Fabricating, estimating, or interpolating a required number to complete the deliverable is never sanctioned. A stated gap is a complete answer. --- ## Reference files - [`references/evaluation-rubric.md`](references/evaluation-rubric.md) - Scoring template with weighted dimensions, 1-5 scale criteria for each dimension, and a worked vendor-comparison example.