Database Deadlock Detector

Database Deadlock Detector

Overview

Detect, analyze, and prevent database deadlocks in PostgreSQL, MySQL, and MongoDB by examining lock wait graphs, parsing deadlock log entries, identifying the application code paths that cause lock ordering conflicts, and implementing preventive patterns.

Prerequisites

• Database credentials with access to lock monitoring views (pg_locks, INNODB_LOCK_WAITS)
• psql or mysql CLI for executing diagnostic queries
• PostgreSQL: log_lock_waits = on and deadlock_timeout = 1s configured
• MySQL: innodb_print_all_deadlocks = ON for deadlock logging to error log
• Access to database error logs for deadlock event parsing
• Application source code access for identifying lock-inducing code paths

Instructions

1. Check for currently blocked transactions and their blockers:

   • PostgreSQL: SELECT blocked.pid AS blocked_pid, blocked.query AS blocked_query, blocking.pid AS blocking_pid, blocking.query AS blocking_query FROM pg_stat_activity blocked JOIN pg_locks bl ON bl.pid = blocked.pid JOIN pg_locks bl2 ON bl2.locktype = bl.locktype AND bl2.relation = bl.relation AND bl2.pid != bl.pid JOIN pg_stat_activity blocking ON blocking.pid = bl2.pid WHERE NOT bl.granted
   • MySQL: SELECT * FROM information_schema.INNODB_LOCK_WAITS

2. Parse recent deadlock events from database logs:

   • PostgreSQL: Search logs for ERROR: deadlock detected entries, which include the two conflicting queries and the lock types
   • MySQL: Run SHOW ENGINE INNODB STATUS\G and examine the LATEST DETECTED DEADLOCK section
   • Extract: transaction IDs, queries involved, tables and rows locked, and which transaction was rolled back

3. Construct the lock wait graph from the deadlock log. Map which transaction held which lock and which lock each transaction was waiting for. The circular dependency reveals the deadlock cycle. Identify the specific rows or index ranges involved.

4. Trace the deadlocking queries back to application code. Use Grep to find the SQL statements in the codebase and identify the transaction boundaries (BEGIN/COMMIT blocks or ORM transaction decorators). Map the full sequence of operations within each transaction.

5. Identify the root cause pattern:

   • Opposite lock ordering: Transaction A locks row 1 then row 2; Transaction B locks row 2 then row 1. Fix by ensuring consistent lock ordering.
   • Index gap locks (MySQL): UPDATE/DELETE on non-existent rows creates gap locks that conflict. Fix by adding the target row first or using READ COMMITTED isolation.
   • Foreign key lock escalation: INSERT into child table acquires shared lock on parent row, conflicting with UPDATE on parent. Fix by locking parent first explicitly.
   • Implicit lock promotion: SELECT with FOR UPDATE followed by UPDATE promotes shared to exclusive lock. Fix by acquiring the exclusive lock upfront.

6. Implement deadlock prevention strategies:

   • Enforce consistent lock ordering: always lock tables/rows in alphabetical or ID order within transactions
   • Minimize transaction duration: move non-database operations (API calls, file I/O) outside the transaction
   • Use SELECT ... FOR UPDATE NOWAIT or SKIP LOCKED to fail fast instead of waiting