Low Level Design/Concepts & Foundations/ Concept

Concurrency & Multithreading

Threads, locks, atomics, thread-safe collections, and producer-consumer — making an LLD design safe under load.

7m readConcurrencyThreadsLocksAtomicsThread Safety

Introduction

Concurrency and multithreading are where many otherwise clean LLD answers break under real traffic. The design question is not only which classes exist, but which objects can be touched by multiple threads, which invariants must stay atomic, and which coordination primitive is the smallest safe choice.

In Amazon, Google, Oracle, and Microsoft interviews, a strong Java answer explains visibility, mutual exclusion, atomicity, thread ownership, and shutdown behavior before naming patterns. Producer-consumer, object pools, schedulers, message queues, caches, and booking systems all depend on these ideas.

Learning Objectives

Differentiate threads, runnables, tasks, and executor-managed worker pools in Java LLD designs.
Use synchronized, intrinsic locks, volatile, atomic classes, and explicit locks with clear tradeoffs.
Apply the Java memory model to visibility, happens-before ordering, and safe publication.
Design producer-consumer flows with BlockingQueue and explain backpressure.
Identify deadlock causes and prevent them with ordering, timeouts, and narrow critical sections.
Choose safe singleton, concurrent collection, and thread pool patterns for interview designs.

Core Theory

Concurrency starts with shared mutable state

A design is concurrent when multiple operations can make progress during overlapping time. It becomes thread-sensitive when those operations touch shared mutable state, such as inventory counts, wallet balances, cache entries, booking seats, or queued tasks.

In LLD, first classify state ownership:

Thread-confined state is owned by one thread or one request and needs no lock.
Immutable state can be shared freely after safe construction.
Shared mutable state needs a coordination rule: lock, atomic variable, concurrent collection, queue, or ownership handoff.

This framing prevents overusing locks. Do not make every method synchronized. Protect the invariant that can break, and keep the critical section small.

Threads versus Runnables

A Thread is an execution vehicle managed by the JVM and operating system. A Runnable is the work to execute. In LLD, prefer modeling work as Runnable or Callable and letting an ExecutorService own thread creation, reuse, naming, and shutdown.

Extending Thread couples the task to execution policy. Implementing Runnable keeps the task reusable: it can run in a fixed pool, scheduled pool, test harness, or direct executor.

public final class EmailJob implements Runnable {
    private final String recipient;
    private final EmailClient emailClient;

    public EmailJob(String recipient, EmailClient emailClient) {
        this.recipient = recipient;
        this.emailClient = emailClient;
    }

    @Override
    public void run() {
        emailClient.sendWelcomeEmail(recipient);
    }
}

ExecutorService pool = Executors.newFixedThreadPool(4);
pool.submit(new EmailJob("user@example.com", emailClient));

The synchronized keyword and intrinsic locks

synchronized provides mutual exclusion and visibility. Every Java object has an intrinsic monitor lock. A synchronized instance method locks this; a synchronized static method locks the Class object; a synchronized block locks the object you choose.

Use it when the rule is simple: only one thread may execute this critical section at a time, and the lock is always acquired and released in a structured way. It is ideal for small counters, guarded maps, and invariant-preserving mutations.

public final class SeatInventory {
    private int availableSeats;

    public SeatInventory(int availableSeats) {
        this.availableSeats = availableSeats;
    }

    public synchronized boolean reserveOne() {
        if (availableSeats == 0) {
            return false;
        }
        availableSeats--;
        return true;
    }

    public synchronized int availableSeats() {
        return availableSeats;
    }
}

volatile and the Java memory model

volatile solves visibility and ordering, not compound atomicity. A write to a volatile field becomes visible to later reads of that field by other threads, creating a happens-before relationship.

Use volatile for simple state signals such as stop flags, readiness flags, or publishing a fully constructed immutable reference. Do not use it for read-modify-write operations such as increment, check-then-act, or balance transfer. Those need atomic classes or locks.

public final class StopSignal {
    private volatile boolean shutdownRequested;

    public void requestShutdown() {
        shutdownRequested = true;
    }

    public void runLoop() {
        while (!shutdownRequested) {
            doOneUnitOfWork();
        }
    }

    private void doOneUnitOfWork() {
        // process a small unit of work
    }
}

Atomic classes for lock-free single-variable updates

Atomic classes such as AtomicInteger, AtomicLong, and AtomicReference provide atomic read-modify-write operations using compare-and-set style CPU instructions. They are excellent for counters, sequence generators, state transitions, and one-reference swaps.

The boundary is important: atomics protect one variable or one reference. If an invariant spans multiple fields, such as debiting one wallet and crediting another, use a lock or a higher-level transaction boundary.

public final class RequestIdGenerator {
    private final AtomicLong nextId = new AtomicLong(1);

    public long next() {
        return nextId.getAndIncrement();
    }
}

public final class CircuitState {
    private final AtomicReference<State> state = new AtomicReference<>(State.CLOSED);

    public boolean openIfClosed() {
        return state.compareAndSet(State.CLOSED, State.OPEN);
    }
}

ReentrantLock and ReadWriteLock

ReentrantLock is an explicit lock with capabilities that intrinsic locks do not expose: timed acquisition, interruptible acquisition, fairness options, multiple conditions, and try-lock based recovery. Always release it in a finally block.

ReadWriteLock separates readers from writers. Many readers can hold the read lock together, but writers need exclusive access. It fits read-heavy structures such as configuration stores, catalog snapshots, and metadata registries. It can hurt write-heavy flows because coordination overhead grows.

public final class Catalog {
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    private final Map<String, Product> products = new HashMap<>();

    public Product find(String id) {
        lock.readLock().lock();
        try {
            return products.get(id);
        } finally {
            lock.readLock().unlock();
        }
    }

    public void upsert(Product product) {
        lock.writeLock().lock();
        try {
            products.put(product.id(), product);
        } finally {
            lock.writeLock().unlock();
        }
    }
}

ExecutorService and thread pools

ExecutorService decouples task submission from thread management. A pool reuses a bounded number of worker threads, which avoids creating a new operating system thread per request and gives the design a clear concurrency limit.

In interviews, name the pool type and why it fits: fixed pools for bounded parallelism, scheduled pools for delayed or periodic jobs, cached pools only for short-lived bursty tasks with strict safeguards, and custom ThreadPoolExecutor when you need queue size, rejection policy, and metrics.

public final class ThumbnailService implements AutoCloseable {
    private final ExecutorService workers = Executors.newFixedThreadPool(8);

    public Future<Thumbnail> createAsync(Image image) {
        return workers.submit(() -> createThumbnail(image));
    }

    private Thumbnail createThumbnail(Image image) {
        return new Thumbnail(image.id());
    }

    @Override
    public void close() {
        workers.shutdown();
    }
}

BlockingQueue and Producer-Consumer

BlockingQueue is the cleanest Java building block for producer-consumer designs. Producers call put or offer to enqueue work. Consumers call take or poll to receive work. The queue owns the waiting logic and can provide backpressure when capacity is bounded.

This pattern is useful in message queues, task schedulers, logging frameworks, notification systems, and object pools. The LLD goal is to keep producers unaware of consumer threads and keep consumers unaware of request threads.

public final class WorkQueue {
    private final BlockingQueue<Task> queue = new ArrayBlockingQueue<>(100);

    public void submit(Task task) throws InterruptedException {
        queue.put(task);
    }

    public void startWorker() {
        Thread worker = new Thread(() -> {
            while (!Thread.currentThread().isInterrupted()) {
                try {
                    Task task = queue.take();
                    task.run();
                } catch (InterruptedException interrupted) {
                    Thread.currentThread().interrupt();
                }
            }
        });
        worker.start();
    }
}

Object Pool as bounded concurrent ownership

The Object Pool pattern reuses expensive resources such as database connections, parser instances, or reusable buffers. In concurrent designs, the pool is also a capacity controller: only callers that borrow a resource can proceed.

A BlockingQueue often represents the idle resources. Borrow with a timeout, return in a finally block, and make double release or leaked resources visible through metrics. This is more explicit than letting every request create its own expensive object.

public final class ConnectionPool {
    private final BlockingQueue<Connection> available;

    public ConnectionPool(List<Connection> connections) {
        available = new ArrayBlockingQueue<>(connections.size());
        available.addAll(connections);
    }

    public Connection borrow(Duration timeout) throws InterruptedException {
        Connection connection = available.poll(timeout.toMillis(), TimeUnit.MILLISECONDS);
        if (connection == null) {
            throw new IllegalStateException("no connection available");
        }
        return connection;
    }

    public void release(Connection connection) {
        if (connection != null) {
            available.offer(connection);
        }
    }
}

Deadlock causes and avoidance

Deadlock appears when threads wait forever in a cycle. The classic causes are mutual exclusion, hold-and-wait, no preemption, and circular wait. In LLD, the most common bug is acquiring multiple locks in inconsistent order, such as wallet A then wallet B in one transfer and wallet B then wallet A in another.

Avoidance tactics are practical: define a global lock order, avoid calling external code while holding locks, keep critical sections short, use tryLock with timeout for recovery, and prefer one lock per invariant rather than one lock per method.

public final class WalletTransferService {
    public void transfer(Wallet from, Wallet to, long cents) {
        Wallet first = from.id().compareTo(to.id()) < 0 ? from : to;
        Wallet second = first == from ? to : from;

        synchronized (first) {
            synchronized (second) {
                from.debit(cents);
                to.credit(cents);
            }
        }
    }
}

Thread-safe Singleton

A singleton is thread-safe only when construction and publication are safe. The simplest Java options are an enum singleton or the initialization-on-demand holder idiom. Both rely on JVM class initialization guarantees.

Avoid naive lazy initialization with a plain static field because two threads can observe null and create two instances. Double-checked locking is valid only when the instance field is volatile, but the holder idiom is usually easier to explain.

public final class MetricsRegistry {
    private MetricsRegistry() {
    }

    private static final class Holder {
        private static final MetricsRegistry INSTANCE = new MetricsRegistry();
    }

    public static MetricsRegistry instance() {
        return Holder.INSTANCE;
    }
}

Concurrent collections

Concurrent collections are not just synchronized wrappers. They are designed for high-concurrency access with internal partitioning, lock-free reads, snapshot semantics, or blocking behavior.

ConcurrentHashMap is the default for shared maps with concurrent reads and updates. Use atomic methods such as computeIfAbsent, merge, and putIfAbsent instead of check-then-act sequences.

CopyOnWriteArrayList copies the array on mutation and gives readers stable snapshots. It is excellent for mostly-read listener lists and configuration observers, but poor for frequent writes or large lists.

public final class ListenerRegistry {
    private final CopyOnWriteArrayList<Listener> listeners = new CopyOnWriteArrayList<>();
    private final ConcurrentHashMap<String, Session> sessions = new ConcurrentHashMap<>();

    public void addListener(Listener listener) {
        listeners.add(listener);
    }

    public Session sessionFor(String userId) {
        return sessions.computeIfAbsent(userId, Session::new);
    }

    public void publish(Event event) {
        for (Listener listener : listeners) {
            listener.onEvent(event);
        }
    }
}

Diagrams

Producer-consumer with bounded queue backpressure

Rendering diagram…

A bounded BlockingQueue becomes the coordination point. Producers are slowed when capacity is full, consumers wait when no work exists, and neither side needs to manage the other side's threads directly.

Comparisons

synchronized vs ReentrantLock

Dimension	synchronized	ReentrantLock	LLD guidance
Lock ownership	Uses an object's intrinsic monitor	Uses an explicit lock object	Prefer synchronized for simple critical sections and ReentrantLock for advanced control
Release behavior	Automatically released when the block exits	Must be released in a finally block	If the candidate forgets finally, the design can freeze under exceptions
Timed acquisition	No built-in try with timeout	Supports tryLock and interruptible lock acquisition	Use explicit locks when deadlock recovery or graceful cancellation matters
Conditions	One wait set per intrinsic lock	Can create multiple Condition objects	Use explicit conditions for complex producer-consumer state machines
Fairness	No fairness configuration	Optional fairness constructor	Fairness can reduce starvation but may lower throughput

volatile vs atomic classes

Dimension	volatile	Atomic classes	LLD guidance
Primary guarantee	Visibility and ordering for reads and writes	Atomic read-modify-write operations	Use volatile for signals and atomics for counters or state transitions
Compound updates	Does not make increment or check-then-act atomic	Supports operations such as incrementAndGet and compareAndSet	Never protect counters with volatile alone
State scope	Best for one independent flag or reference	Best for one independent numeric value or reference	Use locks when multiple fields must change together
Common LLD example	Shutdown flag for a worker loop	Request id generator or retry counter	Explain the invariant before choosing either primitive

Best Practices

✓
Start each LLD answer by naming which state is immutable, thread-confined, or shared mutable.
✓
Use the smallest primitive that protects the invariant: volatile for visibility, atomics for one-variable updates, locks for multi-field invariants, queues for handoff.
✓
Prefer ExecutorService over manual thread creation so concurrency limits, shutdown, and monitoring have a single owner.
✓
Use bounded queues for producer-consumer flows unless the interview explicitly accepts unbounded memory growth.
✓
Always release explicit locks in finally and restore the interrupt flag after catching InterruptedException.
✓
Use ConcurrentHashMap atomic methods instead of separate contains and put calls.
✓
Keep critical sections small and never call slow network, disk, or user callback code while holding a lock.
✓
Design shutdown paths: stop accepting work, drain or cancel queued work, interrupt workers if needed, and expose completion status.

Common Mistakes

×
Assuming volatile makes increment, list updates, or check-then-act logic atomic.
×
Synchronizing every public method without understanding which invariant is being protected.
×
Creating raw threads per request instead of using a bounded executor or queue.
×
Using an unbounded queue in front of a fixed thread pool and calling it scalable.
×
Holding two locks in inconsistent order, causing rare but severe deadlocks.
×
Catching InterruptedException and continuing without restoring the thread interrupt status.
×
Using HashMap or ArrayList as shared state without synchronization or a concurrent alternative.
×
Choosing CopyOnWriteArrayList for write-heavy data and paying a copy cost on every mutation.

Quiz

0/7 answered

1.What is the best LLD distinction between a Thread and a Runnable?
2.Which guarantee does volatile provide?
3.When is ReentrantLock a better fit than synchronized?
4.Why is BlockingQueue useful in Producer-Consumer designs?
5.Which choice best avoids deadlock when two wallet locks are needed?
6.Which singleton implementation is naturally thread-safe in Java?
7.When is CopyOnWriteArrayList a good fit?

Flashcards

Cheat Sheet

Concurrency framing: Identify immutable state, thread-confined state, and shared mutable state before choosing a primitive.

Thread vs Runnable: Thread is execution. Runnable is work. Prefer Runnable or Callable submitted to ExecutorService.

synchronized: Simple mutual exclusion using intrinsic locks. Good for small invariant-preserving critical sections.

volatile: Visibility and ordering for a single field. Good for stop flags and readiness signals, not compound updates.

Atomic classes: Lock-free atomic updates for one variable. Good for counters, ids, and compare-and-set state transitions.

Explicit locks: ReentrantLock adds tryLock, timeout, interruptible acquisition, fairness, and multiple conditions. ReadWriteLock helps read-heavy data.

ExecutorService: Owns thread reuse, bounded parallelism, task submission, shutdown, and rejection behavior.

Producer-Consumer: Use BlockingQueue for safe handoff and bounded backpressure between producers and worker consumers.

Object Pool: Use a bounded pool for expensive reusable resources. Borrow with timeout and always release in finally.

Deadlock avoidance: Use global lock ordering, short critical sections, no external calls under locks, and timeouts when recovery matters.

Safe singleton: Prefer enum singleton or initialization-on-demand holder. Use volatile only if explaining double-checked locking.

Concurrent collections: Use ConcurrentHashMap for shared maps and CopyOnWriteArrayList for mostly-read listener lists.

References

BookJava Concurrency in Practice — Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea
BookEffective Java — Joshua Bloch
DocsOracle Java Documentation: java.util.concurrent
DocsOracle Java Tutorials: Concurrency