Object Pool

Purpose

Some objects are expensive to come by. A database connection costs a TCP handshake, a TLS negotiation, an authentication round trip and, on PostgreSQL, a forked backend process on the server. A thread costs a stack and a kernel object. A DMA capture buffer costs pinned physical memory that has to be reserved before the system is doing anything useful. Creating one of those per request and throwing it away is not a small waste, it is usually the dominant cost of the request.

A pool keeps a supply of them already made. A caller acquires one, uses it exclusively, and releases it back. The borrower gets no say in construction and does not destroy the object when finished; it hands it back for the next borrower. The second half of that sentence is the entire pattern and the entire difficulty, because nothing in most languages will make a borrower give anything back.

Design

A pool holds a free list and, usually, a record of what is currently lent out. acquire takes an object off the free list, or creates one if the pool is below its maximum size, or waits, or fails, and that policy choice is the most consequential decision in the whole implementation. release resets the object and puts it back.

Reset is where pools get dangerous. The object coming back carries whatever the last borrower did to it: an open transaction, a changed timeout, a cached result, a session variable. Either the pool scrubs it on release or every borrower must not care, and "must not care" is not a property you can enforce or even easily test. Add a field to the pooled class two years later and the reset routine will not know about it.

Validation on the way out matters for anything with a peer at the other end. A pooled connection can be perfectly valid in your process and long since closed by the server or a firewall idle timer, so a pool that hands out connections without checking them turns an infrastructure timeout into a random application error somewhere far away.

Example

The Autobot Robotics office-side reporting service builds shift summaries by querying a PostgreSQL database that holds job history for every cell a customer runs. The original code opened a connection per report and closed it at the end, which is the correct instinct and was fine while there were a dozen cells. The overnight run for a large customer opens several thousand of them. Connection setup dominated the run time, and worse, PostgreSQL runs a separate backend process per connection and enforces a hard max_connections, so the reporting job was periodically pushing the server to its limit and causing connection failures in the unrelated ERP that shares that database. The cost of the pattern's absence was not slowness, it was an outage in something else.

A connection pool with a fixed maximum fixed both problems at once, because a pool is also a limiter: the reporting service now cannot open more than its share no matter how many reports are queued. Reports queue instead of the database failing, which is the correct way for a system to behave under overload.

The vision system pools for a different reason entirely. It reserves its camera capture buffers at start-up, because they must be physically contiguous and page-locked for the capture hardware to write into, and asking the operating system for one of those while a part is moving down the line is not something you can rely on. There the pool is not an optimization, it is how you guarantee the resource exists at all. Someone later tried pooling the plain Report objects on the office side by the same reasoning, and it was slower than allocating them, plus it produced a bug where a report showed the previous customer's site name because one field was missed in the reset.

class ConnectionPool
{
    private readonly Stack<DbConnection> free;
    private int lent;
    private readonly int max;

    public DbConnection acquire()
    {
        // Hand back a live one, make a new one, or make the caller wait.
        // Which of the three is the only interesting decision here.
        if (free.Count > 0)
        {
            DbConnection c = free.Pop();
            if (!c.isAlive()) { c.destroy(); return openNew(); }
            lent = lent + 1;
            return c;
        }
        if (lent < max) { lent = lent + 1; return openNew(); }
        return waitForRelease();
    }

    public void release(DbConnection c)
    {
        c.rollbackIfOpen();     // scrub the previous borrower's state
        c.resetSessionVars();
        lent = lent - 1;
        free.Push(c);
    }
}

class ShiftReport
{
    private readonly ConnectionPool pool;

    public void build(int cellId)
    {
        DbConnection c = pool.acquire();
        try
        {
            // run the queries, assemble the summary
        }
        finally
        {
            pool.release(c);    // the one line the whole pattern depends on
        }
    }
}

Seen in the Wild

Connection pooling is the most-deployed instance of this pattern in existence: .NET's SqlClient pools connections by default, HikariCP and Apache Commons Pool do the same job in Java, and PgBouncer sits outside the application entirely and pools PostgreSQL connections for everything that talks to it. Thread pools are the other big one, with the .NET ThreadPool and Java's ThreadPoolExecutor both existing because creating a thread per task does not scale. .NET's ArrayPool<T> and Netty's PooledByteBufAllocator pool buffers specifically to keep large short-lived allocations away from the garbage collector. Go's http.Transport keeps a pool of keep-alive connections per host, which is why an idle HTTP client in Go is holding sockets.

Comparisons

• Flyweight is the one to get straight, because the two are constantly confused. A pool lends an object exclusively and takes it back; a flyweight is shared by everybody at once and is never returned, because it was never borrowed. A pooled object is mutable and gets reset between users; a flyweight is immutable precisely so it can be used simultaneously. A pool exists to avoid the cost of construction, a flyweight to avoid the cost of duplication. If two callers can hold it at the same time it is a flyweight, and if one caller must give it back it is a pool.
• Singleton is how the pool itself is usually reached, and it inherits every one of Singleton's problems when done that way. Inject the pool instead; there is no reason a test should share the production one.
• Proxy is the standard trick for making release automatic: hand out a wrapper whose close returns the object to the pool rather than destroying it, which is exactly what pooled JDBC and ADO.NET connections do. It is also why calling Close on a pooled connection is cheap and correct rather than wasteful.
• Multiton keeps one instance per key permanently. A pool keeps many interchangeable instances and cares which are free, not which is which.
• Lazy Load describes the growth policy a pool usually adopts: create nothing at start-up, create on first demand, up to a ceiling.

Why It Exists

Criticisms

The pattern reintroduces manual memory management on top of a runtime that was built to remove it, and it does so without any of the tooling. A borrower that forgets to release leaks a pooled object, and the symptom is not a crash but a hang: some time later, under load, every caller blocks in acquire forever. That is harder to diagnose than a memory leak and it only appears when the system is busy, which is to say in production. Languages help a little here (using and try-with-resources exist for exactly this) but nothing prevents a reference from escaping the block and being used after it was returned, which is a use-after-free with extra steps.

State leakage between borrowers is the other recurring bug and it is worse than it sounds, because the failures are silent and correlated with what someone else did. A pooled object that keeps a customer identifier, a transaction, a locale or a cached row hands that to the next user. In a multi-tenant system that is a data disclosure bug, and it is created by adding a field to a class, which is not a change anybody thinks to review against a reset routine in a different file.

Finally, pool sizing is guesswork that is invisible until it is not. Too small and everything queues behind the pool while the resource sits idle; too large and you have simply moved the overload downstream to the thing you were trying to protect, which now falls over with a more confusing error. There is no correct static answer, since it depends on concurrency, hold time and what else shares the resource, and every default value shipped in every pooling library is a guess that was right for somebody else's workload.