Tuning JVM Heap for GDS Projections

The failure has a recognisable shape. A routing endpoint works in staging against a city extract, ships, runs fine for a week, and then one request against a larger region takes the whole instance down with an OutOfMemoryError — not the request, the instance. Every other tenant’s queries die with it, the container restarts cold, and the next few minutes are served at storage latency while the cache reloads. The cause is almost never the algorithm. It is that a Graph Data Science projection was built without anyone asking what it would weigh, and that the results were collected onto the heap instead of streamed off it. This page covers both halves: estimating a projection before committing to it, and consuming its output without materialising it.

Prerequisites & Versions

Estimation procedures require GDS 2.x; the async streaming pattern needs a 5.x driver.

Requirement Minimum version Install
Python 3.11 —
neo4j (async driver) 5.20 pip install "neo4j>=5.20"
Neo4j Server 5.15 —
Graph Data Science 2.6 gds.graph.project.estimate

Implementation

The guard below refuses to project a graph it cannot afford, and streams the algorithm’s results rather than collecting them. Both halves matter: the estimate keeps the projection from exhausting off-heap memory at build time, and the streaming consumption keeps the results from exhausting the heap afterwards.

import asyncio
from dataclasses import dataclass
from typing import AsyncIterator

from neo4j import AsyncGraphDatabase
from neo4j.exceptions import ClientError


class ProjectionTooLarge(RuntimeError):
    """Raised before any memory is committed, rather than after it runs out."""


@dataclass(frozen=True)
class Estimate:
    bytes_min: int
    bytes_max: int
    node_count: int
    relationship_count: int

    def headroom_against(self, budget_bytes: int) -> float:
        return budget_bytes / self.bytes_max if self.bytes_max else float("inf")


class SafeProjection:
    """Project, run, drop — with the size checked first and the results streamed.

    The budget is the memory this service is *allowed* to spend on projections,
    which is not the same as the memory currently free: two concurrent requests
    that each check "is there room right now" will both say yes and both build.
    """

    def __init__(
        self,
        uri: str,
        auth: tuple[str, str],
        budget_bytes: int,
        min_headroom: float = 1.5,
    ) -> None:
        self._driver = AsyncGraphDatabase.driver(uri, auth=auth)
        self._budget = budget_bytes
        self._min_headroom = min_headroom
        self._lock = asyncio.Lock()

    async def close(self) -> None:
        await self._driver.close()

    async def estimate(self, node_spec, rel_spec) -> Estimate:
        async with self._driver.session() as session:
            result = await session.run(
                "CALL gds.graph.project.estimate($nodes, $rels) "
                "YIELD bytesMin, bytesMax, nodeCount, relationshipCount "
                "RETURN bytesMin, bytesMax, nodeCount, relationshipCount",
                nodes=node_spec,
                rels=rel_spec,
            )
            record = await result.single()
        return Estimate(
            bytes_min=int(record["bytesMin"]),
            bytes_max=int(record["bytesMax"]),
            node_count=int(record["nodeCount"]),
            relationship_count=int(record["relationshipCount"]),
        )

    async def shortest_path(
        self, graph_name: str, node_spec, rel_spec, source_id: str, target_id: str
    ) -> AsyncIterator[dict]:
        estimate = await self.estimate(node_spec, rel_spec)
        headroom = estimate.headroom_against(self._budget)
        if headroom < self._min_headroom:
            raise ProjectionTooLarge(
                f"{estimate.node_count:,} nodes / {estimate.relationship_count:,} rels "
                f"would need up to {estimate.bytes_max / 1024 ** 3:.1f} GiB against a "
                f"{self._budget / 1024 ** 3:.1f} GiB budget (headroom {headroom:.2f}×)"
            )

        # One projection at a time per process: the estimate answers "does this
        # fit", not "does this fit alongside whatever else is being built".
        async with self._lock:
            async with self._driver.session() as session:
                await session.run(
                    "CALL gds.graph.project($name, $nodes, $rels)",
                    name=graph_name, nodes=node_spec, rels=rel_spec,
                )
                try:
                    result = await session.run(
                        """
                        MATCH (s:RoadNode {id: $source}), (t:RoadNode {id: $target})
                        CALL gds.shortestPath.dijkstra.stream($name, {
                          sourceNode: s, targetNode: t,
                          relationshipWeightProperty: 'drive_s'
                        })
                        YIELD nodeIds, totalCost
                        RETURN totalCost,
                               [nid IN nodeIds | gds.util.asNode(nid).id] AS route
                        """,
                        name=graph_name, source=source_id, target=target_id,
                    )
                    # `async for` pulls records in batches as they arrive. A
                    # `collect()` in Cypher, or a list() here, would put the whole
                    # result on the heap first — which is the second way this
                    # endpoint kills the instance.
                    async for record in result:
                        yield {"cost": record["totalCost"], "route": record["route"]}
                finally:
                    # `false` means "do not fail if it is already gone", so a
                    # failed projection cannot strand a partially built graph.
                    await session.run(
                        "CALL gds.graph.drop($name, false)", name=graph_name
                    )


async def main() -> None:
    NODES = "RoadNode"
    RELS = {"SEGMENT": {"properties": "drive_s"}}

    projection = SafeProjection(
        "neo4j://localhost:7687", ("neo4j", "password"),
        budget_bytes=6 * 1024 ** 3,
    )
    try:
        async for hop in projection.shortest_path(
            "route-req-8841", NODES, RELS, "junction:4471", "junction:9902"
        ):
            print(f"{hop['cost']:.0f}s over {len(hop['route'])} nodes")
    except ProjectionTooLarge as exc:
        print(f"refused: {exc}")
    except ClientError as exc:
        print(f"server rejected the projection: {exc.message}")
    finally:
        await projection.close()


if __name__ == "__main__":
    asyncio.run(main())

How It Works

Three details carry the safety, and each maps to a line above.

The estimate runs before anything is committed. gds.graph.project.estimate walks the same specification the real projection would and reports a range without allocating. It is cheap enough to run on every request and is the only way to turn “this region is too big” from a crash into a rejected request. The range matters: bytesMin assumes the most favourable layout and bytesMax the least, and a service should budget against the maximum, because the difference between them is decided by data it does not control.

The headroom multiplier accounts for what the estimate cannot see. A projection that exactly fits the budget will still fail, because the estimate covers the graph structure and not the algorithm’s own working memory, the result set, or whatever else the instance is doing at that moment. Requiring 1.5× headroom is not superstition — it is the margin between “the graph fits” and “the graph plus the search that runs over it fits”.

Streaming keeps the result off the heap. This is the half that is usually missed, because it does not look like a memory decision. gds.shortestPath.dijkstra.stream yields rows; async for consumes them as they arrive; nothing accumulates. Replace that with a collect() in the Cypher, or a list() in Python, and the whole result set materialises on the server heap before the first row reaches the driver. On a one-to-one route that is harmless. On a one-to-many cost surface over a metropolitan area it is hundreds of megabytes of transaction state, and it is charged to the heap the estimate never covered.

Two separate allocations, only one of which the estimate covers Memory committed by a single routing request. The projection itself is off-heap, is what gds.graph.project.estimate reports, and is bounded by the budget check. The algorithm's working memory and the result set are on the JVM heap, are not covered by the estimate at all, and are bounded only by how the results are consumed. Streaming yields rows as they arrive so the result never accumulates; collecting materialises the whole set on the heap before the first row leaves the server. A service that guards only the projection has guarded the smaller of the two risks. What the estimate covers, and what it does not off-heap — the projection packed topology plus named properties estimate reports this · budget check bounds it on-heap — search state and results frontier, settled set, rows in flight estimate is silent · only consumption bounds it how the result is consumed decides the second one stream + async for rows leave as they are produced; heap holds one batch flat in result size collect() in Cypher, or list() in Python whole set materialised before the first row is sent linear in result size

Common Failure Patterns

1. Checking free memory instead of a fixed budget. Two concurrent requests that each ask “is there room right now” both get told yes, both project, and the second one fails — or worse, both succeed and the third request finds a host with no heap left. The budget has to be a constant the service enforces with a lock, not a reading it takes from the environment.

# WRONG: a race with no loser until the OOM.
if free_memory() > estimate.bytes_max:
    await project(...)

# RIGHT: a fixed allowance, serialised.
async with self._lock:
    if estimate.bytes_max * 1.5 > self._budget:
        raise ProjectionTooLarge(...)
    await project(...)

2. Dropping the projection outside a finally. An exception between project and drop strands the whole graph, and because named graphs are database-scoped rather than session-scoped, nothing cleans it up when the request’s session closes. The symptom is a slow, monotonic loss of memory that does not appear in query profiling at all — the projections are simply there, owned by nobody. Check for them with gds.graph.list() and compare against what the service believes it created.

3. Projecting per request when the topology is stable. The estimate makes a per-request projection safe, not sensible. If the graph changes weekly and the service takes thousands of requests a day, the projection should be built once and reused, and the memory reserved permanently rather than churned. The measurement that decides it is in benchmarking GDS shortestPath against hand-written Cypher.

Performance Notes

Projection memory scales with the topology and with the properties you name, and the two scale differently. Structure grows with node and relationship counts; properties grow with the product of the element count and the number of properties copied. Naming one extra relationship property on a graph with fifty million relationships is not a rounding error — it is another array of fifty million values, held for the projection’s lifetime.

$$M_{\text{proj}} \approx c_n N + c_r R + \sum_{p} s_p \cdot |p|$$

The practical consequence is that the cheapest way to shrink a projection is to stop copying properties into it. A Dijkstra needs exactly one relationship weight; a projection built with properties: '*' because it was convenient carries every other one alongside, for the whole time the graph exists.

The transaction memory limit is worth setting as well as the heap maximum. db.memory.transaction.max caps what a single transaction may hold, which converts an instance-killing OutOfMemoryError into a terminated query with a clear message. That is a far better failure: one request fails, the service stays up, and the error names the query that caused it. Sizing it slightly above the widest legitimate query means the first thing to fail is always the query that went wrong, which is the same reasoning behind the memory budget for the whole instance.

A transaction limit turns an instance failure into a query failure The same oversized query under two configurations. With no transaction memory limit the query grows until the JVM heap is exhausted, the instance dies, every other tenant's in-flight work dies with it, and the container restarts with a cold cache. With a transaction limit set slightly above the widest legitimate query, that one transaction is terminated with a message naming it, every other query continues, and nothing restarts. The second failure is strictly better in every respect, including that it says what went wrong. One oversized query, two configurations no transaction limit the query grows until the heap is gone OutOfMemoryError — the instance every in-flight query on the host dies the container restarts with a cold cache the stack trace names the JVM, not the query minutes of degraded service db.memory.transaction.max set the query hits its ceiling TransactionTerminated — the query every other query keeps running nothing restarts, the cache stays warm the message names the offending query one failed request Set it above the widest query you intend to run, and the first thing to fail is always the query that went wrong.

This guide is part of Graph Memory and Storage Tuning, within Spatial Graph Database Fundamentals for Python.

Siblings