Sizing

Everything on this page is measured, and the method and hardware are stated below so you can check it or redo it. A sizing table that is wrong by 3× is worse than none, because you provision to it.

The unit that predicts cost is the file, not the repository. Two thousand small services and one Kubernetes cost roughly the same if they hold the same number of files, and repository count barely enters into it.

The short answer

Estate Files CPU RAM Disk (beyond your clones)
Small — ~100 repos 200,000 4 cores 4 GB +250 MB
Medium — ~500 repos 1,000,000 8 cores 8 GB +1.2 GB
Large — ~2,500 repos 5,000,000 8–16 cores 16 GB +5.5 GB

Files means files the analysis pass keeps — source, not binaries, not vendored blobs. The repository counts assume 2,000 analysed files per repository, which is a working average and not a measurement; if you know your own number, use the per-file rates below instead of the table.

Disk for the clones themselves is separate and dominates. See below.

Disk

Three things live under instance.dataDir:

Cost
repos/ — the clones 1.0× the on-disk size of the same repositories today
analysis/ — the analysis store 1.1 KB per analysed file
subjects.db — users, credentials, seats Megabytes. Ignore it for sizing; back it up.

The clones dominate, so the honest sizing instruction is: run du -sh over your existing checkouts and budget that, plus 10%. Measured, the product's clones came to 0.98× the source checkouts they were made from — git repacks slightly better on clone than a long-lived working copy does.

The analysis store came to 1.5% of the clones and 16.9% of the analysed source bytes on the sample below. It is the same size whether history is full or shallow, because it is computed from the working tree.

Full history is the default, and it is most of the disk

On the three repositories measured, the full clone was 3.04× the size of a depth: 1 clone. That ratio is a property of your repositories' history, and for an estate with fifteen years of binary assets in it, it can be far higher.

Check before you provision:

du -sh yourrepo/.git yourrepo --exclude=.git

If .git is several times the working tree, that repository is a candidate for the escape hatch:

repositories:
  - url: https://github.example.com/platform/monolith
    groups: [platform]
    fullHistory: false
    depth: 1

It is a per repository setting, so you can shallow the one that is costing you the disk and leave everything else alone.

What it costs, exactly: blame, commit history, hotspots and change coupling for that repository — reported as coverage gaps, never as empty results. Search, symbols, navigation, dependencies, findings and complexity are unaffected, and the commit SHA survives, so provenance is intact. See Source capability tiers.

RAM

RAM ≈ 150 MB floor + 1.4 KB per analysed file + headroom

Set a memory limit but not a CPU limit. The limit stops a pathological repository taking the node with it; throttling CPU makes a slow first index slower without protecting anything.

CPU and first-ingest time

Analysis runs in two phases and they behave differently:

Phase What Rate (8 cores) Bounded by
A — inventory Line counts, languages 7,500 files/s File bytes
B — deep Security, quality, technology 1,470 files/s File count

Phase A is close to disk speed, which is why the estate becomes searchable almost immediately. Phase B is where the cost is, and it is CPU-bound, so it scales with cores.

Worked from those rates:

Estate Searchable after (phase A) Findings complete after (phase B)
200,000 files ~30 seconds ~2 minutes
1,000,000 files ~2 minutes ~11 minutes
5,000,000 files ~11 minutes ~57 minutes

Cloning is usually the longer half of the first hour and is bounded by your network and your git host, not by this product. indexing.cloneConcurrency defaults to 4.

The instance serves throughout. It binds and answers before ingestion finishes; a repository that is not ready yet is reported as a coverage gap rather than as an empty result, and /readyz deliberately never waits for repositories.

Two caveats on the phase rates, because they are the numbers most likely to be wrong for you:

Scale out is not an option

replicaCount is 1 and there is no autoscaler in the chart. One process owns a directory of clones and an in-process analysis store, so a second replica is a second copy of the estate on a second volume, not more capacity for the same one. Scale vertically.

Reducing the cost

Lever Saves Costs
fullHistory: false on deep-history repositories Most of the clone disk (3× on the sample) Blame, history, coupling — for that repository, reported as gaps
indexing.exclude globs Clone size and analysis time Those paths are not searchable at all
indexing.metricsPerRepo: false About 2 Prometheus series and 200–400 bytes of scrape per repository — turn it off above roughly 2,000 repositories Per-repository metric series collapse onto repo="all"
analysis.deepConcurrency CPU contention with other workloads Longer time to complete findings
Fewer repositories Everything The estate is smaller

metricsPerRepo is not a fix for a slow /metrics. The numbers are read out of ingestion whether or not the series are collapsed, so turning it off changes what a scrape prints and not what it costs to produce. If a scrape is slow, that is a defect fixed in current builds, and the answer is to upgrade — the estate gauges are sampled every ten seconds off the request path, so a scrape reads a snapshot and never waits on a clone. /api/v1/status reports the age of that snapshot under observability.metrics.gauge_sampling.

A built-in exclusion set is applied before any of that and needs no configuration: node_modules, vendor, target, dist, build, .venv, Pods, .gradle, __pycache__ and the rest of the well-known vendored, build-output and cache directories. It is the floor under indexing.exclude rather than a default value for it, so setting that key adds to the set instead of replacing it, and a leading ! takes one entry back — exclude: ["!vendor"] on the one repository whose vendored tree is the interesting code, or in indexing.exclude for the whole estate. What it dropped is counted separately, as searchcode_files_skipped{reason="excluded_default"}, so an estate that came out smaller than expected can be explained from /api/v1/status rather than by diffing file counts.

The whole floor, as shipped. Each entry is a directory name, matched at any depth:

vendor, vendors, vendor-patched, Godeps, external, extern, deps, submodules, third_party, third-party, node_modules, bower_components, .yarn, .npm, .next, .nuxt, target, build, out, _build, .build, dist, obj, coverage, .coverage, __pycache__, .venv, venv, .pip-cache, .bundle, Pods, .gradle, .maven, .cargo, .idea, .vscode, .cache

generated, gen and cache are deliberately not on it, because each is sometimes a repository's own source, and excluding too much fails silently. If one of them is not your code, add it to indexing.exclude.

One more rule is part of the floor and is not a name: what pip install -t put into a repository. An AWS Lambda build installs its dependencies beside the handler, in a directory whose name the build script chose, so no name can catch it. Ingestion finds pip's own metadata instead — a *.dist-info directory holding RECORD or METADATA, or a *.egg-info holding PKG-INFO — and excludes exactly what that metadata says was installed, plus the metadata directories themselves. Everything pip did not install stays, the handler included, and a directory merely named like pip's metadata changes nothing. Each path it excludes is recorded in repo.json's excludes as pip-installed:<path> and counted under files_skipped{reason="excluded_default"}. exclude: ["!pip-installed"] turns the rule off for one repository (or, in indexing.exclude, for the estate); !<path> takes back one path. The check is a directory listing per ingest, never on a query: under a millisecond on a repository of a few hundred directories, 44 ms on the Linux kernel's 6,043.

indexing.maxRepoSizeBytes (default 5 GiB) is checked during clone rather than after, so a repository over the limit is abandoned rather than cloned and then deleted.

analysis.skipMinified defaults to true: a file whose longest line exceeds 5,000 bytes is skipped before parsing rather than being allowed to burn the whole perFileTimeout. The skip is counted and reported with reason minified, never silent.


Method and hardware

Hardware: MacBook, macOS 26.5, arm64, 8 cores, 16 GB RAM, local SSD, Go 1.26.4. Rates on a cloud VM with slower cores will be lower; the per-file and per-byte figures are the portable part.

Disk and RAM measurements (this page, 2026-08-06): three real repositories cloned from GitHub at HEAD — boyter/scc, gohugoio/hugo, grpc/grpc-go — totalling 283 MB on disk (211 MB of it .git), 3,831 analysed files and 25.3 MB of analysed source. The product was then pointed at them twice, once with full history and once with fullHistory: false, depth: 1, and du -sk run over instance.dataDir:

Full history depth: 1
repos/ 284,796 KB 93,608 KB
analysis/ 4,272 KB 4,272 KB

Resident set: 148 MB with no repositories configured, 243 MB with the three-repository estate analysed.

Phase throughput and heap come from ../verification/07b-07-phase-throughput.md and ../verification/07b-31-peak-rss.md, measured on kubernetes/kubernetes (29,843 analysed files, 266.8 MB) and on a small repository, on the same hardware. Both records state their own caveats.

What is extrapolated rather than measured: every row of the summary table. The per-file and per-byte rates are measurements; multiplying them by 200,000, 1,000,000 and 5,000,000 files is arithmetic, and the 2,000-files-per-repository assumption is a working average. No estate of that size has been measured end-to-end. Substitute your own file count — scc over your checkouts gives it in seconds — rather than trusting the repository-count column.

Generated from the documentation that ships with searchcode enterprise. Questions: ben@boyter.org