Code-host connections

A connection is a named, declared intent to enumerate repositories from one code host. It is the only thing in searchcode.yaml that can cause the searchcode process to connect to a code host, and this page is what you are agreeing to when you enable one.

What ships today. All of it. The block below is validated, its egress rules are enforced, and the Helm chart derives its NetworkPolicy from it; both providers enumerate — GitHub, for github.com and GitHub Enterprise Server, and GitLab, for a self-managed instance and for gitlab.com; and the scheduler re-enumerates each connection on its own interval, hands what it finds to the one ingestion scheduler, and retires what has gone.

You do not have to write this file at all. An instance with no repositories offers a connect screen at /, and searchcode setup does the same from a terminal — see Without editing this file. searchcode discover is still supported and answers a different question; the table at the end of this page says which you want.

What it costs, stated plainly

Before connections existed, the answer to a security questionnaire was: the searchcode process makes no outbound connection at all unless you enable tracing. A connection is an outbound connection made by the server, so that sentence does not survive it, and pretending otherwise would be worse than losing it.

The replacement:

The searchcode process makes no outbound connection unless you configure one. Three things can configure one: a code-host connection, which you create and which names exactly one host; OTLP tracing, which is off by default; and a remote syslog audit sink, which is off by default. It never connects to the vendor under any circumstances, on any tier.

That is still checkable rather than merely stated:

• The NetworkPolicy the Helm chart generates is derived from the connections you declared, so you can diff what the process may reach against what you configured. Nothing in it is hardcoded.
• A connection names one host and nothing derives a URL from a response. A redirect to another origin is refused rather than followed, and the refusal names both hosts — so a host that starts answering with a Location header cannot be handed your token.
• The vendor's own domains are refused in every mode, on every tier, with no setting that permits them.

On-premise loses nothing

A connection to a code host inside your own network is not internet egress. GitHub Enterprise Server, self-managed GitLab and Bitbucket Data Center live behind the same firewall as this product, so the socket goes to gitlab.internal.example.com:443 and never leaves.

connections.egress: internal-only makes the first row enforceable rather than merely true. Set it, and a connection naming a public code host fails at configuration load, naming the host and the setting; the Helm chart refuses to render at all, before the manifest reaches your GitOps repository. It is a switch you can point a security review at instead of a policy somebody has to read.

connections:
  egress: internal-only

What it refuses: github.com, gitlab.com, bitbucket.org, dev.azure.com, codeberg.org, git.sr.ht, gitea.com, sourceforge.net and their subdomains, plus any URL naming a globally routable IP address. What it cannot do is prove that an arbitrary hostname is inside your network — gitlab.example.com and gitlab.internal.example.com look identical to a string matcher, and resolving the name would itself be an outbound connection. The generated NetworkPolicy is what bounds the rest, which is why it is generated.

Without editing this file

There are two paths to the same record, and neither needs an editor.

The browser. An instance with no repositories renders a connect screen at / instead of a search box over an empty corpus; on an instance that already has some, it is at /connect. Fill in the host, the token and what to enumerate, and the first sync's progress appears on the same screen. It needs the administer permission, so it is a member of a group named in authz.adminGroups — and it is deliberately not on /api/v1/* and not an MCP tool, because an agent that can create a connection can point this instance at a host of somebody else's choosing.

The terminal, for a headless install:

searchcode setup --config /etc/searchcode/searchcode.yaml

It prompts for the host, the token, the targets, the repositoryGroups what it discovers will join and the clone credential, prints the scopes to grant and where to create a token, checks the token against the host before writing anything, and stores the connection. Every prompt has a flag, so the same thing is expressible in a provisioning script — searchcode setup --help lists them. --groups is the one that is required: without it the connection would be visible to nobody, and the command refuses rather than storing one. There is deliberately no --token: a secret on a command line is in the shell history and visible in ps to every user on the host, so it is --token-file, --token-env, or the prompt.

One thing to set up first: connections.keyFile

A token typed into a form has to be kept somewhere. It is not written into this file — that would put a live credential in the artefact you commit — and it is not held in memory only, because a connection that silently stops syncing on the next restart is worse than one that never started. It goes into subjects.db, encrypted.

A key stored beside the ciphertext is encoding rather than encryption, so the key comes from your configuration and an instance with no key configured cannot create a connection from the browser or from setup at all. That refusal is deliberate: it means your deployment made a decision about key custody rather than inheriting one from us.

connections:
  keyFile: /run/secrets/connections.key    # or keyEnv: SEARCHCODE_CONNECTIONS_KEY

Generate one with openssl rand -base64 32. Mount it the way you mount every other secret. Four things follow from it:

• Back up subjects.db and the key together. Restoring one without the other leaves connections that cannot be decrypted; they are listed, and each says so, rather than silently failing to authenticate against a code host.
• Changing the key is a restart, and it invalidates every stored token: each was sealed under the key in force when it was created. The screen tells you which connections are affected; there is no way back short of retyping them.
• The key never appears in searchcode doctor bundle, and neither does the path or the variable name — secrets_resolved is where "which secret is in use" is answered, in one place.
• A connection declared in this file is not editable from the browser. The file is the source of truth for what it declares; the connect screen shows such a connection, marks it, and offers no controls.

Declaring one

You can still write it here, and on an air-gapped instance whose configuration is managed in git you probably should — this is the only path where the credential reference is reviewable in a pull request.

connections:
  egress: any                    # any | internal-only. Restart to change.
  sources:
    - name: acme-github          # identity; see below
      type: github               # github | gitlab
      url: https://github.com    # ONE host. Never derived from a response.
      token:
        env: ACME_GITHUB_TOKEN   # a reference, never a value
      include:                   # required
        orgs: [acme, acme-labs]
        users: [ben]
        self: false
      exclude: ["*-archive"]
      skipArchived: true
      skipForks: true
      groups: [engineering]
      syncInterval: 1h

Four properties of that block are load-bearing.

token is a secret reference, never a literal. There is no field to paste a token into. Write {env: NAME} or {file: /path/to/secret}; a relative file path resolves against the directory holding searchcode.yaml, and the file is re-read on every configuration load, so a rotated Kubernetes Secret needs no image change. An inline value is refused at load, naming the field — and the refusal deliberately does not repeat the value, because an error message carrying a token puts it in the startup log, the journal and any support bundle. If you wrote one, rotate it: it has been in a file.

name is identity. Per-connection state hangs off it, so renaming a connection is a delete plus a create rather than a rename. That is stated rather than silently migrated.

include is required. There is no "everything this token can see" mode. The blast radius of a mis-scoped personal access token is the whole reason: a connection that enumerated whatever the credential could reach would turn one over-broad token into an estate nobody chose. self: true — the token's own account — is the one target that needs no name, and it is a complete configuration on its own.

The block reloads; a token change does not. Everything you edit day to day — which organisations, what to exclude, how often — takes effect without a restart, and so does adding or removing a whole connection. What is rejected on reload, naming the connection, is changing which secret an existing connection points at; connections.egress is rejected the same way. Both need a restart. Rotating the value behind an unchanged reference is likewise not picked up until a restart, and that is worth knowing: the configuration watcher compares the configuration files, and a mounted secret is not one of them.

The token, and what it is not

A connection contributes repositories, never permissions. The token is read-only and metadata-only. Nothing in this product reads, writes or mirrors a code host's access-control lists: authorisation stays declarative and local, in authz.bindings, evaluated against the groups your identity provider asserts.

That is a deliberate refusal rather than a missing feature. Mirroring host ACLs would make your code host the source of truth for who can see what, which turns a GitHub outage into an access-control outage — and it needs per-user OAuth tokens this product never asks for.

Enumeration and cloning are separate concerns and use separate credentials. A connection's token is for the API; git remotes authenticate through credentials: (see the configuration reference). Name the one a discovered repository is cloned with on the connection itself:

    - name: acme-github
      token: {env: ACME_GITHUB_TOKEN}   # LISTS repositories
      credential: internal-https        # CLONES them; names a `credentials:` entry

credential: is a name and never a secret, exactly as repositories[].credential is, and an undeclared name fails at config load. Omit it for a public organisation, where an anonymous clone is correct. A connection that omits it and then discovers private repositories reports one gap on /api/v1/status, on the admin page and on /connect, naming itself and what to add — rather than leaving one git authentication failure per repository, which reads as a token-scope problem and sends you back to the code host. Widening the connection's own token does not help: a GitHub metadata-only scope enumerates and cannot clone. (GitLab differs, and the difference is measured rather than assumed — see the GitLab section below.) Where the two are the same value, that is your choice rather than this product's assumption.

Prefer a group or project access token on GitLab, and a token owned by a service account rather than a person on GitHub. The failure mode otherwise is indexing silently stopping three months after onboarding, when somebody's account is deprovisioned.

GitLab, and what is different about it

A self-managed instance is the primary case, not a variant. It lives behind the same firewall as this product, so the socket never leaves your network and connections.egress: internal-only permits it while refusing gitlab.com. Three differences from GitHub are worth knowing before you write the block, and none of them is cosmetic.

There is no path-prefix rule. /api/v4 is the API path on gitlab.com and on every self-managed install alike, so write the host and nothing else: url: https://gitlab.internal.example.com. (GitHub needs a rule because github.com's API is served from api.github.com and GHES serves it under /api/v3; GitLab has no equivalent, and there is nothing here to get wrong.)

Groups nest, and a group is a path. acme/platform/infra/tooling is ONE entry under include.orgs, and naming a group enumerates the whole tree beneath it. That is the useful behaviour — a platform team's estate is almost always nested, and a tool that read only the top level would silently miss most of it — but it cuts the other way on a large instance: a top-level group can be far more than you expected. Name the subgroup you mean, or bound it with exclude globs, and check the count on /api/v1/status after the first sync.

A user is a personal namespace, with its own endpoint. Projects live under a group or a user, and include.users: [ben] is how you reach the second; self: true is the account that issued the token and needs no name.

One token may be enough, which is the opposite of GitHub and is the one thing here that carries a version bound. On the GitLab CE release this was measured against — 2026-09-16, over HTTP, on a private project, as a non-administrator (a Reporter on the group, to rule out an admin bypass) — a read_api token enumerated and cloned, while a read_repository token cloned and was refused 403 on the enumeration. So on GitLab the token that fails fails at the LISTING end, which is the opposite end of the pipeline from GitHub's. It is one CE release: not gitlab.com, not an older self-managed install, and GitLab has changed which scopes git-over-HTTP accepts before — so if clones are refused, put a read_repository token in that entry, which is always sufficient and never wrong.

"One token" means one SCOPE, not one reference — a private estate still needs the credentials: entry. These are two different sentences and running them together is the reason this paragraph exists. What was measured is a read_api token sitting inside a credentials entry, presented as the clone password; a connection with no credential: presents nothing, so the clone goes out anonymous and every private project is refused however the listing token is scoped. The estate looks the same either way — private repositories missing — and the two have different fixes, so searchcode doctor names which one you have.

connections:
  egress: internal-only          # your instance is inside your network
  sources:
    - name: acme-gitlab
      type: gitlab
      url: https://gitlab.internal.example.com   # no /api/v4 needed
      token:
        env: ACME_GITLAB_TOKEN
      credential: acme-gitlab-clone   # CLONES them; omit only for a public estate
      include:
        orgs: [acme/platform]    # one path; its subgroups come with it
        users: [ben]
      exclude: ["acme/platform/sandbox/*"]
      groups: [engineering]

Prefer a group or project access token. A personal access token dies with the person's account, and the failure mode is the quiet one: indexing stops three months after onboarding, when somebody is deprovisioned, and the estate just gradually goes stale. A group access token on the group you are indexing survives that, and it is scoped to exactly what the connection names.

Connections or searchcode discover?

Both are supported and they answer different questions.

An estate whose repository membership is reviewed in a pull request should keep using the CLI. An estate where somebody has to remember to re-run it is the case connections exist for.

Who can see what a connection finds

A repository's audience is declared, never inferred. That is the property the whole authorisation model rests on, and it is worth stating the consequence before the mechanism: nothing about a repository's name, its owner, its path or its topics affects who can read it.

There are two ways to say what a connection's repositories are visible to, and you choose per connection.

Model B — groups on the connection (the default)

Every repository a connection discovers joins the groups: declared on that connection, and no others.

repositoryGroups:
  - name: engineering

connections:
  sources:
    - name: acme
      type: github
      url: https://github.com
      token: {env: SEARCHCODE_GITHUB_TOKEN}
      include:
        orgs: [acme]
      groups: [engineering]        # every repository this finds joins this

The audience is still declared — you declared it once, for the set, rather than once per repository. A rename at the code host cannot move a repository between groups, because names were never consulted; a repository leaving the organisation leaves the group by leaving the estate, which is audited.

Two audiences means two connections. The common shape is "the acme org is for everyone, acme-secrets is for the platform team", and it needs no pattern anywhere:

connections:
  sources:
    - name: acme
      include: {orgs: [acme]}
      groups: [engineering]
      # … url, token
    - name: acme-secrets
      include: {orgs: [acme-secrets]}
      groups: [platform]
      # … url, token

A connection is the unit of audience as well as the unit of enumeration, which is why include is explicit and why name is identity.

One repository out of its connection's groups is an explicit entry, by name, and it wins:

repositories:
  - url: https://github.com/acme/payments
    groups: [payments]           # beats the acme connection's [engineering]

Explicit beats generated — the same rule conf.d/ already has. The whole entry wins, not only its groups, so a credential, a depth or an exclude list you wrote there is what runs.

What is refused, and why

There are no per-repository pattern rules. No groupRules, no groupPatterns, no mapping from a topic or a name to a group — and a configuration attempting one fails at load, naming the key.

It is the obvious next request. It is also what makes the model collapse: a rule mapping *-infra onto infra means that renaming a repository at the code host changes who can read it, with no change in this product and nothing in a diff for anybody to review. A glob written where a group name belongs (groups: ["team-*"]) is refused for the same reason. Patterns are allowed on the binding side — authz.bindings[].repoGroups — where every expansion is logged at config load and is therefore reviewable.

A connection with no groups

A connection that declares no groups: indexes repositories visible to nobody. That is not a configuration error — you may legitimately stage a connection before the bindings that grant it exist — but it is never silent, because a connection that is green, has the right repository count and returns nothing from every search sends people to look at their token. Four surfaces say it:

• the connect screen and the admin page, as "N repositories, visible to nobody";
• searchcode setup refuses to create one at all. A group is a prompt and --groups is required, and the refusal names the groups this instance declares — or prints the repositoryGroups: and authz.bindings block to add when it declares none. A group that is declared and that no binding grants yet is created, loudly, with the binding to add: that is the staging case above;
• the startup log, once per start, for every declared or stored connection whose repositories nobody can see;
• searchcode doctor, as a warn naming the connection — over connections created by setup as well as declared ones.

Before a connection's first sync applies, the connect screen shows what it would add: how many repositories, the groups they will join, and which identity groups that grants access to, through which binding. That is the last point at which a mis-scoped token or a mistyped group name is cheap to notice.

Model A — a generated file, reviewed

If the estate should change when a pull request merges rather than when a code host does something, keep using searchcode discover. It writes a conf.d/ fragment listing every repository with its groups explicitly, unclassified ones with an empty list, and applies nothing until that file is merged. config validate fails while any repository is unclassified, so the review happens before deployment rather than becoming a runtime state somebody has to notice.

What a connection tells you when it cannot see everything

A connection reports two different things and it is worth knowing which you are looking at. A failure means the connection is broken and nothing was read: a revoked token, an unreachable host. A gap means it is not finished, or saw only part of what you asked for — and an answer with a gap is never presented as a complete one.

searchcode discover prints the same gaps, and refuses to write a fragment when there is one: a file generated from a partial answer deletes every repository the run could not see, and the pull request would carry it to a merge.

Where to look when something is wrong

• /api/v1/status carries the connection block as it is currently in force — names, types, hosts, targets — and the fingerprint of every resolved secret. It never carries a token or a token reference.
• On github.com the host a connection dials is api.github.com, not github.com. Write the URL you think of; the API root is resolved from it at startup, by a fixed rule over what you wrote and never from anything a code host said. GitHub Enterprise Server serves its API under /api/v3 on your own host, which is resolved the same way; GitLab uses /api/v4 on gitlab.com and on every self-managed install alike, so there is no equivalent rule there.
• searchcode config validate -f /etc/searchcode/searchcode.yaml reports every problem in the block, not the first, and exits non-zero.
• searchcode doctor bundle carries the connection block with the token path dropped, which is what a support engineer needs to diagnose a connection that enumerates nothing. See searchcode doctor.