Dolt Remote Repositories: Adding, Pushing, and Pulling Data

Dolt remote repositories act as separate Dolt instances reachable over HTTP, Git, AWS, GCS, or local file transports, with synchronization handled through layered CLI and SQL interfaces that normalize URLs and manage data transfer via gRPC or Git-style protocols.

Dolt extends Git semantics to database tables, enabling version-controlled collaboration on structured data. Configuring Dolt remote repositories allows teams to share datasets through DoltHub, private infrastructure, or cloud object storage. This guide breaks down the implementation in the dolthub/dolt codebase, from the cmd/dolt/commands/remote.go CLI handler to the remotesrv gRPC daemon.

Configuring Dolt Remote Repositories

Dolt stores remote configuration in the dolt_remotes system table and supports multiple transport schemes. The configuration layer abstracts URL handling and credential storage across both command-line and SQL interfaces.

Adding Remotes via CLI and SQL

You can register a remote using the CLI or the dolt_remote stored procedure. Both paths ultimately invoke env.NewRemote and persist metadata to the dolt_remotes system table defined in libraries/doltcore/doltdb/system_table.go (lines 425–428).

Using the CLI:

dolt remote add origin https://github.com/example/project.git

Internally, cmd/dolt/commands/remote.go (lines 61–84) parses the command and calls env.GetAbsRemoteUrl to resolve the raw URL before inserting it via dEnv.AddRemote.

Using SQL:

CALL dolt_remote('add', 'origin', 'https://github.com/example/project.git');

The procedure in libraries/doltcore/sqle/dprocedures/dolt_remote.go (lines 88–92) validates arguments, constructs a Remote object, and persists it through dbd.Rsw.AddRemote.

URL Normalization and Supported Transports

The environment layer normalizes URLs to handle diverse backend schemes. In env/remote.go, the GetAbsRemoteUrl function (referenced in remote.go lines 61–66) processes schemes including http, https, aws, gs (Google Cloud Storage), file, and Git-style URLs.

For Git remotes, the system normalizes URLs to git+… formats (remote.go lines 71–75). Optional parameters such as --aws-region, --ref, or credentials are captured as a map[string]string and stored with the remote configuration. Note that AWS/GCP parameters are only applied in local CLI contexts; the SQL engine rejects them for security (remote.go line 83).

Pushing Data to Remote Repositories

Pushing transfers local commits to a remote Dolt instance. The operation supports force pushes, upstream tracking, and gRPC-based authentication.

The dolt_push Implementation

The CLI command dolt push constructs a SQL query that invokes the dolt_push stored procedure. In cmd/dolt/commands/push.go (lines 48–63), the CLI builds a CALL dolt_push(...) statement with flags like --set-upstream.

The procedure implementation in libraries/doltcore/sqle/dprocedures/dolt_push.go (lines 48–63) parses these flags into a PushOptions struct. It acquires the remote database handle via sess.Provider().GetRemoteDB (line 93), which establishes a client connection over gRPC or Git protocols. After rebasing the remote to fetch its latest state, the operation executes actions.DoPush (lines 18–19).

Error handling distinguishes between ErrUpToDate and datas.ErrMergeNeeded, translating them into user-friendly messages (lines 21–27).

Authentication and Force Push Options

Authentication flows differ by transport. For gRPC-based remotes, pass the --user flag to inject credentials:

CALL dolt_push('origin', '--user', 'alice', '--force', 'main');

The procedure extracts the username in dolt_push.go (lines 86–90) and augments the Remote with GRPCUsernameAuthParam. The --force flag bypasses non-fast-forward checks, while --set-upstream establishes tracking references for future pushes.

Pulling Data from Remote Repositories

Pulling fetches remote refs and merges them into the working set. Dolt handles fast-forward detection, conflict identification, and merge strategy selection through the dolt_pull procedure.

Fetching and Merging with dolt_pull

The CLI wrapper in cmd/dolt/commands/pull.go (lines 46–48) translates dolt pull origin into a stored procedure call. The implementation in libraries/doltcore/sqle/dprocedures/dolt_pull.go (lines 60–78) creates a PullSpec via env.NewPullSpec, resolves the target remote, and invokes sess.Provider().GetRemoteDB (line 38) to connect to the remote server.

After rebasing the remote database, actions.FetchRefSpecs retrieves the requested references. The procedure supports several merge strategies:

CALL dolt_pull('origin', 'feature', '--squash', '--no-ff');

Flags like --squash, --no-ff, and --force are parsed during spec construction (lines 22–27) and determine how fetched commits integrate with the local branch.

Handling Conflicts and Fast-Forwards

The dolt_pull procedure returns a result row with three columns defined in the schema at lines 42–58: fast_forward, conflicts, and message. These indicate whether the update was a fast-forward, whether merge conflicts exist in the working set, and a human-readable status description. The underlying fetch logic detects divergence and applies rebase operations before attempting the merge.

The Remote Server Architecture (remotesrv)

Dolt includes a standalone remotesrv daemon that exposes repositories over gRPC and HTTP, enabling custom remote hosting without DoltHub.

Running the gRPC/HTTP Daemon

The daemon entry point resides in utils/remotesrv/main.go (lines 37–70). You can start a server to expose a local Dolt directory:

remotesrv -grpc-port 50051 -http-port 8080 -dir /path/to/repo

The remotesrv.NewServer function (lines 93–100) initializes the gRPC service and HTTP handlers. The server supports two modes: repo-mode (serving an existing Dolt directory) and in-memory mode (fresh ephemeral storage).

Repository Modes and Concurrency Control

The server implements PushConcurrencyControl_PUSH_CONCURRENCY_CONTROL_IGNORE_WORKING_SET (line 100) to handle concurrent push operations safely. For read-only mirrors, launch with the -read-only flag to prevent write operations.

Clients interact with the server through the RemoteSrvStore interface, which abstracts chunk storage access via the DCache layer. For Git-based transports, gitauth.DisableInteractivePrompts() (line 38 of main.go) ensures non-blocking operation in automated environments.

System Tables for Remote Visibility

Dolt provides read-only system tables for inspecting remote configuration and state:

• dolt_remotes – Lists configured remotes with columns name, url, and params (defined in system_table.go lines 425–428). Query this table to verify remotes: SELECT name, url FROM dolt_remotes;
• dolt_remote_branches – Displays remote-tracking branches showing the name and current hash of fetched refs.

Both tables populate from the on-disk remotes.json configuration and refresh after push or pull operations. Because these tables are read-only, all modifications must occur through the dolt_remote stored procedure or CLI commands.

Summary

• Dolt remote repositories support HTTP/S, Git, AWS, GCS, and local file transports, with URLs normalized through env.GetAbsRemoteUrl in the environment layer.
• Remote configuration persists to the dolt_remotes system table via env.NewRemote, accessible through both CLI (cmd/dolt/commands/remote.go) and SQL (dolt_remote.go).
• Pushing uses actions.DoPush after establishing a connection via sess.Provider().GetRemoteDB, supporting force flags and gRPC authentication.
• Pulling employs actions.FetchRefSpecs through the dolt_pull procedure, returning status rows indicating fast-forward status and conflicts.
• The remotesrv daemon (utils/remotesrv/main.go) provides a standalone gRPC/HTTP server for hosting Dolt remotes with configurable concurrency and read-only modes.

Frequently Asked Questions

How do I authenticate with a private Dolt remote over gRPC?

Pass the --user flag to dolt_push or dolt_pull. In SQL, use CALL dolt_push('origin', '--user', 'username', 'main'). The procedure extracts this in dolt_push.go (lines 86–90) and injects it as GRPCUsernameAuthParam into the remote connection. For Git remotes, ensure credentials are available in the environment or use HTTPS URLs with embedded tokens.

What is the difference between dolt pull and dolt fetch?

dolt pull combines fetching and merging into a single operation. According to the source in dolt_pull.go, the procedure calls actions.FetchRefSpecs to retrieve remote refs, then evaluates fast-forward status and merges the result into the working set. Fetch alone only updates remote-tracking branches in dolt_remote_branches without modifying the local working branch.

Can I run my own Dolt remote server without using DoltHub?

Yes. The remotesrv utility in utils/remotesrv/main.go implements the same gRPC/HTTP protocol used by DoltHub. Start the daemon with remotesrv -dir /repo/path -grpc-port 50051 to expose a local repository. The server handles chunk storage through the RemoteSrvStore interface and supports read-only mode via the -read-only flag for mirrors.

Why does my SQL dolt_push fail with a merge error?

The push procedure translates datas.ErrMergeNeeded into a user-facing error when the remote contains commits not present in your local history (lines 21–27 in dolt_push.go). This indicates a non-fast-forward condition. Either pull the remote changes first to merge them locally, or use the --force flag to overwrite remote history, though the latter risks data loss on the remote.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}