# How to Deploy Mpcium in Production: A Security-Hardened Systemd Setup Guide

> Deploy Mpcium securely in production with systemd unit templates leveraging TLS encrypted communication automated backups and non privileged services. Enhance your deployment today.

- Repository: [Fystack Labs/mpcium](https://github.com/fystack/mpcium)
- Tags: how-to-guide
- Published: 2026-03-02

---

**Deploy Mpcium in production by running a non-privileged systemd service with encrypted SystemD credentials, TLS-only communication, and automated BadgerDB backups using the hardened unit templates and validation scripts provided in `deployments/systemd/`.**

Production deployment of fystack/mpcium requires a hardened, repeatable configuration that protects cryptographic material, ensures reliable inter-node communication, and provides observability. The repository ships a complete systemd-based workflow that implements defense-in-depth safeguards through strict file permissions, encrypted credential injection, and resource isolation. This guide walks through the essential architecture, configuration requirements, and step-by-step deployment process based on the official source code.

## Hardened Systemd Service Architecture

The production deployment centers on a locked-down systemd unit defined in `deployments/systemd/mpcium.service`. This template enforces the principle of least privilege while securely injecting secrets via systemd's native credential mechanism.

### Non-Privileged Execution and File Permissions

Run the node as a dedicated **non-privileged system user** `mpcium` to limit the impact of a compromised process. The [`setup-config.sh`](https://github.com/fystack/mpcium/blob/main/setup-config.sh) script automates this via the `create_user` function, creating the user and group before installation.

Configuration files require strict ownership:
- **[`/etc/mpcium/config.yaml`](https://github.com/fystack/mpcium/blob/main//etc/mpcium/config.yaml)** must be owned by `root:mpcium` with permissions `640` (lines 94-104 of [`setup-config.sh`](https://github.com/fystack/mpcium/blob/main/setup-config.sh))
- The service definition sets `ReadOnlyPaths=/etc/mpcium` to prevent runtime modification
- Data directories under `/opt/mpcium` use `ReadWritePaths` restrictions

### Encrypted Credential Injection

Never store plaintext passwords in the YAML configuration or environment variables. Instead, use **SystemD Credential** files (`*.cred`) injected via `SetCredentialEncrypted=`:

- The BadgerDB password is supplied through a credential file referenced by `loadPasswordFromFile` at startup
- Identity encryption passphrases are stored as `mpcium-identity-password.cred`
- The `update_service_credentials` function in [`setup-config.sh`](https://github.com/fystack/mpcium/blob/main/setup-config.sh) manages these files

This approach ensures secrets never appear in `/proc/<pid>/environ` or process listings.

### System Call Filtering and Capability Bounding

The `mpcium.service` template (lines 26-35) implements aggressive sandboxing:

```ini
[Service]
User=mpcium
Group=mpcium
NoNewPrivileges=yes
ProtectSystem=full
CapabilityBoundingSet=CAP_DAC_READ_SEARCH
LimitNOFILE=65535
LimitNPROC=4096

```

These directives prevent privilege escalation, restrict filesystem access to read-only for system paths, and bound capabilities to the minimum required for database operations.

## Secure Configuration Management

Production settings are validated against `config.prod.yaml.template` using logic in [`setup-config.sh`](https://github.com/fystack/mpcium/blob/main/setup-config.sh).

### Production Environment Enforcement

Set **`environment: production`** in [`config.yaml`](https://github.com/fystack/mpcium/blob/main/config.yaml) to guarantee TLS-only defaults enforced in [`pkg/constant/env.go`](https://github.com/fystack/mpcium/blob/main/pkg/constant/env.go). The `check_environment` function in the setup script validates this setting and warns if NATS URLs do not use `tls://` or if Consul addresses lack HTTPS.

### Storage and Backup Configuration

Enable **AES-256 encrypted BadgerDB** storage by supplying the password via systemd credentials rather than the config file. The database initialization in [`pkg/kvstore/badger.go`](https://github.com/fystack/mpcium/blob/main/pkg/kvstore/badger.go) derives the encryption key from this credential.

For disaster recovery, configure:
- `backup_enabled: true`
- `backup_dir: /opt/mpcium/backups`
- `backup_period_seconds: 300`

The `StartPeriodicBackup` function in [`cmd/mpcium/main.go`](https://github.com/fystack/mpcium/blob/main/cmd/mpcium/main.go) (lines 54-62) invokes the logic in [`pkg/kvstore/badger_backup.go`](https://github.com/fystack/mpcium/blob/main/pkg/kvstore/badger_backup.go), creating encrypted snapshots using the same password as the live database.

## Cryptographic Material Management

Mpcium relies on several key types that must be generated and distributed according to strict protocols.

### Peer Registry Generation

Generate the shared peer registry on a single designated node using:

```bash
mpcium-cli generate-peers -n 3

```

This command (implemented in the CLI source) creates [`peers.json`](https://github.com/fystack/mpcium/blob/main/peers.json) under `/opt/mpcium`. Distribute this file to all nodes with permissions `640` and ownership `root:mpcium`. All nodes must share an identical `chain_code` (64-hex-character string) validated by `validate_config_credentials`.

### Identity Key Lifecycle

**Initiator Identity**: Create the event initiator using `mpcium-cli generate-initiator --encrypt` (source: [`cmd/mpcium-cli/generate-initiator.go`](https://github.com/fystack/mpcium/blob/main/cmd/mpcium-cli/generate-initiator.go)). This produces an Age-encrypted private key (`initiator.key.age`) and a public key. Copy the public key to the `event_initiator_pubkey` field in [`/etc/mpcium/config.yaml`](https://github.com/fystack/mpcium/blob/main//etc/mpcium/config.yaml).

**Node Identity**: Each node generates its own unique identity using:

```bash
mpcium-cli generate-identity --node $(hostname) --encrypt

```

The resulting private key never leaves the host. After loading keys, the node invokes [`pkg/security/zeroize.go`](https://github.com/fystack/mpcium/blob/main/pkg/security/zeroize.go) to wipe sensitive buffers from memory.

## Observability and Health Monitoring

The deployment provides multiple observability vectors:

- **Journal Integration**: All logs route to systemd journal via `StandardOutput=journal`. Monitor with `journalctl -u mpcium -f`
- **Health Endpoints**: Enable `healthcheck.enabled: true` to activate the HTTP server defined in [`pkg/healthcheck/server.go`](https://github.com/fystack/mpcium/blob/main/pkg/healthcheck/server.go)
- **Consul Integration**: The node automatically registers health checks via `infra.GetConsulClient` when Consul is configured

## Step-by-Step Production Deployment Workflow

Execute the following sequence on each node:

1. **Install Binaries**
   ```bash
   sudo make install
   ```

2. **Initialize System Configuration**
   ```bash
   sudo /opt/mpcium/deployments/systemd/setup-config.sh
   ```

   This interactive wizard creates the `mpcium` user, validates TLS settings via `check_environment`, and installs the systemd unit.

3. **Generate Peer Registry** (on one node only)
   ```bash
   mpcium-cli generate-peers -n 3
   # Distribute peers.json to all nodes at /opt/mpcium/peers.json

   ```

4. **Create Initiator Identity** (on one node)
   ```bash
   mpcium-cli generate-initiator --encrypt
   # Copy the public key to event_initiator_pubkey in config.yaml

   ```

5. **Create Node Identity**
   ```bash
   mpcium-cli generate-identity --node $(hostname) --encrypt
   ```

6. **Validate Deployment**
   ```bash
   sudo /opt/mpcium/deployments/systemd/setup-config.sh verify
   ```

   The `verify_deployment` function confirms directory permissions, key presence, and credential files.

7. **Start Service**
   ```bash
   sudo systemctl start mpcium
   sudo systemctl status mpcium
   ```

## Summary

- **Use the hardened systemd template** from `deployments/systemd/mpcium.service` with `NoNewPrivileges` and `ProtectSystem` to isolate the process
- **Inject secrets via systemd credentials** managed by [`setup-mpcium-cred.sh`](https://github.com/fystack/mpcium/blob/main/setup-mpcium-cred.sh) and consumed by `loadPasswordFromFile`, never via environment variables
- **Enforce TLS** for all external communication (NATS, Consul) by setting `environment: production` and validating with [`setup-config.sh`](https://github.com/fystack/mpcium/blob/main/setup-config.sh)
- **Automate encrypted backups** using [`pkg/kvstore/badger_backup.go`](https://github.com/fystack/mpcium/blob/main/pkg/kvstore/badger_backup.go) with the same credential protecting the live BadgerDB
- **Verify before starting** using `setup-config.sh verify` to ensure cryptographic material and permissions are correctly configured

## Frequently Asked Questions

### How should I store the BadgerDB password in production?

Store the password as a systemd credential file using [`setup-mpcium-cred.sh`](https://github.com/fystack/mpcium/blob/main/setup-mpcium-cred.sh), which creates an encrypted blob loaded via `SetCredentialEncrypted=` in the service unit. The application reads this via `loadPasswordFromFile` at startup, ensuring the secret never appears in environment variables or command-line arguments accessible via `ps` or `/proc`.

### What systemd security features protect the Mpcium process?

The `mpcium.service` unit implements `NoNewPrivileges=yes` to prevent privilege escalation, `ProtectSystem=full` to make most of the filesystem read-only, and `CapabilityBoundingSet=CAP_DAC_READ_SEARCH` to restrict available kernel capabilities. These settings, combined with the dedicated `mpcium` user, create a minimal attack surface as defined in the template (lines 26-35).

### How are database backups encrypted and managed?

Automated backups are handled by `StartPeriodicBackup` in [`cmd/mpcium/main.go`](https://github.com/fystack/mpcium/blob/main/cmd/mpcium/main.go), which uses the same AES-256 encryption key as the live BadgerDB instance (defined in [`pkg/kvstore/badger_backup.go`](https://github.com/fystack/mpcium/blob/main/pkg/kvstore/badger_backup.go)). Backups are written to `backup_dir` with the `.enc` extension. You should periodically copy these files to an off-site immutable store such as S3 with bucket-level encryption.

### Can I rotate node identity keys without cluster downtime?

Yes. Generate a new identity using `mpcium-cli generate-identity --node <name> --encrypt`, update the corresponding systemd credential using [`setup-mpcium-cred.sh`](https://github.com/fystack/mpcium/blob/main/setup-mpcium-cred.sh), and execute `systemctl restart mpcium`. The node rejoins the cluster using the existing [`peers.json`](https://github.com/fystack/mpcium/blob/main/peers.json) and `chain_code`, provided the network addresses remain consistent.