How Paperclip Handles Secret Scrubbing and Collision During Company Import and Export

Paperclip's import/export pipeline scrubs all secret values before storage and detects namespace collisions to prevent accidental overwrites, using a three-stage manifest-based architecture.

The Paperclip platform treats secrets as first-class assets in its company portability system, but never allows plaintext credentials to touch logs, databases, or transfer manifests. This article examines how paperclipai/paperclip implements secret scrubbing and collision detection across its import/export stack, from client-side manifest creation through server-side validation.

The Three-Stage Import/Export Architecture

Paperclip's company portability system splits the transfer process into coordinated stages that isolate secret handling:

Stage	Purpose	Key File
Manifest Creation	Build a declaration of all objects to transfer without secret values	ui/src/lib/import-transfer.ts or cli/src/commands/client/company.ts
Secret Scrubbing	Strip credential strings and mask URLs before any persistence	packages/shared/src/company-import-transfer.ts
Collision Detection	Identify duplicate or conflicting secrets and resolve or reject	server/src/services/company-import-transfers.ts

This separation ensures that secret values exist only in encrypted storage and temporary memory during the transfer window.

Stage 1: Manifest Creation Without Secret Values

When a user initiates a company export, the client constructs a CompanyImportTransferDeclaration that enumerates every transferable object—skills, pipelines, and secrets. The critical architectural decision is that secret entries contain only metadata, never the actual value.

In ui/src/lib/import-transfer.ts, the buildImportTransferManifest function processes the export ZIP and extracts secret rows with their name, provider, and scope fields intact, but explicitly omits any value field:

// ui/src/lib/import-transfer.ts
import { buildImportTransferManifest } from '@/lib/import-transfer';
import { readFileSync } from 'fs';

const zipBuffer = readFileSync('company-export.zip');
const manifest = await buildImportTransferManifest(zipBuffer);
// manifest.parts contains secret metadata only; values never enter the manifest
await api.importTransferCreate(manifest);

The CLI implementation in cli/src/commands/client/company.ts follows identical semantics, ensuring consistent behavior across interfaces:

// cli/src/commands/client/company.ts
import { uploadCompanyImportTransfer } from '@/cli/commands/client/company';
import { readFile } from 'fs/promises';

const zipBytes = await readFile('company-export.zip');
const transferId = await uploadCompanyImportTransfer(api, zipBytes);
// SecretCollision errors surface here with full context

Both implementations rely on shared validation schemas that structurally prevent secret values from being included in the transfer declaration.

Stage 2: Server-Side Secret Scrubbing

When the server receives import chunks, the scrubber runs before any database write. This module—shared with Git credential handling—ensures defense in depth even if a malformed client sends unexpected data.

In packages/shared/src/company-import-transfer.ts, the schema definition omits the value field by design:

// packages/shared/src/company-import-transfer.ts
export const companyImportSecretSchema = z.object({
  name: z.string(),
  provider: z.enum(['aws_secrets_manager', 'gcp_secret_manager', 'env']),
  scope: z.enum(['company', 'project']),
  // Intentionally absent: no 'value' field allowed in import transfers
});

The server worker in server/src/services/company-import-transfers.ts applies additional scrubbing to catch any stray credential strings that might appear in URLs or descriptions:

// server/src/services/company-import-transfers.ts
import { scrubGitCredentialText } from './git-credentials';

async function insertSecret(row) {
  const scrubbed = scrubGitCredentialText(row);
  // Removes patterns like https://token:****@host.com/ from any text fields
  await db.insert(companySecrets).values(scrubbed);
}

The scrubGitCredentialText utility—proven in Git credential storage—masks embedded tokens using pattern matching before persistence. This protects against accidental credential leakage in skill configurations or pipeline definitions that reference secret-backed URLs.

Stage 3: Collision Detection and Resolution

After scrubbing, Paperclip checks for namespace collisions before committing secrets to the target company. The collision engine distinguishes two cases with different outcomes:

• Exact duplicate: Same name, provider, scope, and equivalent value hash—treated as idempotent, import continues
• Namespace clash: Same name but different provider or scope—import aborts with detailed conflict information

The implementation in server/src/services/company-import-transfers.ts runs this check before any INSERT operation:

// server/src/services/company-import-transfers.ts (conceptual flow)
async function checkCollision(secret) {
  const existing = await db
    .select()
    .from(companySecrets)
    .where(
      and(
        eq(companySecrets.name, secret.name),
        eq(companySecrets.provider, secret.provider)
      )
    );

  if (existing.length > 0) {
    if (existing[0].valueHash === secret.valueHash) {
      return { alreadyCompleted: true }; // Idempotent: no action needed
    }
    
    // Partial match: same name, different provider or scope
    throw new ConflictError('SecretCollision', {
      name: secret.name,
      existingProvider: existing[0].provider,
      existingScope: existing[0].scope,
    });
  }
}

Test coverage in server/src/__tests__/company-portability-import-batching.test.ts and server/src/__tests__/company-import-transfer-routes.test.ts verifies:

• Duplicate secrets across multiple import attempts resolve without error
• Provider mismatches (e.g., aws_secrets_manager vs env) trigger 409 Conflict
• Endpoint-level error formatting matches UI and CLI expectations

Client Experience for Collision Errors

When the server returns a SecretCollision error, both UI and CLI receive structured data for consistent handling:

// Error response shape
{
  error: "SecretCollision",
  name: "production_api_key",
  existingProvider: "aws_secrets_manager",
  existingScope: "company",
  suggestion: "Rename the incoming secret or delete the existing one"
}

The UI renders this as a warning badge with inline diff visualization. The CLI surfaces it as a catchable exception with exit code 1, enabling scripted remediation:

$ paperclip company import production-backup.zip
Error: SecretCollision: production_api_key already exists with provider=aws_secrets_manager
Use --force to overwrite or rename the secret in the source archive.

Security and Compliance Implications

Paperclip's secret scrubbing design satisfies several operational requirements:

• Encryption at rest: Secret values exist only in company_secret_proposals table rows encrypted with company-specific keys
• Audit logging: Import logs record that a secret was transferred, never what its value was
• Least privilege: The import worker service account has no read access to the secrets store's decryption keys

The collision detection mechanism prevents accidental privilege expansion—a scenario where importing a development company's secrets could overwrite production credentials with weaker configurations.

Summary

• Secret scrubbing occurs at multiple layers: client manifest creation omits values, shared schema validation prevents their inclusion, and server-side scrubbing masks embedded credentials
• Collision handling distinguishes exact duplicates (idempotent) from namespace clashes (conflict), with comprehensive test coverage in company-portability-import-batching.test.ts
• Architecture files: packages/shared/src/company-import-transfer.ts defines safe schemas; server/src/services/company-import-transfers.ts implements validation; UI and CLI clients enforce pre-transfer scrubbing

Frequently Asked Questions

What happens if I accidentally include a secret value in an import manifest?

The server-side schema validation in packages/shared/src/company-import-transfer.ts rejects manifests containing unexpected value fields. If a malformed client bypasses this, the scrubber in server/src/services/git-credentials.ts masks any credential patterns before database persistence. The import fails safe rather than risking plaintext storage.

Can I force overwrite an existing secret during import?

The current implementation does not expose a --force flag for secret overwrites. Namespace clashes require manual resolution: rename the incoming secret in the source export, or delete the existing secret through the Paperclip UI before re-importing. This protects against destructive automation errors.

How does Paperclip detect duplicate secrets across batched imports?

Each import part carries a deterministic hash of secret metadata. The server checks against existing rows before insertion and returns alreadyCompleted: true for exact matches. The test suite in server/src/__tests__/company-portability-import-batching.test.ts validates idempotency across chunked uploads.

Are secret scopes preserved during cross-company imports?

Yes. The scope field (either company or project) transfers intact and participates in collision detection. A secret scoped to a specific project will not collide with a company-scoped secret of the same name, though both will be created in the target company.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}