How to Use Custom Hash Algorithms (xxhash, imohash, md5, highway) in croc

To use a custom hash algorithm in croc, pass the algorithm name to the --hash flag (e.g., croc send --hash md5 file.txt), which routes the request through utils.HashFile in src/utils/utils.go to the corresponding implementation.

croc is a secure file transfer tool that verifies file integrity using checksums. By default, it uses xxhash, but you can switch between four supported algorithms—xxhash, imohash, md5, and highway—depending on your speed, security, or compatibility requirements. This guide explains how to configure these algorithms via the CLI and leverage them programmatically.

Configuring Hash Algorithms via CLI

The --hash flag controls which algorithm croc uses to fingerprint files during transfer. This flag is defined in src/cli/cli.go (lines 74–75) and defaults to xxhash if not specified.

When you initiate a transfer, croc computes the hash of the file before sending and validates it upon receipt. If you request an unsupported algorithm, the tool aborts with an "unspecified algorithm" error propagated from the dispatcher.

Command-Line Examples

Use the --hash flag with either send or receive commands:

# Default xxhash (fast, 64-bit)

croc send myfile.bin

# MD5 for compatibility (slower, 128-bit output)

croc send --hash md5 legacy-archive.zip

# Imohash for large files (sampling-based, fast)

croc send --hash imohash 10gb-movie.iso

# HighwayHash for cryptographic strength (256-bit key, 64-bit output)

croc send --hash highway sensitive-data.pdf

Available Hash Algorithms

The algorithm dispatcher utils.HashFile (lines 119–148 in src/utils/utils.go) selects the implementation based on the string supplied. Each algorithm serves different use cases regarding speed, collision resistance, and output size.

xxhash (Default)

xxhash provides the fastest non-cryptographic hashing and is the default choice. Implemented in XXHashFile (lines 41–71), it uses the cespare/xxhash/v2 package:

// From src/utils/utils.go
h := xxhash.New()

This algorithm is ideal for general-purpose file transfers where speed matters more than cryptographic security.

imohash

imohash uses a sampling-based approach to hash large files quickly without reading the entire content. The implementation resides in IMOHashFile (lines 27–31) and initializes a custom hasher configured in lines 24–26:

// Sampling parameters: 16*16*8*1024 sample size, 128*1024 threshold
imohash.NewCustom(16*16*8*1024, 128*1024)

Use this for multi-gigabyte files where a full-file hash would create unacceptable latency, accepting the trade-off of reduced collision resistance compared to full-file hashing.

md5

md5 uses the standard library crypto/md5 wrapped in MD5HashFile (around line 92 in src/utils/utils.go). While slower than xxhash and cryptographically broken for security purposes, it remains useful for compatibility with legacy systems that expect 128-bit MD5 checksums.

highway

highway employs the minio/highwayhash package for high-performance cryptographic hashing. The HighwayHashFile function (lines 53–89) uses a hard-coded 256-bit key to produce a 64-bit output. This option suits scenarios requiring strong integrity guarantees against malicious tampering.

Using Hash Functions Programmatically

You can leverage croc’s hashing utilities directly in Go applications by importing the utils package. The HashFile function accepts a filepath and algorithm name, returning a hex-encoded string.

package main

import (
    "context"
    "fmt"
    "github.com/schollz/croc/v10/src/utils"
)

func main() {
    // Compute xxhash
    hash, err := utils.HashFile("document.pdf", "xxhash")
    if err != nil {
        panic(err)
    }
    fmt.Printf("xxhash: %s\n", hash)

    // Compute imohash with context cancellation support
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    
    hash2, err := utils.HashFileCtx(ctx, "video.mkv", "imohash", false)
    if err != nil {
        panic(err)
    }
    fmt.Printf("imohash: %s\n", hash2)
}

The HashFileCtx variant supports cancellation via context, useful for UI applications where users might abort large file operations.

WebAssembly and Browser Support

For the web-based client compiled to WebAssembly, croc exposes hashing functions through web/wasm/main.go (lines 13–44). The WASM bridge currently implements xxhash via exported functions hashInit, hashUpdate, and hashFinal. This allows the browser client to verify file integrity using the same algorithm as the CLI default, though the browser implementation does not currently expose the other algorithms.

Summary

• Configure via --hash: Pass xxhash, imohash, md5, or highway to the CLI flag defined in src/cli/cli.go.
• Dispatcher logic: utils.HashFile in src/utils/utils.go (lines 119–148) routes requests to specific implementations.
• Performance trade-offs: Use imohash for large files, xxhash for general speed, highway for cryptographic strength, and md5 only for legacy compatibility.
• Programmatic access: Import github.com/schollz/croc/v10/src/utils and call HashFile or HashFileCtx from your Go code.
• Error handling: Invalid algorithm names return an "unspecified algorithm" error before transfer begins.

Frequently Asked Questions

What is the default hash algorithm in croc?

xxhash is the default algorithm. It provides excellent speed and reasonable collision resistance for file integrity checks, making it ideal for most file transfers. You can verify this default value in src/cli/cli.go where the --hash flag is initialized.

Why would I use imohash instead of xxhash?

Imohash is optimized for very large files (multiple gigabytes) because it samples portions of the file rather than hashing the entire content. According to the implementation in src/utils/utils.go, it uses a sample size of 16*16*8*1024 bytes with a threshold of 128*1024, significantly reducing I/O overhead on massive files while still producing a unique fingerprint for most practical purposes.

Is HighwayHash cryptographically secure?

Yes, HighwayHash is designed as a cryptographic hash function resistant to collision attacks, unlike xxhash or imohash. As implemented in HighwayHashFile (lines 53–89), it uses a hard-coded 256-bit key from the minio/highwayhash package. Use this algorithm when transferring files over untrusted networks where malicious tampering is a concern.

Can I add a custom hash algorithm to croc?

Yes, you can extend croc by modifying src/utils/utils.go. Add a new case to the HashFile dispatcher function (around line 119) and implement your hashing logic following the pattern of XXHashFile or MD5HashFile. You must also update the CLI flag definition in src/cli/cli.go to include your new algorithm name in the help text so users can select it with --hash.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}