# How llmfit Community Benchmark Submission Works Without CLI or GitHub Accounts

> Discover how to submit llmfit community benchmarks without CLI or GitHub accounts. Learn about local JSON storage, OAuth device flow, and direct API calls for anonymous contributions.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: how-to-guide
- Published: 2026-08-20

---

**llmfit enables anonymous benchmark contributions through local JSON storage, OAuth device flow authentication, and direct GitHub API calls—no `gh` CLI or pre-existing account required.**

The `llmfit` open-source project simplifies community-driven LLM benchmarking by removing traditional barriers to participation. Instead of requiring users to install the GitHub CLI or maintain active accounts, the tool implements a self-contained submission flow using standard HTTP requests and OAuth device authentication. This design makes community benchmark submission accessible from CI pipelines, containerized environments, and headless machines.

## Local Benchmark Storage First

Every benchmark run begins with local persistence. The `store_local` function in [`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) writes results as JSON payloads to the user's data directory.

```rust
// Record benchmark results locally (automatic hardware detection)
let results = vec![bench_result];
let specs   = SystemSpecs::detect()?;               // probes CPU, GPU, memory
let path = llmfit_core::share::store_local(&results, &specs)?;

```

Results land in `$XDG_DATA_HOME/llmfit/benchmarks/pending/*.json` (typically `~/.local/share/llmfit/benchmarks/pending/` on Linux). This decouples **benchmark execution** from **submission timing**—users can run benchmarks offline and share results later.

Key implementation details from [`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) lines 95-111:

- Each payload includes detected hardware specs, timestamps, and benchmark metrics
- Filenames use stable hashes to prevent duplicate storage
- The function returns a `PathBuf` for verification or manual inspection

## Authentication Without Existing Credentials

The `resolve_token_noninteractive` function implements a three-tier authentication strategy that requires zero pre-configuration:

1. **Environment variables** — Checks `GITHUB_TOKEN` or `GH_TOKEN` first
2. **Cached tokens** — Reuses tokens from previous device-flow logins via `write_cached_token`
3. **OAuth device flow** — Initiates fresh authentication when no token exists

When device flow begins, `llmfit` prints instructions to **stderr** (preserving stdout for piping):

```text
To share benchmarks, please authenticate with GitHub:

1. Visit: https://github.com/login/device
2. Enter code: XXXX-XXXX

Waiting for authorization...

```

The flow uses GitHub's public OAuth device flow with `DEFAULT_CLIENT_ID` ([`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) lines 39-65). After user authorization, `device_flow_poll` exchanges the device code for an access token. The token caches to disk for subsequent non-interactive runs.

This design means **first-time contributors need only a browser**—no GitHub account setup, no SSH keys, no CLI installation.

## Payload Preparation and Idempotency

Before upload, `llmfit` reprocesses stored benchmarks to ensure clean, deduplicated submissions ([`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) lines 58-76 and 160-178):

- **Re-timestamping** — Updates timestamps to submission time
- **Hardware slug generation** — `payload_slug` creates normalized identifiers like `nvidia-rtx-4090-linux-x86_64`
- **Stable filenames** — `short_hash` derives consistent names from content hash

These steps guarantee that re-running `llmfit bench --share` won't create duplicate files if previous submissions failed mid-way.

## GitHub API Integration Without External Tools

All repository operations use `ureq` (a lightweight Rust HTTP client) rather than shelling out to `gh`. The flow in [`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) lines 84-131 and 160-176:

| Step | Function | API Endpoint |
|------|----------|--------------|
| Fork creation | `ensure_fork` | `POST /repos/{owner}/{repo}/forks` |
| Branch creation | `create_branch` | `POST /repos/{user}/{repo}/git/refs` |
| File upload | `put_file` | `PUT /repos/{user}/{repo}/contents/{path}` |
| PR discovery | `find_open_bench_pr` | `GET /repos/{owner}/{repo}/pulls` |
| PR creation | `open_pr` | `POST /repos/{owner}/{repo}/pulls` |

Branch names follow the pattern `bench/{hardware-slug}-{hash}`, making PRs discoverable and organized. Existing files are skipped during upload, and new benchmarks append to open PRs rather than spawning duplicates.

The generated PR includes a markdown table summarizing all contributed results for maintainer review.

## Hardware Validation Gate

To maintain leaderboard quality, `llmfit` rejects submissions from unidentified hardware. The `hardware_is_identifiable` check ([`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs) lines 85-92) blocks uploads when:

- GPU detection returns "N/A" or empty strings
- Critical system specs are missing

This prevents polluted benchmarks from misconfigured containers or VMs without proper GPU passthrough.

## Complete Sharing Workflow

```rust
// Share all pending benchmarks with automatic authentication
let opts = llmfit_core::share::ShareOptions { 
    dry_run: false,       // set true to preview without uploading
    assume_yes: true,     // skip interactive prompts
};

match llmfit_core::share::share_all_pending(&opts, None) {
    Ok(Some(outcome)) => println!("PR opened: {}", outcome.pr_url),
    Ok(None) => println!("No pending benchmarks or dry-run mode."),
    Err(e) => eprintln!("Share failed: {e}"),
}

```

Running this from a CI job triggers device flow on first execution, caches the token, and proceeds automatically thereafter. The binary handles the entire lifecycle—no container modifications or secret mounting required beyond standard environment variable support.

## Key Source Files

- **[`llmfit-core/src/share.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/share.rs)** — Core implementation: local storage, authentication, payload preparation, GitHub submission
- **[`llmfit-tui/src/tui_app.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_app.rs)** (lines 335-388) — TUI integration providing the `--share` checkbox
- **[`scripts/validate_community_benchmarks.py`](https://github.com/AlexsJones/llmfit/blob/main/scripts/validate_community_benchmarks.py)** — CI helper for pre-submission JSON validation
- **[`Cargo.toml`](https://github.com/AlexsJones/llmfit/blob/main/Cargo.toml)** — Declares `ureq` and `base64` powering API calls

## Summary

- **Local-first design** — Benchmarks store as JSON before any network operation, enabling offline workflows
- **Zero-dependency authentication** — OAuth device flow replaces CLI tools; environment variables skip interaction entirely
- **Self-contained GitHub operations** — `ureq` HTTP client handles all API calls without external binaries
- **Deduplication and safety** — Content hashing, hardware validation, and PR appending prevent noisy submissions
- **CI-friendly** — Headless operation with cached tokens supports automated benchmark pipelines

## Frequently Asked Questions

### Does llmfit require a GitHub account to submit benchmarks?

No pre-existing account is required for the initial device flow, but you will need to create a GitHub account during the OAuth authorization step if you don't have one. The tool guides you through this process in the browser. After first authentication, cached tokens enable fully non-interactive submissions.

### Can I use llmfit in CI pipelines without interactive prompts?

Yes. Set `GITHUB_TOKEN` or `GH_TOKEN` as an environment variable, or run the device flow once locally to cache a token, then copy the cache file to your CI environment. Use `assume_yes: true` in `ShareOptions` to suppress all interactive prompts.

### What happens if my hardware isn't detected properly?

Submissions are rejected locally before any network request. Ensure your GPU drivers are correctly installed and accessible from the runtime environment. Containerized workloads need proper GPU device passthrough for detection to succeed.

### Is there a way to preview what would be submitted?

Set `dry_run: true` in `ShareOptions`. This validates all pending benchmarks, shows the generated payloads and branch names, and reports what PR operations would occur—without creating any forks, branches, or pull requests.