Understanding the Scoring Algorithm Used for Ranking Ideas in ADHD

The ADHD (Attention-Deficit-Hyper-Focus) ideation engine ranks ideas using a weighted sum of three LLM-evaluated dimensions—novelty (35%), viability (40%), and fit (25%)—followed by a two-stage sorting process that filters "trap" warnings and selects a final "non-obvious" pick.

The open-source repository UditAkhourii/adhd implements an automated brainstorming system that evaluates generated concepts through quantitative scoring. Understanding the specific scoring algorithm helps developers customize the ranking behavior or debug why certain ideas surface over others in the final output.

The Three Evaluation Dimensions

Every generated idea is evaluated across three quantitative dimensions supplied by an LLM critic. These dimensions are defined in the Score type within src/types.ts:

• Novelty – Measures how far the idea deviates from the obvious default solution (range: 0–10).
• Viability – Assesses whether the idea could realistically be shipped or work in practice (range: 0–10).
• Fit – Evaluates how directly the idea addresses the original problem statement (range: 0–10).

Each dimension is scored independently by the LLM module (src/llm.ts) before being passed to the scoring engine for aggregation.

The Weighted Scoring Formula

In src/engine.ts at line 218, the scoreIdeas function combines the three dimensions into a single total score using a weighted linear sum:

total = novelty * 0.35 + viability * 0.4 + fit * 0.25

The algorithm prioritizes viability at 40%, followed by novelty at 35%, and finally fit at 25%. This weighting ensures that impractical concepts are penalized despite high creativity, while ensuring alignment with the problem statement remains a secondary consideration.

The Two-Stage Ranking Process

After computing totals, the engine ranks ideas through two distinct stages to produce the final shortlist and selection.

Primary Ranking

First, the engine filters out any ideas containing a "trap" warning—indicators that the LLM identified hidden flaws or deceptive complexity. The remaining ideas are sorted by the total field in descending order:

const ranked = allIdeas
  .filter(i => i.score && !i.score.trap)
  .sort((a, b) => b.score!.total - a.score!.total);

This produces the main shortlist used for downstream processing.

Non-Obvious Pick Selection

From the top-K shortlist, the engine selects a single "non-obvious" pick by re-sorting candidates using a secondary heuristic that balances creativity against practicality:

const nonObviousPick = [...shortlist].sort(
  (a, b) => (b.score!.novelty + b.score!.viability * 0.5) -
            (a.score!.novelty + a.score!.viability * 0.5)
)[0];

This calculation gives full weight to novelty and half weight to viability, favoring ideas that are both surprising and moderately feasible over those that merely scored highest in the primary ranking.

Implementation in Code

The scoring system relies on type definitions in src/types.ts and the main logic in src/engine.ts. A simplified implementation demonstrating the complete workflow:

type Idea = { id: string; text: string; score?: Score };
type Score = {
  novelty: number;
  viability: number;
  fit: number;
  total: number;
  trap?: string;
  strength?: string;
};

function computeTotal(s: Score): number {
  return s.novelty * 0.35 + s.viability * 0.4 + s.fit * 0.25;
}

// Example usage
const ideas: Idea[] = [
  { 
    id: "1", 
    text: "Use a serverless edge function", 
    score: { novelty: 7, viability: 8, fit: 6, total: 0 } 
  },
  { 
    id: "2", 
    text: "Rewrite the whole stack in Rust", 
    score: { novelty: 9, viability: 4, fit: 8, total: 0 } 
  },
];

// Compute totals
ideas.forEach(i => {
  if (i.score) i.score.total = computeTotal(i.score);
});

// Primary ranking (exclude traps)
const ranked = ideas
  .filter(i => i.score && !i.score.trap)
  .sort((a, b) => b.score!.total - a.score!.total);

// Select non-obvious pick from top 3
const topK = ranked.slice(0, 3);
const nonObvious = topK.sort(
  (a, b) =>
    (b.score!.novelty + b.score!.viability * 0.5) -
    (a.score!.novelty + a.score!.viability * 0.5)
)[0];

Summary

• The ADHD scoring algorithm evaluates ideas on three dimensions: novelty, viability, and fit, each scored 0–10 by an LLM critic.
• Final scores use a weighted formula (0.35/0.4/0.25) implemented in src/engine.ts that prioritizes practical viability over pure creativity.
• Two-stage ranking first filters traps and sorts by total score, then selects a "non-obvious pick" using a secondary heuristic balancing novelty against half-weighted viability.
• Core implementation resides in src/engine.ts (scoring logic), src/types.ts (type definitions), and src/llm.ts (dimension evaluation).

Frequently Asked Questions

What are the exact weights used in the ADHD idea scoring algorithm?

The algorithm applies fixed weights of 35% for novelty, 40% for viability, and 25% for fit. This specific weighting is hardcoded in the scoreIdeas function within src/engine.ts at line 218, ensuring viability receives the highest priority in the final ranking.

How does the non-obvious pick differ from the top-ranked idea?

The top-ranked idea is determined solely by the weighted total score (novelty * 0.35 + viability * 0.4 + fit * 0.25), while the non-obvious pick is selected from the top-K shortlist using a different formula: novelty + (viability * 0.5). This secondary selection specifically favors ideas that maximize creative surprise while maintaining moderate practicality, potentially surfacing different concepts than the primary winner.

Where is the scoring logic implemented in the codebase?

The primary scoring implementation resides in src/engine.ts in the scoreIdeas function (around line 218). Type definitions for the Score interface are located in src/types.ts, and the LLM calls that generate the raw dimension scores are handled in src/llm.ts.

Can the scoring weights be customized for different projects?

While the current implementation in src/engine.ts uses hardcoded weights (0.35, 0.4, 0.25), the modular structure of the codebase allows customization. Developers can modify the computeTotal logic or override the scoreIdeas function to adjust the weighting coefficients based on whether a particular use case prioritizes novelty over viability or vice versa.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}