How the Personal Cosmetics Recommendation Report Template Analyzes Skin Tone and Lip Color

The template uses a multi-stage pipeline: facial image analysis extracts RGB values for skin tone and lip regions, color conversion maps these to perceptual categories, and the markdown template renders the results with dynamic placeholders.

The cosmetics recommendation system in freestylefly/awesome-gpt-image-2 demonstrates how AI-powered image analysis can transform a simple portrait into personalized beauty guidance. This article breaks down exactly how the docs/templates.md report template captures and presents skin tone and lip color analysis, from raw image upload to finished markdown output.

The Three-Stage Analysis Pipeline

Stage 1: Image Upload and Vision API Processing

User portraits enter the system through src/main.jsx, which handles file selection and sends binary image data to the backend. The api/generate-image.js endpoint processes this through a vision model—either OpenAI's DALL·E or a custom CLIP-based implementation—that performs facial landmark detection and region sampling.

The vision model returns a structured JSON payload containing:

• Skin-tone region — average RGB values sampled from cheek and forehead areas
• Lip-color region — average RGB or HSV values extracted from the mouth bounding box

This extraction happens server-side to keep model weights and processing logic centralized.

Stage 2: Color Space Conversion and Categorization

Raw RGB values are perceptually non-uniform, so src/main.jsx transforms them to CIE L*a*b* space using the color-convert library. This conversion enables accurate distance calculations and consistent categorization across different lighting conditions.

// src/main.jsx – convert RGB → L*a*b* and map to cosmetic categories
import convert from 'color-convert';

function classifyColors({ skinToneRGB, lipColorRGB }) {
  const skinLab = convert.rgb.lab(skinToneRGB);
  const lipLab  = convert.rgb.lab(lipColorRGB);

  const skinTone = mapSkinTone(skinLab);   // → 'Winter', 'Spring', 'Summer', 'Autumn'
  const lipColor = mapLipColor(lipLab);    // → 'Berry', 'Coral', 'Nude', etc.
  
  const skinHex = `#${convert.rgb.hex(skinToneRGB)}`;
  const lipHex  = `#${convert.rgb.hex(lipColorRGB)}`;

  return { skinTone, skinHex, lipColor, lipHex };
}

The mapSkinTone() and mapLipColor() functions implement rule-based thresholds in Lab space. Skin tones map to seasonal color analysis categories (Winter, Spring, Summer, Autumn), while lip colors receive cosmetic-industry labels that align with product naming conventions.

Stage 3: Template Population and Rendering

The docs/templates.md file serves as a declarative skeleton containing Mustache-style placeholders. The renderReport() helper substitutes analyzed values into these positions, producing the final markdown document.

// src/main.jsx – render the markdown template with analyzed values
import template from '../docs/templates.md';

function renderReport(data) {
  return template
    .replace('{{skinTone}}', data.skinTone)
    .replace('{{skinToneHex}}', data.skinHex)
    .replace('{{lipColor}}', data.lipColor)
    .replace('{{lipColorHex}}', data.lipHex);
}

Template placeholders recognized by the system:

• {{skinTone}} — categorical label (e.g., "Warm Autumn")
• {{skinToneHex}} — hexadecimal color code for swatch rendering
• {{lipColor}} — cosmetic terminology (e.g., "Deep Berry")
• {{lipColorHex}} — corresponding hex value for visual reference

The completed markdown travels back to the client, where a markdown-to-HTML component renders it with src/styles.css providing color swatch styling.

Complete Frontend Implementation

The analysis and rendering flow is orchestrated from src/main.jsx as follows:

// src/main.jsx – full image-to-report pipeline
async function analyzePhoto(blob) {
  const form = new FormData();
  form.append('image', blob);
  
  const resp = await fetch('/api/generate-image', {
    method: 'POST',
    body: form
  });
  
  const rawColors = await resp.json();
  const classified = classifyColors(rawColors);
  const reportMarkdown = renderReport(classified);
  
  return reportMarkdown;
}

This single async function encapsulates the complete journey from user upload to printable cosmetic guidance.

Key Architectural Decisions

Separation of concerns — The template defines presentation; src/main.jsx handles analysis logic; api/generate-image.js manages model inference. This structure allows template updates without touching color science code.

Perceptual color spaces — Converting to L*a*b* before categorization compensates for RGB's non-linear response to human vision, ensuring consistent recommendations across device displays.

Hexadecimal preservation — Both Lab categories and original RGB hex codes are retained, giving users both semantic guidance ("Cool Winter") and precise color references for product matching.

Source Code Reference

File	Responsibility
docs/templates.md	Markdown skeleton with {{skinTone}}, {{lipColor}}, and related placeholders
src/main.jsx	Image upload, color conversion, categorization mapping, and template rendering
api/generate-image.js	Backend vision model endpoint returning sampled RGB values
src/styles.css	Color swatch presentation for the rendered report

Summary

• The personal cosmetics recommendation report template in docs/templates.md is a passive markdown structure filled by upstream analysis
• Skin tone and lip color analysis begins with vision model extraction of regional RGB values in api/generate-image.js
• Color conversion to CIE L*a*b* enables perceptually accurate categorization into seasonal and cosmetic industry labels
• Template rendering replaces placeholders with computed categories and hexadecimal codes via renderReport() in src/main.jsx
• The final output combines semantic guidance with precise color references for practical product selection

Frequently Asked Questions

Where is the actual color categorization logic located?

The mapping functions mapSkinTone() and mapLipColor() are implemented in src/main.jsx alongside the color conversion utilities. These functions contain threshold-based rules that classify Lab space coordinates into user-facing categories. The vision model in api/generate-image.js returns only raw RGB samples, leaving all interpretive logic in the frontend.

Why convert to L*a*b* instead of using RGB directly?

RGB values vary significantly under different lighting and camera sensors. CIE L*a*b* is designed to approximate human vision, making distance calculations between colors perceptually meaningful. This ensures that a "Warm Autumn" classification remains consistent whether the source photo was taken in sunlight or fluorescent office lighting.

Can the template be customized for different cosmetic brands?

Yes. Since docs/templates.md is a static markdown file with simple placeholder substitution, you can modify category names, add brand-specific product recommendations, or restructure the layout. The renderReport() function performs literal string replacement, so any new placeholders added to the template can be populated by extending the data object passed to it.

What determines the precision of the lip color extraction?

Precision depends on the facial landmark model's mouth region detection and the sampling strategy within api/generate-image.js. The current implementation averages RGB values across the detected lip bounding box, which smooths over texture variations like gloss or natural lip lines. For higher precision, the sampling region could be narrowed or semantic segmentation could isolate the vermilion border specifically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}