How WeChat Article Exporter Handles Pagination for Large Article Lists

WeChat Article Exporter paginates through extensive article collections by sending offset-based requests with a fixed page size of 20, looping through the getArticleList utility until the API returns an empty result set.

WeChat Article Exporter is an open-source tool designed to archive content from WeChat public accounts. When dealing with accounts containing hundreds of articles, the application implements a robust pagination strategy in apis/index.ts to fetch data incrementally without overwhelming the WeChat API or client memory.

Pagination Architecture Overview

The pagination system relies on a stateless, offset-based approach where the client maintains the current position and requests fixed-size chunks until no data remains.

Configuration Constants

The page size is centralized in the configuration layer. In config/index.ts, the constant ARTICLE_LIST_PAGE_SIZE is defined as 20, establishing the number of articles fetched per request.

The Core API Function

The primary pagination logic resides in apis/index.ts within the getArticleList function. This async function accepts an account object, a begin offset parameter, and an optional keyword, then forwards the request to the Nitro server endpoint /api/web/mp/appmsgpublish.

// apis/index.ts
export async function getArticleList(
  account: MpAccount, 
  begin: number, 
  keyword?: string
): Promise<[AppMsgEx[], boolean, number]>

How the Offset-Based Pagination Works

Request Parameters

Each pagination request includes two critical parameters sent to the internal endpoint:

• begin: The zero-based offset indicating the starting position in the article list
• size: Fixed at the value of ARTICLE_LIST_PAGE_SIZE (20 items)

The getArticleList function constructs the request with these parameters along with authentication tokens derived from the account object.

Detecting the End of the List

Inside apis/index.ts (lines 42-46), the code examines the publish_page.publish_list array returned by the WeChat API. If this array is empty, the function returns isCompleted = true, signaling that all articles have been retrieved.

// apis/index.ts
const publishList = res.data?.publish_page?.publish_list || []
if (publishList.length === 0) {
  return [[], true, totalCount] // isCompleted = true
}

When articles are present, the function maps the raw data to AppMsgEx objects and returns isCompleted = false, prompting the client to initiate another request with an incremented offset.

Optional Progress Tracking

The API response includes publish_page.total_count, which getArticleList extracts and returns as the third element of its tuple (line 65). This allows the UI to display progress indicators (e.g., "Downloading 45 of 200") without requiring additional API calls.

Client-Side Pagination Loop

Composable Implementation

High-level composables such as useDownloader and useBatchDownload orchestrate the pagination loop. They maintain an accumulator array and an offset counter, repeatedly calling getArticleList until the returned isCompleted value is true.

The following pattern demonstrates how these composables aggregate large datasets:

import { getArticleList, ARTICLE_LIST_PAGE_SIZE } from '@/apis'

async function fetchAllArticles(account: MpAccount) {
  const all: AppMsgEx[] = []
  let offset = 0
  let completed = false
  let total = 0

  while (!completed) {
    const [page, isCompleted, totalCount] = await getArticleList(account, offset)
    all.push(...page)
    completed = isCompleted
    total = totalCount          // optionally show progress
    offset += ARTICLE_LIST_PAGE_SIZE
  }

  console.log(`Fetched ${all.length} / ${total} articles`)
  return all
}

This loop continues incrementing the offset by 20 until the backend reports an empty page, ensuring complete data retrieval regardless of list length.

Handling Keyword Searches

When a keyword parameter is provided to getArticleList, the same pagination mechanism applies; however, the results are not cached locally. The conditional logic in apis/index.ts only persists unfiltered results to the cache, ensuring that keyword-specific subsets do not pollute the main article store.

Summary

• Fixed Page Size: The constant ARTICLE_LIST_PAGE_SIZE in config/index.ts sets a 20-item limit per request, balancing API load and network efficiency.
• Offset Tracking: The begin parameter in getArticleList enables stateless pagination, allowing the client to resume or restart downloads without server-side session management.
• Completion Detection: An empty publish_list array triggers isCompleted = true, providing a clear termination condition for the download loop.
• Total Count Metadata: The total_count field enables accurate progress bars in the UI components.
• Composable Orchestration: Files like composables/useDownloader.ts and useBatchDownload.ts implement the while-loop logic that aggregates paginated results into complete datasets.

Frequently Asked Questions

What page size does WeChat Article Exporter use for pagination?

The application fetches 20 articles per request, defined by the ARTICLE_LIST_PAGE_SIZE constant exported from config/index.ts. This value is passed as the size parameter in every call to the WeChat API via the /api/web/mp/appmsgpublish endpoint.

How does the application know when it has fetched all articles?

The getArticleList function in apis/index.ts checks if the returned publish_page.publish_list array is empty. When the API returns no items for a given offset, the function sets isCompleted to true, signaling composables like useDownloader to exit the pagination loop.

Can the pagination handle keyword searches?

Yes. When a keyword argument is passed to getArticleList, the same offset-based pagination applies, filtering results server-side. However, the codebase specifically excludes keyword-filtered results from local caching to prevent cache pollution, as seen in the conditional logic within apis/index.ts.

Where is the pagination logic implemented in the codebase?

The core pagination logic spans three layers: configuration in config/index.ts, API abstraction in apis/index.ts (specifically the getArticleList function), and orchestration in client-side composables like composables/useDownloader.ts and composables/useBatchDownload.ts, which manage the looping and state accumulation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}