How SmsForwarder Prevents Duplicate Message Forwarding with Its Deduplication Mechanism

SmsForwarder implements a time-based deduplication system that compares message timestamps against a user-configurable window stored in SharedPreferences, skipping forwarding when identical messages arrive within the defined time limit.

The open-source Android app SmsForwarder automates SMS forwarding for privacy and automation workflows. To prevent downstream services from receiving spammed duplicates, the app implements a lightweight deduplication mechanism that operates at the worker level. This system leverages Android SharedPreferences to store previous message timestamps and evaluates incoming candidates against configurable millisecond thresholds.

How the Deduplication Mechanism Works

The deduplication logic resides primarily in SendWorker.kt and relies on timestamp comparison rather than heavy hashing or database lookups. This design keeps the background service fast and memory-efficient.

Timestamp Comparison in SendWorker.kt

When SendWorker processes a message, it retrieves the limit from SettingUtils and converts seconds to milliseconds:

val duplicateMessagesLimits = SettingUtils.duplicateMessagesLimits * 1000L

If the limit exceeds zero, the worker compares the current message timestamp against the previously stored value:

if (duplicateMessagesLimits > 0L) {
    Log.d(TAG, "duplicateMessagesLimits=$duplicateMessagesLimits, timestamp=$timestamp, timestampPrev=$timestampPrev, msgInfo=$msgInfo")
    if (timestampPrev != timestamp && timestamp - timestampPrev <= duplicateMessagesLimits) {
        // Duplicate detected – skip forwarding
        return Result.success()
    }
}

This check executes in app/src/main/kotlin/cn/ppps/forwarder/workers/SendWorker.kt at lines 71-78. When the time difference between timestamp and timestampPrev is less than or equal to the configured limit, the worker returns Result.success() immediately without forwarding, preventing duplicate SMS from flooding webhook endpoints.

Extended Protection for SMS Commands

The SmsCommandUtils.kt file implements the same logic for SMS-based system commands with an additional safety buffer:

var duplicateMessagesLimits = SettingUtils.duplicateMessagesLimits * 1000L
if (duplicateMessagesLimits > 0L) {
    duplicateMessagesLimits += 10_000L   // Add 10s safety buffer for commands
    if (timestampPrev != timestamp && timestamp - timestampPrev <= duplicateMessagesLimits) {
        // Duplicate command detected – ignore
        return
    }
}

Located in app/src/main/kotlin/cn/ppps/forwarder/utils/SmsCommandUtils.kt at lines 96-103, this implementation adds a 10-second grace period specifically for command messages. This ensures critical system commands are not accidentally filtered due to rapid successive handling while still protecting against true duplicates.

Configuration and Storage

The deduplication window is user-configurable through the Settings UI and persists across app restarts via Android SharedPreferences.

SettingUtils.kt and SharedPreferences

In app/src/main/kotlin/cn/ppps/forwarder/utils/SettingUtils.kt, the deduplication limit is defined as a delegated property backed by SharedPreferences at line 72:

var duplicateMessagesLimits: Int by SharedPreference(SP_DUPLICATE_MESSAGES_LIMITS, 0)

The constant SP_DUPLICATE_MESSAGES_LIMITS serves as the storage key, with a default value of 0 meaning deduplication is disabled by default.

SettingsFragment.kt User Interface

Users configure the limit through a seek bar in the settings fragment. The following code in app/src/main/kotlin/cn/ppps/forwarder/fragment/SettingsFragment.kt at lines 162-164 binds the UI to the storage layer:

binding!!.xsbDuplicateMessagesLimits.setDefaultValue(SettingUtils.duplicateMessagesLimits)
binding!!.xsbDuplicateMessagesLimits.setOnSeekBarListener { _: XSeekBar?, newValue: Int ->
    SettingUtils.duplicateMessagesLimits = newValue
}

This writes the selected value (in seconds) back to SettingUtils, which immediately persists it to SharedPreferences.

Key Source Files

The deduplication mechanism spans four critical files in the pppscn/SmsForwarder repository:

• app/src/main/kotlin/cn/ppps/forwarder/workers/SendWorker.kt: Core forwarding worker that performs the timestamp comparison and early return logic
• app/src/main/kotlin/cn/ppps/forwarder/utils/SmsCommandUtils.kt: Utility class handling SMS commands with extended deduplication buffers
• app/src/main/kotlin/cn/ppps/forwarder/utils/SettingUtils.kt: SharedPreferences wrapper managing the duplicateMessagesLimits configuration and SP_DUPLICATE_MESSAGES_LIMITS key
• app/src/main/kotlin/cn/ppps/forwarder/fragment/SettingsFragment.kt: UI fragment exposing the deduplication controls via xsbDuplicateMessagesLimits

Summary

• Time-based filtering: SmsForwarder prevents duplicate forwarding by comparing message timestamps against a configurable millisecond window stored in SettingUtils.duplicateMessagesLimits
• Early exit pattern: When duplicates are detected within the limit, workers return Result.success() immediately without executing network operations
• Dual protection: Both standard SMS forwarding (SendWorker.kt) and SMS command processing (SmsCommandUtils.kt) implement the check, with commands receiving an additional 10-second buffer
• Lightweight storage: The system uses Android SharedPreferences via SP_DUPLICATE_MESSAGES_LIMITS to persist the timestamp limit, avoiding database overhead
• User control: The SettingsFragment exposes a seek bar to configure the window, with a default of 0 seconds (deduplication disabled)

Frequently Asked Questions

How do I enable deduplication in SmsForwarder?

Navigate to the Settings screen and adjust the Duplicate Messages Limits seek bar to your desired time window in seconds. Setting the value to 0 disables deduplication, while any positive value activates the timestamp comparison logic in SendWorker.kt.

What happens when a duplicate message is detected?

When SendWorker identifies a duplicate (where current timestamp minus previous timestamp is less than or equal to the limit), it returns Result.success() immediately without forwarding. This silently drops the duplicate while acknowledging the work as complete to the Android WorkManager system.

Why does SmsCommandUtils add an extra 10 seconds?

The SmsCommandUtils.kt file adds a 10,000-millisecond buffer to the user-configured limit specifically for SMS commands. This safety margin prevents critical system commands from being filtered due to rapid successive processing or minor timestamp inconsistencies, ensuring command reliability while maintaining protection against duplicates.

Where is the deduplication setting stored?

The deduplication limit persists in Android SharedPreferences under the key SP_DUPLICATE_MESSAGES_LIMITS. The SettingUtils class wraps this storage with a delegated property, allowing both the UI (SettingsFragment) and background workers (SendWorker) to access the current configuration synchronously.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}