# How to Add Custom Context Features to the context_features_extractor Module in RPDNN

> Learn to add custom context features to the context features extractor module. Extend functions, increment dimensions, and verify output shape for your RPDNN models.

- Repository: [jerrygao/rpdnn](https://github.com/jerrygaolondon/rpdnn)
- Tags: how-to-guide
- Published: 2026-03-04

---

**To add custom context features to the `context_features_extractor` module, extend the `extract_social_numerical_features` function in [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py), increment `NUMERICAL_FEATURE_DIM`, and verify the output shape matches `EXPECTED_CONTEXT_INPUT_SIZE`.**

The `context_features_extractor` module in the [jerrygaolondon/rpdnn](https://github.com/jerrygaolondon/rpdnn) repository generates the numeric representation of reaction tweets—replies and retweets—that feeds into the Rumour Detection RNN. Adding custom context features allows you to enrich this representation with new measurable properties, such as hashtag counts or sentiment scores, by extending the core extraction pipeline. This guide identifies the exact source code locations, constants, and validation steps required to safely integrate new feature dimensions.

## Understanding the Context Feature Extraction Pipeline

Before modifying code, understand the three-stage architecture that processes each reaction tweet:

1. **Numeric Feature Extraction**: The `extract_social_numerical_features` function in [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py) computes a flat numeric vector from user-profile, tweet-level, and temporal attributes.
2. **Embedding Concatenation**: `context_feature_extraction_from_context_status` calls the numeric extractor and optionally appends ELMo embeddings for user profiles and reply content via `sentence_embedding_elmo`.
3. **Batch Stacking**: `context_feature_extraction` loops over all context tweets and stacks per-reaction vectors into a matrix of shape `(N, EXPECTED_CONTEXT_INPUT_SIZE)` for the downstream RNN.

To add a custom feature, you must extend the numeric vector produced in stage one and ensure the dimensionality constants reflect the change.

## Step-by-Step Implementation Guide

### Step 1: Define Your Custom Feature

Select a measurable property of a reaction tweet that can be computed solely from the `reaction_status_json` object passed to the extractor. Examples include the number of hashtags, sentiment polarity, or domain-specific flags. Avoid features requiring external I/O during extraction.

### Step 2: Implement the Feature in `extract_social_numerical_features`

Open [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py) and locate the `extract_social_numerical_features` function (lines 45–61). Append your custom logic after the existing `user_features` and `tweet_features` calculations:

```python
def extract_social_numerical_features(reaction_status_json, source_tweet_user_id, 
                                      source_tweet_user_screen_name, source_text):
    # Existing feature extraction

    user_features = user_features_main(reaction_status_json, source_tweet_user_id)
    tweet_features = tweet_features_main(reaction_status_json,
                                         source_tweet_user_screen_name,
                                         source_text)
    
    # Build base numerical vector

    numerical_features = []
    numerical_features.extend((user_features + tweet_features))
    
    # 👇 Custom feature: count hashtags in the reaction tweet

    hashtags = reaction_status_json.get('entities', {}).get('hashtags', [])
    num_hashtags = len(hashtags)
    numerical_features.append(num_hashtags)
    
    # Append temporal and meta features (preserve existing order)

    t_diff = calculate_time_diff(...)  # existing variable

    c_type = determine_context_type(...)  # existing variable

    has_description = check_description(...)  # existing variable

    numerical_features.extend([t_diff, c_type, has_description])
    
    return np.array(numerical_features)

```

This ensures every reaction processed by `context_feature_extraction` includes your new numeric cue.

### Step 3: Update Dimensionality Constants

Two module-level constants in [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py) control the tensor shape:

- `NUMERICAL_FEATURE_DIM`: Defines the length of the flat numeric vector (default is `28`)
- `EXPECTED_CONTEXT_INPUT_SIZE`: Defines the total width expected by the RNN (default is `20 * 3`)

If you add a single numeric field, increment `NUMERICAL_FEATURE_DIM` accordingly:

```python

# src/context_features_extractor.py

NUMERICAL_FEATURE_DIM: int = 29   # Increased from 28 to include hashtag count

EXPECTED_CONTEXT_INPUT_SIZE: int = 20 * 3  # Adjust only if changing total embedding size

```

Search the repository for hard-coded slices or references to the old dimension (e.g., `[..., :28]`) and update them to prevent shape mismatches downstream.

### Step 4: Validate with the Test Harness

Run the built-in validation to confirm the new feature integrates without breaking the expected tensor shape:

```python
from src.context_features_extractor import test_context_features_shape_by_tweet_id

# Verify shape matches EXPECTED_CONTEXT_INPUT_SIZE

test_context_features_shape_by_tweet_id("500350777844457473")

```

If the assertion passes, your custom feature is correctly integrated. If it fails, check that `NUMERICAL_FEATURE_DIM` matches the actual length of your modified `numerical_features` array.

## Optional: Extending User and Tweet Feature Modules

For features that fit naturally into user-profile or tweet-level categorizations, consider extending the helper modules instead of inline code:

- **[`src/context_features/user_features.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features/user_features.py)**: Add user-specific custom cues (e.g., account age buckets) by modifying `user_features_main`.
- **[`src/context_features/tweet_features.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features/tweet_features.py)**: Add tweet-specific cues (e.g., media attachment flags) by modifying `tweet_features_main`.

Return the new values as additional list elements, then handle the concatenation in `extract_social_numerical_features` as shown in Step 2.

## Summary

- **Inject new numeric cues** exclusively in `extract_social_numerical_features` within [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py).
- **Synchronize dimensionality constants** by incrementing `NUMERICAL_FEATURE_DIM` to match your new vector length.
- **Avoid shape errors** by grepping for hard-coded dimension references (e.g., `NUMERICAL_FEATURE_DIM` or `28`) across the codebase.
- **Validate changes** using `test_context_features_shape_by_tweet_id` to ensure the RNN receives tensors of the expected width.
- **Document additions** with inline comments explaining feature semantics for future maintainers.

## Frequently Asked Questions

### Where exactly do I modify the code to add a custom numeric feature?

Add your computation logic inside the `extract_social_numerical_features` function in [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py), specifically after the existing `user_features` and `tweet_features` lists are combined but before the final conversion to a NumPy array (around line 55). This is the only location where the flat numeric vector is constructed.

### What happens if I forget to update `NUMERICAL_FEATURE_DIM`?

If you add features to the vector but leave `NUMERICAL_FEATURE_DIM` at its default value (28), downstream components that rely on this constant for padding, slicing, or shape validation will encounter dimension mismatches. The RNN may raise tensor shape errors during training or inference because `EXPECTED_CONTEXT_INPUT_SIZE` will no longer align with the actual data width.

### Can I add features that require external API calls or database lookups?

No. The `context_features_extractor` is designed to operate solely on the `reaction_status_json` dictionary passed to it. External I/O would significantly slow down the batch processing loop in `context_feature_extraction`. Pre-compute external data and embed it into the JSON before extraction, or add lookup logic upstream in your data pipeline.

### How do I test that my custom feature is working correctly?

Use the `test_context_features_shape_by_tweet_id` function provided in [`src/context_features_extractor.py`](https://github.com/jerrygaolondon/rpdnn/blob/main/src/context_features_extractor.py). Pass a valid tweet ID string to this function; it will extract context features for that tweet and assert that the output matrix shape matches `EXPECTED_CONTEXT_INPUT_SIZE`. If the assertion passes, your dimensionality updates are correct.