FlyRank ML Research · Capstone Paper

Applied Search Intelligence: Algorithmic Content Decay Prioritization

A probabilistic machine learning framework for prioritizing organic search refresh opportunities under finite editorial bandwidth.

Author: Hamza Imran

Track: Applied Search Intelligence

Repository: github.com/HamzaKhanBUIC/flyrank-ml-internship

Release: August 2026

Abstract

Published web content quietly loses search traffic over time. When an enterprise library holds 30,000 pages and an editorial team can only review 50 each month, guessing what to update is costly. Simple rules—like "refresh old pages with traffic"—fail, hitting only 24% precision on decaying content. In this project, we built a machine learning ranking engine trained on 30,000 real URLs across 32 enterprise clients. Using strict client-holdout validation and clean, leakage-free signals, our Random Forest model hits 74.0% Precision@50 (a 3.08× lift over the baseline rule) with an ROC-AUC of 0.785. We then packaged these predictions into an editorial action playbook with clear reason codes, turning content maintenance into a focused, high-ROI workflow.

74.0%

Model Precision@50 (Test Clients)

3.08×

Empirical Lift vs Heuristic Baseline

30,000

Analyzed URLs (32 Client Domains)

0.785

Test ROC-AUC Discrimination

01 Introduction & Problem Statement

In search engine optimization (SEO), traffic decline is rarely catastrophic overnight. It is a slow, quiet leak. Over months, algorithm updates, SERP layout changes, and fresh competitors steadily push top-3 rankings down into striking distance (ranks 4–10) or off page one entirely.

For an enterprise managing 30,000 URLs, a content team can realistically rewrite or update only ~50 pages each month. Most teams rely on basic rules like "refresh any page older than 6 months that gets traffic." Our audit shows this rule hits only a 24.0% accuracy rate on truly declining content—wasting 76% of editorial time on healthy pages while decaying revenue-drivers slip away unnoticed.

02 Data Contract & Schema Boundaries

This study uses the FlyRank Enterprise Search Intelligence warehouse release (v20260703), tracking 30,000 pseudonymized URLs across 32 enterprise client domains over a 90-day trailing observation window ending 2026-06-30.

Field Bucket	Columns	Role & Constraints
Features (14)	`content_age_days`, `days_since_last_update`, `impressions_90d`, `avg_position`, `ctr`, `engagement_rate`, `scroll_rate`, `search_volume`, `cpc`, `competition`, `word_count`, flags	Pre-decision signals observable prior to prediction horizon.
Target Label	`is_declining_label` (binary 0/1)	Derived strictly from trailing 90d trend direction (`trend_direction == 'down'`).
Context Keys	`content_id`, `client_id`	Used exclusively for row indexing and group-partitioned splits; excluded from model inputs.
Forbidden (Leakage)	`trend_pct`, `trend_direction`, `health_score`	Excluded to prevent target leakage and circular heuristic logic.

03 Methodology & Validation Design

To make sure our findings generalize to new client domains without memorization tricks, we follow three practical rules:

Grouped Client-Holdout Split: We split strictly by client_id (80% train clients, 20% sealed test clients). No client ever appears in both partitions.
Structured Missingness Handling: Content types that lack keyword volume (like syndication feeds) carry explicit missingness flags instead of blind zero-fills.
Adversarial Leakage Audit: We deliberately tried feeding the model future columns; models with leaky fields scored a fake 1.000 precision, proving our clean feature set is 100% honest.

04 Benchmark Results vs Baseline

All models were trained on 25 client domains and evaluated strictly on the 7 sealed holdout test clients across Precision@20, Precision@50, and ROC-AUC against the population base rate (54.2%):

Architecture	Precision@20	Precision@50	ROC-AUC	Lift vs Baseline
Heuristic Baseline Rule	0.250	0.240	0.521	1.00× (Base)
Logistic Regression ($L_2$)	0.550	0.520	0.684	2.17×
Decision Tree (depth=4)	0.700	0.640	0.722	2.67×
Random Forest (n=100) — Champion	0.800	0.740	0.785	3.08×
HistGradientBoosting	0.750	0.720	0.779	3.00×

The Big Finding

Random Forest hits 74.0% Precision@50, catching 37 out of 50 decaying URLs compared to only 12 out of 50 by the standard heuristic rule.

05 Limitations & Honest Framing

Good engineering means being clear about what this model cannot do:

Observational, Not Causal: The model spots statistical patterns tied to past decay. It does not prove that making one edit (like adding 400 words) automatically restores traffic.
No Claims on Google's Algorithm: We do not reverse-engineer Google's internal code; we model site telemetry to help editors allocate time wisely.
Zero-Position Sentinel Limits: Pages with zero recorded search impressions need separate keyword retargeting.

06 Operational Content Action Playbook

We convert raw probabilities into an actionable queue using a composite Opportunity Score: $S = P(\text{decay}) \times \ln(1 + \text{impressions})$. Every recommendation includes an explainable reason code:

Action Code	Trigger Condition	Recommended Editorial Action
`REFRESH_METADATA`	Striking rank (4–10) + Low CTR (<25%)	Rewrite title tags & meta descriptions to win Page 1 clicks.
`EXPAND_CONTENT`	High impressions + Low engagement	Add clear comparison tables, FAQs, and expert insights.
`STALE_CONTENT_REFRESH`	Age > 180d + High remaining volume	Update outdated stats, refresh links, and verify facts.
`ROUTINE_MONITOR`	Decay probability < 40%	Healthy content requiring no immediate intervention.

07 Reproducibility & Code Repository

All models, data contracts, and validation scripts run with fixed random seeds (random_state=42) across Python 3.12 and scikit-learn.

View Code on GitHub Run in Google Colab

08 Acknowledgments & Data Credit

Built on the FlyRank ML Internship dataset provided by FlyRank AI (Enterprise Search Intelligence telemetry, release v20260703). Data credit is gratefully acknowledged.