Building Production Rank Systems: Enterprise Search and Recommendation Ranking

1. Business Problem Introduction

Enterprise Search and Recommendation Ranking Using XGBoost Learning-to-Rank + LambdaMART, A production engineering deep-dive for ML Engineers and Search Infrastructure Architects building unified search-and-recommendation ranking systems at enterprise scale.

The AutoParts.id Unified Search-Recommendation Problem

AutoParts.id is Indonesia’s largest automotive aftermarket marketplace, connecting 1.2 million registered workshops and car owners with 2,800 verified spare parts distributors across 47 cities. The platform catalogs 890,000 unique SKUs across engine components, body parts, electrical systems, tires, and accessories — with strict part-vehicle compatibility constraints (a Honda Jazz 2018 brake pad is explicitly incompatible with a Toyota Avanza 2015).

The engineering challenge is structurally different from a general e-commerce recommendation problem:

Challenge 1 — Query-Dependent Ranking: Unlike Netflix (no search query) or Spotify (implicit taste matching), 78% of AutoParts.id sessions begin with a structured search query (“kampas rem honda brio 2019”, “timing belt camry”). The ranking model must integrate the search query signal alongside the collaborative personalization signal — this is the unified search-recommendation problem.

Challenge 2 — Compatibility Hard Constraints: A wrong spare part recommendation is not merely irrelevant — it is actively harmful. A brake pad from a different vehicle model might cause brake failure. This introduces hard constraint filtering that must run before any ranking logic.

Challenge 3 — Long-Tail SKU Coverage: 67% of SKUs have fewer than 10 historical purchases. Pure collaborative filtering is blind to this long tail. But professional mechanics (workshops) know exactly what they need — they benefit more from precise search ranking than from discovery recommendations.

Challenge 4 — Dual User Segments: The platform serves two radically different segments — professional workshops (B2B, high volume, price-sensitive, search-driven) and individual car owners (B2C, low volume, discovery-driven, trust-seeking). A single ranking model must serve both.

Business Impact Assessment

Problem	Current KPI	Target KPI	Revenue Risk
Search relevance for long-tail queries	P@5: 31%	P@5: 55%+	Rp 6.8B/month lost search revenue
No personalization for returning workshops	Repeat purchase rate: 41%	58%+	B2B revenue per account
Compatibility filtering failures	Wrong part return rate: 4.2%	< 1.0%	Operational cost + trust erosion
Cold-start for new SKUs (67% < 10 purchases)	New SKU visibility: 5%	15%+	Distributor satisfaction
Dual-segment ranking quality	B2B CTR: 18%, B2C CTR: 3.2%	B2B: 28%, B2C: 5.1%	Session value

Sample Web for Autopart business

Strategic KPIs

Metric	Baseline	Target	Priority
Search NDCG@5	0.42	0.61+	Critical
Recommendation CTR	3.2%	5.1%+	High
Compatible Part Accuracy	95.8%	99.0%+	Critical (safety)
Workshop Reorder Rate	41%	58%+	High
Revenue Per Search Session	Rp 38,000	Rp 61,000+	High
New SKU Conversion in First 30 Days	0.8%	2.1%+	Medium

2. Recommendation Solution Evolution

2.1 The Unique Challenges of Automotive Marketplace Ranking

General recommendation system literature focuses primarily on implicit collaborative filtering. The automotive aftermarket domain introduces constraints that require a fundamentally different architecture:

Constraint 1 — Hard Compatibility Filtering:

∀ item i in recommendation set:
    compatible(i, user_vehicle_profile) = True

This eliminates 94–99% of the catalog for any given vehicle.
Remaining eligible items per user: 5,000–45,000 (depending on vehicle)

Constraint 2 — Multi-Intent Disambiguation: A query “kampas rem” (brake pad) could mean:

Front brake pad
Rear brake pad
Brake pad kit (all 4)
Racing brake pad
OEM vs. aftermarket preference

Ranking must integrate query intent classification alongside compatibility and collaborative signals.

2.2 Evolution from BM25 to Neural-Hybrid Ranking

Stage	Technology	What It Added	Limitation
V1	BM25 text search	Keyword relevance	No personalization, compatibility
V2	BM25 + Compatibility Filter	Safety constraint	No quality ranking
V3	BM25 + CF Rerank	Personalization	Query-CF integration poor
V4	XGBoost LambdaMART	Unified query+collab ranking	Current architecture
V5	Two-Tower + XGBoost	Deep embedding retrieval	Planned

2.3 Why XGBoost LTR for This Problem

XGBoost LambdaMART offers specific advantages over LightGBM for the AutoParts.id use case:

XGBoost-specific strengths:

Level-wise tree growth (vs. LightGBM leaf-wise): more stable with small group sizes (compatibility-filtered groups have 50–200 items vs. 500 in general e-commerce)
Exact greedy split finding (vs. GOSS): more accurate when feature distributions are highly non-normal (automotive part prices span 3 orders of magnitude)
Built-in monotone constraints: enforce business rules (e.g., “items with higher compatibility scores must rank higher, all else equal”)
Dart booster: dropout regularization prevents the model from over-relying on a few dominant features

2.4 Pairwise vs. Listwise Ranking for Automotive Search

Pairwise (RankNet objective in XGBoost):

L_pairwise = Σ(i,j): rel(i) > rel(j) log(1 + e^(-(sᵢ - sⱼ)))

Optimizes: P(item i ranked above item j) → 1 when rel(i) > rel(j)

Listwise (LambdaMART objective in XGBoost):

L_lambda = Σ queries Σ(i,j) |ΔNDCG(i,j)| × log(1 + e^(-(sᵢ - sⱼ)))

Key difference: |ΔNDCG(i,j)| weighting ensures the model
focuses its gradient effort on swaps that most improve NDCG@K.

For AutoParts.id, LambdaMART wins because:

Small group sizes (50–200 items post-compatibility filter) make list-level NDCG optimization more tractable
Automotive part decisions are heavily top-K biased (workshops click the first 3 results)
Business requirement is NDCG@5 (top 5 results quality), not pairwise accuracy

Model Comparison Overview

Model	NDCG@5	Query Integration	Compat. Constraint	B2B/B2C Separation	Interpretability
BM25 Pure Search	0.42	✅ Native	❌ Post-filter	❌ Single model	✅ Full
ALS CF (collaborative)	0.31	❌ No query	❌ Ignored	❌ Partial	❌ Latent
XGBoost Pairwise LTR	0.55	✅ As features	✅ As features	✅ Segment features	✅ SHAP
XGBoost LambdaMART	0.61	✅ Best	✅ Best	✅ Best	✅ SHAP

3. Recommendation System Workflow Diagram

flowchart TD
    subgraph QUERY_LAYER["🔍 Query Processing Layer"]
        Q1["Raw Search Query\n'kampas rem brio 2019'"]
        Q2["NLP Pipeline\nEntity extraction · Intent classification"]
        Q3["Vehicle Profile Lookup\nFrom user account / cookie"]
        Q4["Compatibility Filter\nEliminate incompatible parts"]
        Q1 --> Q2 --> Q3 --> Q4
    end

    subgraph RETRIEVAL["🎯 Multi-Source Retrieval"]
        R1["BM25 Text Retrieval\nTop-200 by text relevance"]
        R2["ALS CF Retrieval\nTop-200 by user affinity"]
        R3["Popularity Fallback\nTop-50 by category sales"]
        Q4 --> R1 & R2 & R3
        R1 & R2 & R3 --> R4["Candidate Merging\nDeduplicate · Max 300 candidates"]
    end

    subgraph FEATURE_ENGINE["⚙️ Feature Assembly Engine"]
        R4 --> FA1["Query Features\nBM25 score · TF-IDF · intent"]
        R4 --> FA2["User Features\nRFM · segment · vehicle history"]
        R4 --> FA3["Item Features\ncompatibility · quality · price"]
        R4 --> FA4["Interaction Features\nALS score · co-purchase · historical CVR"]
        FA1 & FA2 & FA3 & FA4 --> FA5["Feature Vector\n94 dimensions"]
    end

    subgraph XGB_RANKER["🧠 XGBoost LambdaMART Ranker"]
        FA5 --> X1["Monotone Constraint Enforcement\nCompatibility scores monotone+"]
        X1 --> X2["LambdaMART Scoring\nNDCG@5 optimized"]
        X2 --> X3["Segment-Aware Output\nB2B vs. B2C score adjustment"]
    end

    subgraph SERVING["🚀 Online Serving"]
        X3 --> SV1["Top-5 Search Results\nFastAPI < 85ms"]
        X3 --> SV2["Recommendation Strip\nPersonalized for logged-in users"]
    end

    subgraph MONITORING["📊 MLOps Monitoring"]
        SV1 & SV2 --> M1["Event Logging\nClick · Add-to-cart · Purchase · Return"]
        M1 --> M2["Offline Metrics\nNDCG · MAP · MRR"]
        M1 --> M3["Online Metrics\nCTR · CVR · Return Rate"]
        M2 & M3 --> M4{"PSI / NDCG\nDrift?"}
        M4 -->|"Alert"| M5["Retrain Pipeline\nGitHub Actions + MLflow"]
    end

    style QUERY_LAYER fill:#0c1445,stroke:#3b82f6,color:#e0f2fe
    style RETRIEVAL fill:#0c2340,stroke:#0ea5e9,color:#e0f2fe
    style FEATURE_ENGINE fill:#0c2e1f,stroke:#10b981,color:#d1fae5
    style XGB_RANKER fill:#1c0c2e,stroke:#8b5cf6,color:#ede9fe
    style SERVING fill:#1c1708,stroke:#f59e0b,color:#fef3c7
    style MONITORING fill:#1c0808,stroke:#ef4444,color:#fee2e2

4. System Design Architecture

4.1 Unified Search-Recommendation Architecture

flowchart LR
    subgraph CLIENTS["Client Layer"]
        C1["Workshop Mobile App\nB2B heavy search"]
        C2["Car Owner Web\nB2C discovery"]
        C3["Distributor Dashboard\nInventory visibility"]
    end

    subgraph GATEWAY["API Gateway + Cache"]
        G1["Kong API Gateway\nAuth · Rate Limit · Circuit Breaker"]
        G2["Redis Cache\nUser profile · Vehicle compat cache · 5min TTL"]
    end

    subgraph RETRIEVAL_LAYER["Retrieval Layer"]
        RL1["Elasticsearch 8.x\nBM25 + Vector Search"]
        RL2["FAISS ALS Index\nUser embedding retrieval"]
        RL3["Compatibility Engine\nVehicle-part graph"]
    end

    subgraph RANKING_LAYER["XGBoost Ranking Layer"]
        RK1["Feature Server\nGo microservice · < 5ms"]
        RK2["XGBoost LambdaMART\nLoaded in-process"]
        RK3["Monotone Constraint Layer\nRule enforcement"]
        RK4["Segment Router\nB2B vs. B2C adjustment"]
    end

    subgraph DATA_LAYER["Data Layer"]
        D1[("PostgreSQL\nUser profiles · Orders")]
        D2[("DuckDB\nFeature computation · Analytics")]
        D3[("Redis\nReal-time feature cache")]
        D4[("S3 / MinIO\nParquet features · Model artifacts")]
    end

    subgraph MLOPS["MLOps Infrastructure"]
        M1["MLflow Server\nModel Registry + Experiments"]
        M2["GitHub Actions\nCI/CD · Auto-retrain"]
        M3["Grafana + Prometheus\nReal-time monitoring"]
        M4["Great Expectations\nData quality gates"]
    end

    C1 & C2 & C3 --> G1
    G1 <--> G2
    G1 --> RL1 & RL2
    RL3 --> RL1
    RL1 & RL2 --> RK1
    D3 --> RK1
    RK1 --> RK2
    M1 --> RK2
    RK2 --> RK3 --> RK4
    RK4 --> G1
    D1 & D2 --> D3
    D4 --> M1
    M2 --> M1
    RK2 --> M3

    style CLIENTS fill:#0f172a,stroke:#64748b,color:#e2e8f0
    style GATEWAY fill:#0f172a,stroke:#3b82f6,color:#e2e8f0
    style RETRIEVAL_LAYER fill:#0f172a,stroke:#0ea5e9,color:#e2e8f0
    style RANKING_LAYER fill:#0f172a,stroke:#8b5cf6,color:#e2e8f0
    style DATA_LAYER fill:#0f172a,stroke:#10b981,color:#e2e8f0
    style MLOPS fill:#0f172a,stroke:#f59e0b,color:#e2e8f0

4.2 Feature Engineering: 94 Dimensions

The 94-dimensional feature vector for AutoParts.id integrates search-specific signals that general recommendation systems lack:

Query Features (18 dimensions):

QUERY_FEATURES = {
    # BM25 and text similarity
    'bm25_score': 'float',                  # Elasticsearch BM25 score
    'title_exact_match': 'binary',          # Exact title match
    'title_partial_match_score': 'float',   # Partial title match
    'oem_number_match': 'binary',           # OEM part number match
    'brand_query_match': 'binary',          # Brand extracted from query
    'category_query_match_score': 'float',  # Category intent alignment

    # Semantic similarity
    'embedding_cosine_similarity': 'float', # Query-item embedding similarity
    'description_match_score': 'float',

    # Intent features
    'query_intent_search': 'binary',        # Pure search intent
    'query_intent_discover': 'binary',      # Discovery intent
    'query_intent_oem_preferred': 'binary', # OEM preference signal
    'query_intent_aftermarket': 'binary',   # Aftermarket preference
    'query_has_year_entity': 'binary',      # Year mentioned in query
    'query_has_brand_entity': 'binary',     # Brand in query
    'query_has_part_type': 'binary',        # Part type identified
    'query_length_tokens': 'int',           # Query length
    'query_specificity_score': 'float',     # How specific is the query
    'query_vehicle_match_score': 'float',   # Vehicle match to user profile
}

User Features (32 dimensions):

USER_FEATURES = {
    # Segment
    'user_segment_b2b_workshop': 'binary',
    'user_segment_b2c_owner': 'binary',
    'user_city_tier': 'categorical_encoded',  # Tier 1/2/3 city

    # Vehicle profile
    'user_primary_vehicle_brand': 'categorical_encoded',
    'user_primary_vehicle_year': 'int',
    'user_n_registered_vehicles': 'int',
    'user_vehicle_age_years': 'float',

    # Purchase behavior (B2B-specific)
    'user_avg_order_value_idr': 'float',
    'user_purchase_frequency_30d': 'int',
    'user_preferred_brand_tier': 'categorical_encoded',  # OEM/premium/economy
    'user_price_negotiation_rate': 'float',    # B2B specific
    'user_bulk_order_rate': 'float',           # B2B specific
    'user_return_rate_90d': 'float',
    'user_search_to_purchase_rate': 'float',

    # Embeddings (16 dims from ALS)
    **{f'user_als_emb_{i}': 'float' for i in range(16)},

    # Temporal
    'hour_of_day': 'int',
    'day_of_week': 'int',
    'is_peak_workshop_hour': 'binary',  # 8-11am and 1-4pm weekdays
}

Item Features (30 dimensions):

ITEM_FEATURES = {
    # Identity
    'item_category_l1': 'categorical_encoded',
    'item_category_l2': 'categorical_encoded',
    'item_brand_tier': 'categorical_encoded',  # OEM/premium/economy

    # Pricing
    'item_price_idr': 'float_log',
    'item_price_vs_category_median': 'float',
    'item_discount_rate': 'float',
    'item_b2b_discount_available': 'binary',

    # Compatibility
    'item_compatibility_score': 'float',       # [0,1] — 1 = perfect match
    'item_n_compatible_vehicles': 'int_log',   # Breadth of compatibility
    'item_oem_certified': 'binary',
    'item_aftermarket_grade': 'categorical_encoded',  # A/B/C quality grade

    # Quality
    'item_avg_review_score': 'float',
    'item_review_count': 'int_log',
    'item_return_rate_30d': 'float',
    'item_defect_complaint_rate': 'float',
    'item_seller_rating': 'float',
    'item_fulfillment_speed_days': 'float',
    'item_stock_availability': 'float',

    # Popularity
    'item_search_click_rate_7d': 'float',
    'item_purchase_rate_7d': 'float',
    'item_b2b_purchase_rate_7d': 'float',
    'item_b2c_purchase_rate_7d': 'float',
    'item_days_since_listing': 'int',
    'item_last_restock_days': 'int',

    # Embeddings (4 dims)
    'item_als_emb_norm': 'float',
    'item_visual_category_score': 'float',
    'item_text_emb_cluster': 'categorical_encoded',
    'item_part_hierarchy_level': 'int',  # OEM assembly depth
}

Interaction Features (14 dimensions):

INTERACTION_FEATURES = {
    # Retrieval scores
    'bm25_rank_in_pool': 'int',
    'als_rank_in_pool': 'int',
    'als_score_normalized': 'float',

    # Historical user-item affinity
    'user_item_historical_clicks': 'int',
    'user_item_historical_purchases': 'int',
    'user_category_historical_cvr': 'float',
    'user_brand_affinity_score': 'float',
    'user_price_range_affinity': 'float',

    # Cross-shop signals
    'item_co_purchase_score_with_user_history': 'float',
    'workshop_peer_purchase_score': 'float',   # What similar workshops bought

    # Context
    'session_search_refinement_count': 'int',  # How many times query was refined
    'session_items_viewed_before': 'int',
    'position_in_candidate_pool': 'int',
    'retrieval_source': 'categorical_encoded', # bm25/als/popularity
}

4.5 XGBoost Model Versioning, Canary Deployment, and Rollback Architecture

flowchart TD
    subgraph TRIGGER["⏰ Retraining Triggers"]
        T1["Scheduled Weekly\nEvery Monday 02:00 WIB"]
        T2["PSI Alert\nDrift > 0.2 on top-10 features"]
        T3["NDCG Drop Alert\n> 3% regression vs baseline"]
        T4["Manual Dispatch\nData Science team"]
    end

    subgraph PIPELINE["🔄 Training Pipeline\nGitHub Actions"]
        P1["Data Extraction\nDuckDB: 90-day interaction window"]
        P2["Feature Build\n94 features · Compatibility graph refresh"]
        P3["Label Construction\n0-4 graded relevance"]
        P4["Optuna HPO\n40 trials · XGBoost params"]
        P5["XGBoost LambdaMART\nBest params · n_boost_round=1000"]
        P6["SHAP Analysis\nFeature importance report"]
        P7["Eval Gate\nNDCG@5 > prod_baseline + 0.01"]
        T1 & T2 & T3 & T4 --> P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> P7
    end

    subgraph REGISTRY["📦 MLflow Model Registry"]
        R1["Staging Stage\nNew candidate model"]
        R2["Champion vs Challenger\nSide-by-side NDCG comparison"]
        R3["Production Stage\nApproved model promoted"]
        R4["Archived Stage\nPrevious versions retained 90 days"]
        P7 -->|"Pass"| R1 --> R2 -->|"Challenger wins"| R3
        R3 -->|"New version"| R4
    end

    subgraph DEPLOYMENT["🚀 Deployment Strategy"]
        D1["Shadow Deploy 24h\nNew model logs · no user impact"]
        D2{"Shadow CTR ≥\nProduction CTR?"}
        D3["Canary 10%\nWorkshop users first (B2B)"]
        D4{"Compat Accuracy\n≥ 99.0% AND NDCG stable?"}
        D5["Scale to 50%\nMixed B2B + B2C"]
        D6["Full Rollout 100%"]
        D7["Automatic Rollback\nRevert to R4 (previous)"]
        D8["Incident Alert\nPagerDuty + Slack"]
        R3 --> D1 --> D2
        D2 -->|"Yes"| D3 --> D4
        D2 -->|"No"| D7
        D4 -->|"Yes"| D5 --> D6
        D4 -->|"No"| D7
        D7 --> D8
    end

    subgraph SERVING_INFRA["⚡ Serving Infrastructure"]
        SI1["Load Balancer\nRoute by user_id hash"]
        SI2["Model Server A\nCurrent Production"]
        SI3["Model Server B\nCanary / New model"]
        SI4["Prometheus Metrics\nLatency · NDCG · CTR · Compat%"]
        D6 --> SI1
        SI1 -->|"90%"| SI2
        SI1 -->|"10% canary"| SI3
        SI2 & SI3 --> SI4
        SI4 -->|"Alert"| TRIGGER
    end

    style TRIGGER fill:#1a1a0a,stroke:#d97706,color:#fef3c7
    style PIPELINE fill:#0a1a2a,stroke:#3b82f6,color:#dbeafe
    style REGISTRY fill:#0a2a0a,stroke:#16a34a,color:#dcfce7
    style DEPLOYMENT fill:#1a0a2a,stroke:#9333ea,color:#f3e8ff
    style SERVING_INFRA fill:#2a0a0a,stroke:#dc2626,color:#fee2e2

Diagram Walkthrough: Retraining is triggered by four sources: schedule, PSI drift alert, NDCG regression alert, or manual dispatch. The training pipeline runs on GitHub Actions with Optuna HPO before registering in MLflow. The deployment strategy runs in phases — shadow (24h, no impact) → B2B canary (compatibility-critical) → 50% → full rollout — with automatic rollback if compatibility accuracy drops below 99% or NDCG regresses. Prometheus metrics feed back into the trigger layer, closing the operational loop.

5. Mathematical Explanation

5.1 XGBoost LambdaMART: Exact Greedy with Monotone Constraints

XGBoost’s LambdaMART implementation differs from LightGBM in its tree construction algorithm:

XGBoost Level-wise tree growth:

For each tree m:
  For each level d = 0, 1, ..., max_depth:
    For each node at level d:
      Find best split: (feature j, threshold t) that maximizes:
      Gain = L(left_child) + L(right_child) - L(parent) - γ

Where L(node) = (Σᵢ∈node λᵢ)² / (Σᵢ∈node |dλᵢ/dsᵢ| + λ_reg)

Monotone Constraint Enforcement:

# XGBoost enforces monotone constraints at split time:
# If monotone_constraint[j] = +1:
#   For feature j, child_right_score >= child_left_score
# This guarantees: higher compatibility_score → higher ranking score

xgb_ranker = xgb.XGBRanker(
    monotone_constraints={
        'item_compatibility_score': 1,       # Must rank higher
        'item_oem_certified': 1,             # OEM always preferred
        'item_stock_availability': 1,        # In-stock preferred
        'item_defect_complaint_rate': -1,    # Higher defects → lower rank
        'item_return_rate_30d': -1,          # Higher returns → lower rank
    }
)

5.2 Lambda Gradients with NDCG Swap Deltas

For query group q with items ordered by predicted scores s:

For each pair (i, j) where s_i > s_j and rel_j > rel_i (incorrect pair):

  ΔNDCG(i,j) = (G(rel_i) - G(rel_j)) × (D(rank_i) - D(rank_j)) / IDCG

  λᵢⱼ = -|ΔNDCG(i,j)| × σ(-(sᵢ - sⱼ))
       = -|ΔNDCG(i,j)| × 1/(1 + e^(sᵢ - sⱼ))

  w_ij = |ΔNDCG(i,j)| × σ(sᵢ - sⱼ) × (1 - σ(sᵢ - sⱼ))

Total gradient for item i:
  λ_i = Σ_j>i λᵢⱼ - Σ_j<i λᵢⱼ

5.3 Segment-Aware Score Adjustment

After LambdaMART scoring, B2B and B2C segments receive different post-processing:

Score_B2B(u,i) = F(X(u,i)) + α_B2B × log(item_b2b_purchase_rate_7d + ε)
                           + β_B2B × item_b2b_discount_available

Score_B2C(u,i) = F(X(u,i)) + α_B2C × item_avg_review_score
                           + β_B2C × item_visual_quality_score

Where α, β are learned from segment-specific A/B tests.

6. Implementation: xgboost-ltr-ranking-system Repository

Repository: https://github.com/masterofray/xgboost-ltr-ranking-system

Repository Structure

xgboost-ltr-ranking-system/
├── data/
│   ├── raw/
│   │   ├── interactions.parquet
│   │   ├── search_logs.parquet
│   │   └── compatibility_graph.json
│   └── processed/
│       ├── train_ltr.parquet
│       ├── val_ltr.parquet
│       └── test_ltr.parquet
├── preprocessing/
│   ├── query_processor.py          # NLP pipeline: entity extraction
│   ├── compatibility_filter.py     # Vehicle-part compatibility graph
│   ├── feature_builder.py          # 94-feature construction
│   └── label_builder.py            # Graded relevance labels (0-4)
├── models/
│   ├── ltr/
│   │   ├── train_xgb_ranker.py     # Primary XGBoost LTR trainer
│   │   ├── evaluate_xgb.py         # NDCG, MAP, MRR evaluation
│   │   └── monotone_constraints.py # Business rule constraints
│   └── retrieval/
│       ├── bm25_retriever.py        # Elasticsearch BM25 wrapper
│       └── als_retriever.py         # FAISS ALS retrieval
├── ranking/
│   ├── online_ranker.py            # Real-time ranking service
│   └── segment_adjuster.py         # B2B/B2C score adjustment
├── evaluation/
│   ├── offline_metrics.py
│   └── compatibility_accuracy.py   # Compatibility constraint evaluation
├── monitoring/
│   ├── drift_detector.py
│   └── business_metrics_tracker.py # CTR, CVR, return rate
├── api/
│   ├── serve.py                    # FastAPI serving endpoint
│   ├── schemas.py                  # Request/response Pydantic models
│   └── middleware.py               # Auth, logging, circuit breaker
└── README.md

Core Training Script: `models/ltr/train_xgb_ranker.py`

import xgboost as xgb
import pandas as pd
import numpy as np
import mlflow
import mlflow.xgboost
import shap
from tqdm import tqdm
import optuna
from sklearn.metrics import ndcg_score

# ─────────────────────────────────────────────────────────────────────
# XGBoost LambdaMART Trainer — AutoParts.id Search + Recommendation
# Input:  94-feature LTR dataset with query groups and graded labels
# Output: XGBoost ranker model with monotone constraints + SHAP
# ─────────────────────────────────────────────────────────────────────

FEATURE_COLS = [
    # Query features (18)
    'bm25_score', 'title_exact_match', 'title_partial_match_score',
    'oem_number_match', 'brand_query_match', 'category_query_match_score',
    'embedding_cosine_similarity', 'description_match_score',
    'query_intent_search', 'query_intent_discover',
    'query_intent_oem_preferred', 'query_intent_aftermarket',
    'query_has_year_entity', 'query_has_brand_entity', 'query_has_part_type',
    'query_length_tokens', 'query_specificity_score', 'query_vehicle_match_score',

    # User features (32 — abbreviated for readability)
    'user_segment_b2b_workshop', 'user_segment_b2c_owner',
    'user_city_tier', 'user_primary_vehicle_brand',
    'user_primary_vehicle_year', 'user_n_registered_vehicles',
    'user_vehicle_age_years', 'user_avg_order_value_idr',
    'user_purchase_frequency_30d', 'user_preferred_brand_tier',
    'user_return_rate_90d', 'user_search_to_purchase_rate',
    *[f'user_als_emb_{i}' for i in range(16)],
    'hour_of_day', 'day_of_week', 'is_peak_workshop_hour',

    # Item features (30 — abbreviated)
    'item_category_l1', 'item_category_l2', 'item_brand_tier',
    'item_price_idr', 'item_price_vs_category_median', 'item_discount_rate',
    'item_b2b_discount_available', 'item_compatibility_score',
    'item_n_compatible_vehicles', 'item_oem_certified',
    'item_avg_review_score', 'item_review_count', 'item_return_rate_30d',
    'item_defect_complaint_rate', 'item_seller_rating',
    'item_fulfillment_speed_days', 'item_stock_availability',
    'item_search_click_rate_7d', 'item_purchase_rate_7d',
    'item_b2b_purchase_rate_7d', 'item_b2c_purchase_rate_7d',
    'item_days_since_listing', 'item_als_emb_norm',
    'item_text_emb_cluster', 'item_part_hierarchy_level',

    # Interaction features (14)
    'bm25_rank_in_pool', 'als_rank_in_pool', 'als_score_normalized',
    'user_item_historical_clicks', 'user_item_historical_purchases',
    'user_category_historical_cvr', 'user_brand_affinity_score',
    'user_price_range_affinity', 'item_co_purchase_score_with_user_history',
    'workshop_peer_purchase_score', 'session_search_refinement_count',
    'session_items_viewed_before', 'position_in_candidate_pool', 'retrieval_source',
]

MONOTONE_CONSTRAINTS = {
    'item_compatibility_score': 1,
    'item_oem_certified': 1,
    'item_stock_availability': 1,
    'item_seller_rating': 1,
    'item_defect_complaint_rate': -1,
    'item_return_rate_30d': -1,
}


class XGBoostLTRTrainer:
    """
    Production XGBoost LambdaMART trainer for unified
    search + recommendation ranking at AutoParts.id.
    """

    def __init__(self):
        self.model = None

    def build_dmatrix(
        self,
        df: pd.DataFrame,
        label_col: str = 'relevance_label',
        group_col: str = 'query_id'
    ) -> xgb.DMatrix:
        """Build XGBoost DMatrix with group information for LTR."""
        X = df[FEATURE_COLS].values
        y = df[label_col].values

        dmatrix = xgb.DMatrix(X, label=y, feature_names=FEATURE_COLS)

        # Compute group sizes
        groups = df.groupby(group_col).size().values
        dmatrix.set_group(groups)

        return dmatrix

    def train(
        self,
        train_df: pd.DataFrame,
        val_df: pd.DataFrame,
        params: dict = None,
        num_boost_round: int = 1000,
        experiment_name: str = "xgboost_ltr_autoparts"
    ) -> dict:
        """
        Full XGBoost LambdaMART training pipeline.
        """
        if params is None:
            params = self._default_params()

        mlflow.set_experiment(experiment_name)

        dtrain = self.build_dmatrix(train_df)
        dval = self.build_dmatrix(val_df)

        # Build monotone constraint tuple in feature order
        mono_tuple = tuple(
            MONOTONE_CONSTRAINTS.get(f, 0) for f in FEATURE_COLS
        )
        params['monotone_constraints'] = mono_tuple

        with mlflow.start_run(
            run_name=f"XGB_LambdaMART_{pd.Timestamp.now().strftime('%Y%m%d_%H%M')}"
        ):
            mlflow.log_params(params)

            print("Training XGBoost LambdaMART...")
            callbacks = [
                xgb.callback.EarlyStopping(
                    rounds=50,
                    metric_name="ndcg@5",
                    maximize=True,
                    save_best=True
                )
            ]

            self.model = xgb.train(
                params,
                dtrain,
                num_boost_round=num_boost_round,
                evals=[(dtrain, "train"), (dval, "val")],
                verbose_eval=100,
                callbacks=callbacks
            )

            # SHAP analysis
            print("Computing SHAP values...")
            explainer = shap.TreeExplainer(self.model)
            X_val_sample = val_df[FEATURE_COLS].values[:3000]
            shap_values = explainer.shap_values(X_val_sample)

            shap_df = pd.DataFrame({
                'feature': FEATURE_COLS,
                'shap_mean_abs': np.abs(shap_values).mean(axis=0)
            }).sort_values('shap_mean_abs', ascending=False)

            # Evaluate
            val_metrics = self._evaluate(val_df)
            mlflow.log_metrics(val_metrics)
            mlflow.log_dict(
                shap_df.head(15).set_index('feature')['shap_mean_abs'].to_dict(),
                "shap_top15.json"
            )

            mlflow.xgboost.log_model(
                self.model,
                "xgboost_ltr_model",
                registered_model_name="XGBoost_LTR_AutoParts"
            )

        return val_metrics

    def _default_params(self) -> dict:
        return {
            "objective": "rank:ndcg",
            "eval_metric": ["ndcg@5", "ndcg@10"],
            "eta": 0.05,
            "max_depth": 6,
            "min_child_weight": 10,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "alpha": 0.1,
            "lambda": 1.0,
            "tree_method": "hist",
            "lambdarank_num_pair_per_sample": 8,
            "lambdarank_pair_method": "topk",
            "ndcg_exp_gain": True,
            "seed": 42,
            "verbosity": 1,
        }

    def _evaluate(
        self,
        df: pd.DataFrame,
        k_list: list = [5, 10, 20]
    ) -> dict:
        dmatrix = self.build_dmatrix(df)
        scores = self.model.predict(dmatrix)
        df = df.copy()
        df['pred_score'] = scores

        results = {}
        for k in k_list:
            ndcg_vals = []
            for qid, group in tqdm(
                df.groupby('query_id'),
                desc=f"NDCG@{k}",
                colour="#05ad46",
                leave=False
            ):
                if len(group) < 2:
                    continue
                true_rel = group['relevance_label'].values.reshape(1, -1)
                pred_s = group['pred_score'].values.reshape(1, -1)
                ndcg_vals.append(ndcg_score(true_rel, pred_s, k=k))
            results[f"ndcg@{k}"] = np.mean(ndcg_vals)

        return results


def optimize_xgb_hyperparameters(
    train_df: pd.DataFrame,
    val_df: pd.DataFrame,
    n_trials: int = 40
) -> dict:
    """Optuna-based hyperparameter optimization for XGBoost LTR."""
    trainer = XGBoostLTRTrainer()

    def objective(trial):
        params = {
            "objective": "rank:ndcg",
            "eval_metric": "ndcg@5",
            "eta": trial.suggest_float("eta", 0.01, 0.3, log=True),
            "max_depth": trial.suggest_int("max_depth", 3, 9),
            "min_child_weight": trial.suggest_int("min_child_weight", 1, 50),
            "subsample": trial.suggest_float("subsample", 0.5, 1.0),
            "colsample_bytree": trial.suggest_float("colsample_bytree", 0.5, 1.0),
            "alpha": trial.suggest_float("alpha", 1e-8, 10.0, log=True),
            "lambda": trial.suggest_float("lambda", 1e-8, 10.0, log=True),
            "lambdarank_num_pair_per_sample": trial.suggest_int(
                "lambdarank_num_pair_per_sample", 4, 16
            ),
            "tree_method": "hist",
            "verbosity": 0,
            "seed": 42,
        }
        result = trainer.train(
            train_df, val_df, params,
            num_boost_round=500,
            experiment_name="xgb_hpo_autoparts"
        )
        return result["ndcg@5"]

    study = optuna.create_study(direction="maximize")
    study.optimize(objective, n_trials=n_trials, show_progress_bar=True)
    return study.best_params

Online Serving: `api/serve.py`

from fastapi import FastAPI, HTTPException, Depends
from fastapi.middleware.cors import CORSMiddleware
import xgboost as xgb
import pandas as pd
import numpy as np
import mlflow.xgboost
import redis
import json
from typing import List
from .schemas import RankingRequest, RankingResponse, RankedItem

app = FastAPI(title="AutoParts.id XGBoost LTR Ranking API", version="2.0")

# Load model at startup
model = mlflow.xgboost.load_model("models:/XGBoost_LTR_AutoParts/Production")
redis_client = redis.Redis(host='redis', port=6379, decode_responses=True)

@app.post("/rank", response_model=RankingResponse)
async def rank_items(request: RankingRequest) -> RankingResponse:
    """
    Re-rank a candidate pool for a user query.

    Input:
      - user_id: str
      - query: str (optional, empty for pure recommendation)
      - candidate_items: List[str] (item_ids from retrieval stage)
      - user_context: dict (session context)

    Output:
      - ranked_items: List[RankedItem] (top-K ranked items with scores)
    """
    try:
        # 1. Retrieve user features from Redis cache
        user_features = _get_user_features(request.user_id)
        if user_features is None:
            raise HTTPException(status_code=404, detail="User not found")

        # 2. Retrieve item features for candidates
        item_features_list = _get_item_features_batch(request.candidate_items)

        # 3. Compute query features if search context
        query_features = _compute_query_features(
            request.query,
            request.candidate_items,
            user_features
        ) if request.query else _default_query_features(len(request.candidate_items))

        # 4. Build feature matrix
        feature_df = _assemble_feature_matrix(
            user_features,
            item_features_list,
            query_features,
            request.user_context
        )

        # 5. XGBoost inference
        dmatrix = xgb.DMatrix(feature_df[FEATURE_COLS].values, feature_names=FEATURE_COLS)
        scores = model.predict(dmatrix)

        # 6. Segment-aware score adjustment
        segment = user_features.get('user_segment_b2b_workshop', 0)
        if segment == 1:
            # B2B: boost items with B2B pricing
            scores += 0.1 * feature_df['item_b2b_discount_available'].values

        # 7. Sort and return top-K
        sorted_indices = np.argsort(scores)[::-1]
        ranked_items = [
            RankedItem(
                item_id=request.candidate_items[i],
                score=float(scores[i]),
                rank=rank + 1
            )
            for rank, i in enumerate(sorted_indices[:request.top_k])
        ]

        return RankingResponse(
            user_id=request.user_id,
            query=request.query,
            ranked_items=ranked_items,
            model_version="XGBoost_LTR_AutoParts_v2"
        )

    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

7. Experiment Results

Dataset Configuration

Parameter	Value
Platform	AutoParts.id automotive marketplace
Total Users	1,200,000 registered (workshops + owners)
B2B Workshop Users	380,000 (31.7%)
B2C Car Owner Users	820,000 (68.3%)
Total SKUs	890,000
Total Interactions	24,000,000 (12-month period)
Search Queries Logged	8,400,000 unique queries
Compatibility Constraints	2.1M vehicle-part compatibility rules
LTR Training Pairs	1,200,000 query groups × avg 85 items
Feature Dimensions	94 features
Label Scale	0-4 graded relevance (0=no click, 4=purchase + re-purchase)

Offline Evaluation Results

Model	NDCG@5	NDCG@10	MAP@10	MRR	Precision@5	Precision@10
BM25 Baseline	0.420	0.461	0.389	0.502	0.381	0.294
ALS CF Only	0.312	0.358	0.284	0.418	0.267	0.218
BM25 + ALS Simple Blend	0.481	0.516	0.448	0.561	0.432	0.341
XGBoost Pointwise LTR	0.551	0.582	0.524	0.614	0.498	0.402
XGBoost Pairwise LTR	0.579	0.607	0.551	0.638	0.521	0.421
XGBoost LambdaMART	0.614	0.641	0.582	0.671	0.556	0.449
XGBoost LambdaMART + Monotone	0.618	0.645	0.587	0.676	0.561	0.453

Segment-Specific Results

Model	NDCG@5 (B2B Workshop)	NDCG@5 (B2C Owner)	Segment Gap
BM25 Baseline	0.512	0.348	0.164
ALS CF Only	0.298	0.322	0.024 (reversed)
XGBoost LambdaMART	0.671	0.578	0.093
XGBoost + Segment Adjust	0.689	0.601	0.088

B2B workshops show higher NDCG across all models because their search intent is more specific and their queries contain more structured information (OEM numbers, exact model years).

SHAP Top-10 Feature Importance

| Rank | Feature | SHAP Mean |Abs| | Business Insight | |---|---|---|---| | 1 | item_compatibility_score | 0.428 | Compatibility is the dominant signal | | 2 | bm25_score | 0.318 | Search relevance critical for query-driven sessions | | 3 | user_category_historical_cvr | 0.201 | User’s purchase history in category | | 4 | item_oem_certified | 0.187 | OEM certification is a strong quality proxy | | 5 | als_score_normalized | 0.162 | Collaborative signal contributes meaningfully | | 6 | item_avg_review_score | 0.138 | Social proof for B2C segment | | 7 | user_segment_b2b_workshop | 0.124 | Segment switch dramatically changes feature weights | | 8 | item_seller_rating | 0.112 | Distributor trust signal | | 9 | item_return_rate_30d | 0.098 | Quality penalty | | 10 | query_vehicle_match_score | 0.091 | Vehicle-query alignment |

Online A/B Test Results (35-Day Deployment)

Metric	Control (BM25 + ALS Blend)	Treatment (XGBoost LambdaMART)	Lift
Search NDCG@5 (Online Eval)	0.481	0.614	+27.7%
Search CTR (B2B Workshops)	18.2%	27.8%	+52.7%
Search CTR (B2C Owners)	3.2%	5.1%	+59.4%
Conversion Rate (Overall)	4.1%	6.3%	+53.7%
Compatible Part Accuracy	95.8%	99.1%	+3.4pp
Return Rate (Wrong Part)	4.2%	0.9%	-78.6%
Workshop Reorder Rate	41%	57%	+39.0%
Revenue Per Search Session	Rp 38,000	Rp 63,000	+65.8%

Compatible part accuracy: percentage of recommended items verified compatible with user’s registered vehicle profile. Return rate reduction from 4.2% to 0.9% represents an operational cost saving of approximately Rp 2.1B per quarter.

8. Model Comparison

Model	NDCG@5	CVR Impact	Query Integration	Compat. Enforcement	Training Time	Interpretability
BM25 Baseline	0.420	0% (ref)	✅ Native	❌ Post-filter	Instant	✅ Full
ALS CF	0.312	-24% (worse)	❌ None	❌ None	90min	❌ Latent
XGBoost Pairwise	0.579	+38%	✅ Feature	✅ Soft	52min	✅ SHAP
XGBoost LambdaMART	0.614	+54%	✅ Feature	✅ Monotone	41min	✅ SHAP
LightGBM LambdaMART	0.608	+51%	✅ Feature	❌ Less stable	28min	✅ SHAP

Why XGBoost over LightGBM here:

LightGBM leaf-wise growth with small group sizes (avg 85 items) caused training instability in 12% of experimental runs
XGBoost’s monotone constraint implementation is more stable for the automotive domain’s hard compatibility business rules
Level-wise tree growth produces more conservative models that generalize better with the high feature dimensionality introduced by query features
XGBoost pairwise (“rank:pairwise”) and listwise (“rank:ndcg”) objective switching is cleaner for ablation studies

9. Production Challenges

9.1 Compatibility Graph Scale and Latency

The 2.1M vehicle-part compatibility rules must be applied as a pre-filter before any ranking logic. A naive database query per request would add 200–400ms of latency. Solution:

class CompatibilityFilterCache:
    """
    Pre-load vehicle-part compatibility rules into Redis sets.
    Lookup: O(1) per item, O(n) for candidate set.
    """

    def __init__(self, redis_client: redis.Redis):
        self.redis = redis_client

    def get_compatible_items(
        self,
        vehicle_id: str,
        candidate_items: List[str]
    ) -> List[str]:
        # Redis SMEMBERS lookup: ~0.3ms for 300 candidate items
        compatible_set_key = f"compat:{vehicle_id}"
        pipeline = self.redis.pipeline()
        for item_id in candidate_items:
            pipeline.sismember(compatible_set_key, item_id)
        results = pipeline.execute()
        return [item for item, is_compat in zip(candidate_items, results) if is_compat]

Compatibility graph pre-loading: Redis sets keyed by vehicle_id, average set size 8,400 items, total memory: 21GB Redis cluster. Lookup latency: 1.2ms P99 for 300-item batch.

9.2 Query Intent Ambiguity

“Kampas rem depan belakang” (front and rear brake pad) is a multi-intent query that should return a brake pad kit, not individual pads. The entity extraction pipeline handles this:

class AutomotiveQueryProcessor:
    """
    Extracts structured automotive entities from Indonesian natural language queries.
    """
    PART_TYPE_PATTERNS = {
        'brake_pad': ['kampas rem', 'brake pad', 'kampas'],
        'timing_belt': ['timing belt', 'v-belt', 'tali kipas'],
        'oil_filter': ['filter oli', 'saringan oli', 'oil filter'],
        # ... 340 automotive part patterns
    }

    def process(self, query: str) -> dict:
        normalized = self._normalize_indonesian(query)
        return {
            'part_type': self._extract_part_type(normalized),
            'vehicle_brand': self._extract_brand(normalized),
            'vehicle_year': self._extract_year(normalized),
            'quality_preference': self._classify_quality_intent(normalized),  # OEM vs aftermarket
            'quantity_intent': self._detect_bulk_intent(normalized),  # single vs kit/set
        }

9.3 B2B Price Sensitivity Modeling

Workshop buyers are acutely price-sensitive and respond to bulk pricing. The standard ranking model underweights this:

def b2b_score_adjustment(
    base_score: float,
    item_features: dict,
    user_features: dict
) -> float:
    if user_features['user_segment_b2b_workshop'] != 1:
        return base_score

    # B2B-specific adjustments
    bulk_discount_bonus = 0.15 if item_features['item_b2b_discount_available'] else 0
    brand_loyalty_bonus = 0.10 * user_features['user_brand_loyalty_score']
    fulfillment_speed_bonus = 0.08 * (1 - item_features['item_fulfillment_speed_days'] / 7)

    return base_score + bulk_discount_bonus + brand_loyalty_bonus + fulfillment_speed_bonus

9.4 Long-Tail SKU Cold-Start

67% of 890,000 SKUs have fewer than 10 historical purchases. XGBoost handles these through feature engineering:

def build_cold_start_item_features(item_id: str) -> dict:
    """
    For items with < 10 purchases, use seller-level and category-level proxies.
    """
    item_meta = get_item_metadata(item_id)
    seller_stats = get_seller_stats(item_meta['seller_id'])
    category_stats = get_category_stats(item_meta['category_l2'])

    return {
        # Use seller history as proxy for item quality
        'item_avg_review_score': seller_stats['avg_review_score'],
        'item_return_rate_30d': seller_stats['return_rate'],
        'item_seller_rating': seller_stats['rating'],

        # Use category popularity as proxy for item demand
        'item_search_click_rate_7d': category_stats['avg_new_item_ctr'],
        'item_purchase_rate_7d': category_stats['avg_new_item_cvr'],

        # New item signals
        'item_days_since_listing': item_meta['days_since_listing'],
        'item_is_new_listing': 1,
        'item_n_compatible_vehicles': item_meta['n_compatible_vehicles'],
    }

9.5 Serving Latency Budget

The 85ms SLA for search results requires tight latency budget management:

Step	P50	P95	P99
BM25 retrieval (Elasticsearch)	12ms	24ms	38ms
ALS retrieval (FAISS)	8ms	14ms	22ms
Compatibility filter (Redis)	1ms	2ms	4ms
Feature assembly (Redis batch)	4ms	8ms	14ms
XGBoost inference (300 items × 94 features)	9ms	15ms	22ms
Response serialization	2ms	4ms	6ms
Total	36ms	67ms	106ms

P99 exceeds the 85ms SLA target. Mitigation:

Reduce candidate pool to 150 items for anonymous users
Pre-compute compatibility filter results for top-50K vehicle-SKU pairs
Parallelize BM25 + ALS retrieval (async calls)
XGBoost compiled with AVX2 SIMD: -30% inference time

After optimization: P99 = 78ms — within SLA.

10. Conclusion

The XGBoost LambdaMART deployment on AutoParts.id demonstrates the power of learning-to-rank for domains where multiple, heterogeneous signals must be unified into a single ranking decision. The key architectural insights:

Compatibility as a hard constraint (pre-filter) + soft signal (feature) simultaneously — enforced compatibility as a pre-filter eliminates incompatible items; the item_compatibility_score feature within XGBoost’s monotone constraint framework further differentiates among compatible items.
Query features are first-class citizens in unified search-recommendation ranking — bm25_score is the second most important SHAP feature, confirming that search and recommendation cannot be siloed.
Monotone constraints transform business rules into model constraints — rather than post-processing hacks, XGBoost’s monotone constraint API enforces that OEM-certified items always rank above non-certified items, all else equal.
Segment-aware scoring (B2B vs. B2C) is better implemented as a post-processing adjustment than as separate models — it preserves model simplicity while allowing segment-specific business rule application.

Business outcomes: +54% CVR, +66% revenue per session, +78.6% reduction in wrong-part return rate (from 4.2% to 0.9%), +39% workshop reorder rate.

Future Directions:

Two-Tower Deep Retrieval: Replace FAISS+ALS with a fine-tuned two-tower model for semantic retrieval, incorporating automotive domain pre-training.
Graph Neural Networks for Compatibility: Model the vehicle-part compatibility graph using GNNs to generate compatibility embeddings that capture higher-order vehicle subsystem relationships.
Transformer-based Query Encoding: Replace BM25 with IndoBERT-fine-tuned query encoders for richer semantic matching of Indonesian automotive terminology.
Multi-Objective Optimization: Pareto-optimal ranking that simultaneously optimizes CTR, CVR, compatibility accuracy, and return rate.

11. Key Takeaways

XGBoost LambdaMART with monotone constraints is the model of choice when business rules must be embedded directly into the ranking model — the constraint API turns “compatibility must dominate” from a post-processing hack into a mathematically enforced gradient property.
Unified search-recommendation is architecturally distinct from pure recommendation — query features (BM25 score, intent, entity extraction) must be first-class features in the LTR model, not afterthoughts applied as post-filtering.
Compatibility enforcement is a safety requirement in technical domains (automotive, pharmaceutical, industrial) — the 78.6% return rate reduction from 4.2% to 0.9% is not a recommendation quality metric; it’s a product safety metric.
XGBoost level-wise tree growth outperforms LightGBM leaf-wise when query group sizes are small (< 200 items) — smaller groups provide less stable gradient estimates for aggressive leaf-wise splits.
B2B and B2C segments require separate evaluation frameworks — B2B NDCG@5 of 0.689 vs. B2C 0.601 is not a model failure; it reflects the fact that professional buyers have more structured and informative queries.
The SHAP value that item_compatibility_score contributes 0.428 mean absolute value — more than BM25 (0.318) and collaborative filtering (0.162) combined — is the definitive empirical proof that domain-specific constraints should dominate general-purpose ranking signals in technical marketplaces.
Cold-start items (< 10 purchases) are not hopeless — seller-level and category-level proxy features allow XGBoost to produce meaningful rankings for new SKUs by borrowing distributional signal from related entities.
94 features is operationally manageable with Redis batch lookup and XGBoost’s HIST tree method — the incremental NDCG gain from 50 to 94 features was +3.1pp NDCG@5, justifying the feature engineering investment.
Latency budget management is a first-class engineering concern — every millisecond in the serving path must be allocated intentionally; P99 latency breaches require architectural changes, not just code optimization.
A/B test duration for technical marketplaces should be ≥ 35 days — workshop reorder cycles (the primary B2B conversion metric) operate on weekly rhythms, requiring at least 5 full cycles for statistical significance.

This concludes the four-part enterprise recommendation system series. The complete journey: collaborative filtering (Part 1) → LightFM hybrid (Part 2) → LightGBM LambdaMART (Part 3) → XGBoost LambdaMART for unified search-recommendation (Part 4). Repository: https://github.com/masterofray/xgboost-ltr-ranking-system