Performance Analysis — Metrics, Methodology, Validation

Why Performance Metrics Matter

An architecture without measured performance is just a design. The examiner will ask: "How do you know your system actually works?" The answer lies in simulation metrics that quantify end-to-end behaviour across the full 780-day synodic period, validated against known real-world baselines.

The Core Metrics

Delivery Ratio — The fraction of generated bundles that reach their destination before expiring. The most fundamental metric: delivery_ratio = delivered / total. In the baseline simulation, AETHERIX targets >95% delivery ratio, with P0/P1 bundles at near-100%.

End-to-End Delay — The time from bundle creation to delivery, measured as deliver_time - creation_time. This includes light-time propagation (irreducible: 3–22 minutes one-way), queuing delay (waiting for the next contact), and processing overhead. The presentation claims DTN overhead under 5% — meaning total transit ≈ 13 minutes for a 12.5-minute light-time path.

Throughput — Total bytes delivered per unit time. For the optical link: 2–200 Mbps (distance-dependent). For Ka-band RF: 0.5–6 Mbps. The simulator measures throughput_mb = total_delivered_bytes / (1024 × 1024).

Average Hops — The mean number of intermediate nodes a bundle traverses. AETHERIX's 7-hop Earth-Mars path (surface → orbiter → Lagrange → Earth LEO → DSN → MOC) is typical. More hops mean more custody overhead and more failure risk.

Per-Priority and Per-Node Statistics

AETHERIX doesn't just compute aggregate metrics — it breaks them down:

Per-Priority Stats: Each BundlePriority level tracks delivered count, dropped count, and average delay separately. This reveals whether P0 emergency bundles truly get preferential treatment (they should: near-100% delivery, lowest delay) or whether the scheduler is fair across classes.

Per-Node Stats: Each node tracks queue size, bundles forwarded, bundles stored, and buffer utilization. This identifies bottlenecks — if one Lagrange relay consistently has 90%+ buffer utilization, it's a capacity constraint.

Simulation Methodology

The Simulator class in src/simulation/simulator.py is a time-stepped discrete simulation:

1. Build topology (create_default_topology)
2. Create forwarding engines for all nodes
3. Step from t=0 to t=duration in time_step_seconds increments
4. At each step:
   a. Probabilistically generate a new bundle (source: Tiers 4-5, dest: Tiers 1-2)
   b. Inject into source node's forwarding engine
   c. Call engine.tick() for every node
   d. Propagate forwarded bundles to next-hop engines
   e. Accumulate delivery/drop/expire statistics
5. Build SimulationResult with all metrics

The default configuration: - Duration: 720 hours (30 days) - Time step: 60 seconds - Seed: 42 (for reproducibility) - Earth-Mars distance: 225M km (average) - Optical data rate: 50 Mbps - RF data rate: 2 Mbps - Bundle generation rate: 10 per hour

Bundle generation is weighted by priority: [0.05, 0.10, 0.50, 0.20, 0.15] for P0–P4, meaning 50% of traffic is P2 Standard (the bulk of routine science), with small fractions of emergency and bulk.

Validation Layers

What Is NOT Yet Validated (Honesty)

How to state this in the oral: "I built the architecture and validated the algorithms on a transparent, dependency-free model. The reward function, the policy engine, and the conjunction-recovery logic are all proven. The absolute numbers are model-level estimates, not flight-measured. Phase 6 ports the validated policies into ION-DTN's convergence layers and re-runs them under ns-3 for packet-level realism."

Identifying Bottlenecks

The per-node statistics reveal where the system is constrained:

• Lagrange relays during conjunction: buffer utilization spikes as they become the only path
• Areostationary relays: queue depth grows if multiple surface assets transmit simultaneously
• DSN ground stations: rarely the bottleneck (10 TB buffer), but scheduling conflicts can delay

The policy engine's congestion_control (buffer > 90%) and the RL agent's proactive forwarding (penalised for drops) work together to prevent these bottlenecks from causing cascade failures.