Architecture Trade Studies

Structured Comparison of Architecture Alternatives

Architecture decisions (Lesson 1) determine the system's fundamental structure. But how do you choose between alternatives when each has different strengths? You cannot optimize every dimension simultaneously — a cheaper architecture may be riskier, a more performant architecture may be harder to maintain, a safer architecture may take longer to develop.

A trade study is the disciplined process of comparing alternatives against weighted criteria using evidence rather than intuition. It transforms the question "which architecture is best?" into the answerable question "which architecture best satisfies the criteria that matter most for this system?"

Defining Evaluation Criteria

The first step is making the evaluation dimensions explicit. Criteria come from the architecture drivers identified in Lesson 1 — performance, scalability, safety, cost, schedule, maintainability — but they must be made specific enough to evaluate.

"Performance" is not a criterion. "Processing latency for sensor-to-actuator loop under maximum load" is a criterion. "Cost" is not a criterion. "Non-recurring engineering cost through first unit delivery" is a criterion. Vague criteria produce vague evaluations. Specific criteria force specific analysis.

Common criteria categories across engineering domains:

Technical performance. The system's ability to meet functional and performance requirements. Latency, throughput, accuracy, range, capacity, efficiency. These are usually quantifiable through analysis, simulation, or model-derived data from the system model (DGTLENG 201).

Cost. Development cost, production cost, operational cost, lifecycle cost. Trade studies that optimize for development cost often create systems that are expensive to operate. Lifecycle cost captures the full picture but is harder to estimate.

Risk. Technical risk (will it work?), schedule risk (will it be ready?), and programmatic risk (can the organization execute it?). Risk is assessed as the combination of likelihood and consequence. Architecture alternatives with novel technology carry higher technical risk; alternatives with tighter schedules carry higher schedule risk.

Schedule. Time to first capability, time to full deployment, critical path duration. Some architectures enable incremental delivery (federated, modular); others require the whole system to be ready before any capability is available (tightly coupled, centralized).

Manufacturability and producibility. How difficult is the architecture to build, assemble, test, and produce at scale? An architecture that performs brilliantly in simulation but requires manufacturing tolerances beyond current capability is not viable.

Maintainability and evolvability. How easily can the architecture be modified, upgraded, or repaired after deployment? This is often underweighted in trade studies because the pressure to deliver on schedule biases decisions toward near-term simplicity.

Weighting Criteria

Not all criteria matter equally. A safety-critical aerospace system weights safety and reliability above cost. A consumer software product weights time-to-market and user experience above theoretical performance. A civil infrastructure project weights lifecycle cost and maintainability above development schedule.

Weighting makes priorities explicit. Without weights, trade studies default to the criteria that the most senior person in the room cares about — which is a form of implicit weighting that is neither transparent nor repeatable.

Approaches to Weighting

Direct assignment. Stakeholders assign percentage weights that sum to 100%. Simple but subjective. Works well when the team has aligned priorities and enough experience to calibrate weights.

Pairwise comparison (AHP). The Analytic Hierarchy Process compares criteria in pairs — "Is performance more important than cost? By how much?" — and derives weights from the comparison matrix. More rigorous than direct assignment, and it surfaces inconsistencies in stakeholder preferences.

Swing weighting. Considers the range of variation across alternatives for each criterion. If all alternatives have similar cost but widely varying performance, performance gets higher weight because it differentiates more. This method ties weights to the decision context rather than to abstract importance.

The choice of weighting method matters less than the discipline of making weights explicit. Hidden weights produce hidden biases. Explicit weights produce auditable decisions.

Evaluating Alternatives

With criteria defined and weighted, each architecture alternative is evaluated against each criterion. Evaluation uses the best available evidence — and knowing what "best available" means is critical.

Sources of Evaluation Data

Model-derived data. The system model (DGTLENG 201) provides structural, behavioral, and parametric data for each alternative. Mass budgets, power budgets, interface counts, behavioral simulations, and parametric analyses all generate quantitative scores. This is where MBSE directly supports trade studies: alternatives modeled in the system model can be evaluated computationally rather than subjectively.

Simulation results. Performance simulations, Monte Carlo analyses, thermal models, structural analyses. Simulation provides quantitative performance data but inherits the assumptions and fidelity limitations of the simulation model.

Historical analogy. Past systems with similar architectures provide data on cost, schedule, reliability, and maintenance. Analogy is most useful when the alternatives are well-precedented; it is least useful for novel architectures.

Expert judgment. When data is unavailable or insufficient, calibrated expert judgment fills the gap. Expert judgment is not guessing — it is structured assessment by people with relevant experience. Techniques like Delphi method (independent assessments followed by facilitated convergence) reduce the influence of group dynamics.

Scoring

Each alternative receives a score on each criterion. Scoring can be quantitative (actual values normalized to a common scale) or qualitative (ordinal ratings like poor/fair/good/excellent mapped to numerical values). The critical requirement is that scoring is traceable — the rationale for each score is documented so the study can be reviewed and challenged.

The weighted score for each alternative is the sum of (criterion weight times criterion score) across all criteria. The alternative with the highest weighted score is the analytical winner — but not necessarily the decision.

The Pareto Frontier

When there are multiple objectives, no single alternative may be best on all criteria. The Pareto frontier identifies alternatives where improving one criterion necessarily worsens another — these are the efficient alternatives.

Consider two criteria: performance and cost. Some alternatives are clearly dominated — another alternative is both cheaper and higher-performing. Dominated alternatives can be eliminated. The remaining alternatives form the Pareto frontier: each is the best performer at its cost level, or the cheapest at its performance level. Moving along the frontier means trading performance for cost or vice versa.

The Pareto frontier is valuable because it reduces the decision space to the alternatives that actually matter. A ten-alternative trade study may have only three or four on the Pareto frontier. The rest are dominated and can be set aside. The remaining choice — where on the frontier to land — is a value judgment that the criteria and weights inform but do not dictate.

For more than two criteria, the Pareto frontier becomes a Pareto surface, and visualization becomes harder. But the principle holds: identify the dominated alternatives, eliminate them, and focus the decision on the efficient set.

When Data Is Not Enough

A trade study produces a ranked list of alternatives. But the ranked list is an input to the decision, not the decision itself. Several factors mean that the analytical winner may not be the right choice.

Sensitivity. If the top-ranked alternative's lead is within the uncertainty of the scores, the ranking is not robust. Sensitivity analysis — varying weights and scores within plausible ranges and observing which alternatives change rank — reveals whether the result is stable or fragile.

Threshold effects. An alternative that scores second overall but is the only one that meets a mandatory safety threshold is the right choice — weighted scoring cannot capture hard constraints. Hard constraints must be applied as filters before scoring, not folded into the weighted sum.

Organizational reality. An architecture that requires capabilities the organization does not have — skills, tools, supplier relationships, cultural alignment — may score well on paper and fail in execution. Trade studies evaluate the architecture; the decision must also evaluate the organization's ability to execute it.

Irreversibility asymmetry. If one alternative is easier to change course from later, it may be the better choice even if another scores higher today. Keeping options open has value when uncertainty is high — and architecture decisions early in the lifecycle always face high uncertainty.

The trade study provides structured evidence. The decision integrates that evidence with judgment about factors the study cannot quantify. Both are necessary; neither is sufficient alone.

Step through the four stages of the trade study process above. Notice how each stage builds on the previous one and how MBSE model data informs each stage. The process is sequential in concept but iterative in practice — evaluating alternatives often reveals that criteria need refinement or additional alternatives should be considered.

Assessment