Designing for Autonomy

The Architecture of Decision-Making

When you design a conventional product, you specify its behavior: in this condition, do this. The behavior is encoded in hardware, firmware, or software that implements the specification. The product does what it was told to do.

When you design an agentic product, you specify its decision-making architecture: how it perceives, how it reasons, how it selects actions. The specific behavior in any given situation is not authored by you — it emerges from the architecture you designed interacting with conditions you may not have anticipated.

This is a fundamentally different kind of design. You are not designing behavior. You are designing the capacity for behavior.

Intelligence Architectures

Fixed Behavior

The simplest architecture. The product's response to every input is determined at design time. Lookup tables, hard-coded thresholds, fixed control laws.

Design challenge: Specify all relevant input conditions and the correct response for each. For simple products in controlled environments, this is tractable. For complex products in variable environments, the specification becomes impossibly large.

Where it works: Safety interlocks, alarm systems, basic control loops. Products where the environment is bounded and the correct response is unambiguous.

Rule-Based Architecture

A set of condition-action rules, often organized by priority. If multiple rules apply, the highest-priority rule fires. Rules may reference internal state variables that track the product's history.

Design challenge: Managing rule interactions. Individual rules may be correct, but their combination may produce unintended behavior. As the rule set grows, the interaction space explodes. Rule conflicts, gaps (situations where no rule applies), and cascading rule activations create bugs that are difficult to find through inspection alone.

Where it works: Industrial control systems, building automation, process monitoring. Products where expert knowledge can be expressed as rules and the environment is structured enough that the rule set is manageable.

Sense-Plan-Act

The classical robotics architecture. Three sequential stages:

1. Sense: Perceive the environment through sensors and build an internal representation (a map, a scene graph, a state vector)
2. Plan: Given the internal representation and the goal, compute a sequence of actions to achieve the goal
3. Act: Execute the planned actions through actuators

Design challenge: The sequential nature creates latency. In a fast-changing environment, the world may change between sensing and acting, invalidating the plan. Planning is computationally expensive for complex action spaces. The internal representation must be rich enough for planning but fast enough to update in real time.

Where it works: Robot navigation in semi-structured environments, drone path planning, warehouse automation. Products where the environment changes slowly enough for the sense-plan-act cycle to keep up.

Behavior Trees

A hierarchical architecture that composes simple behaviors into complex strategies. A behavior tree is a directed tree where:

• Leaf nodes are actions (move forward, turn, grasp) or conditions (is obstacle detected? is goal reached?)
• Internal nodes control execution flow: sequence (do these in order), selector (try these until one succeeds), parallel (do these simultaneously)

Design challenge: Designing behavior trees that produce robust behavior across diverse situations. The modularity is an advantage — behaviors can be developed and tested independently — but the interactions between branches can produce surprising emergent behavior.

Where it works: Game AI, robot behavior control, autonomous vehicle tactical decision-making. Products where behavior can be decomposed into composable sub-behaviors.

Hierarchical Task Networks

A planning architecture that decomposes high-level goals into sub-goals, sub-goals into tasks, and tasks into primitive actions. The decomposition is guided by domain knowledge encoded as methods (recipes for achieving sub-goals).

Design challenge: Encoding the domain knowledge that guides decomposition. The quality of the product's behavior depends on the completeness and correctness of the methods. Missing methods mean the product cannot handle certain situations. Incorrect methods mean it handles them badly.

Where it works: Mission planning for autonomous systems, complex task sequencing, manufacturing process control. Products with well-structured goal hierarchies and known decomposition patterns.

End-to-End Learned Systems

A neural network (or ensemble of networks) that maps directly from sensor inputs to control outputs. No explicit internal model, no hand-coded rules, no engineered planning algorithm. The decision-making is learned from data.

Design challenge: The behavior is implicit in the network weights. There is no specification to inspect, no rules to review, no architecture to trace. The product does what it learned to do — and understanding why it makes a specific decision in a specific situation requires interpretability techniques that are still maturing. Training data quality and coverage directly determine product quality.

Where it works: Image-based perception (what is in the scene?), certain control tasks where physics-based models are impractical, and situations where the optimal behavior is too complex to specify explicitly.

The MBSE Dimension

For agentic products, the system model (MBSE) includes something that conventional products do not: behavioral specifications that ARE the product's intelligence, not just its structure.

In conventional MBSE, the model captures:

• What the system is (structure)
• What the system does (behavior as specifications that get implemented)
• What the system must satisfy (requirements)
• How elements relate (interfaces, allocations, traces)

For agentic products, the model must also capture:

• Decision architectures: How does the product decide? What architecture pattern (sense-plan-act, behavior tree, learned system) is used and what are its parameters?
• Goal specifications: What should the product achieve? Goals may have priorities, may conflict, and may change with context.
• Constraint envelopes: What must the product never do? Safety constraints, operational boundaries, ethical limits.
• Learning specifications: What does the product learn from? What data feeds its learning? What bounds constrain what it can learn (preventing catastrophic drift)?
• Environmental assumptions: What does the product assume about its operating environment? Where do those assumptions break?

This is a richer model than conventional MBSE demands — and it is essential because the behavior of an agentic product cannot be verified by testing it against a static specification. The specification itself includes the decision-making architecture, which must be modeled, analyzed, and validated as part of the engineering process.

Choosing an Architecture

The choice of decision-making architecture depends on:

Environment complexity. Bounded, well-characterized environments favor rule-based or sense-plan-act architectures. Open-ended, unpredictable environments push toward learned systems or hybrid approaches.

Safety criticality. When incorrect decisions have severe consequences, inspectable architectures (rule-based, behavior trees) are preferred because their behavior can be traced and audited. End-to-end learned systems are harder to certify for safety-critical applications precisely because their decision logic is not inspectable.

Response time. Real-time requirements constrain architecture choice. Planning-heavy architectures (hierarchical task networks) may be too slow for millisecond response requirements. Reactive architectures or pre-trained neural networks can respond faster.

Available data. End-to-end learned systems require large, high-quality training datasets. If operational data is scarce, knowledge-based architectures (rules, behavior trees, HTN) may be more practical.

Hybrid approaches are common in practice. An autonomous vehicle might use end-to-end learning for perception, a behavior tree for tactical decisions, and a planner for route selection. Different parts of the product use different architectures matched to their specific requirements.

Assessment