第2章:智能体(Intelligent Agents)
一、导读
1.1 本章学习目标
第2章引入了贯穿全书的核心概念——智能体(Agent)。如果说第1章回答了"什么是AI"这个哲学问题,那么第2章则回答了"如何设计AI系统"这个工程问题。智能体是AI系统的基本抽象模型:一个能感知环境并采取行动以最大化目标实现的实体。理解智能体的设计原则,是构建任何AI系统的基础。
本章的核心学习目标包括:
- 掌握智能体的基本模型:理解智能体通过传感器感知环境、通过执行器作用于环境的基本框架,以及这种抽象如何统一各种AI系统。
- 理解"理性"的精确含义:理性不是"完美",而是在给定信息和约束下做出期望效用最大化的决策。理解理性与最优、满意之间的区别。
- 掌握 PEAS 描述框架:学会用性能度量(Performance)、环境(Environment)、执行器(Actuators)、传感器(Sensors)来精确描述一个AI任务。
- 理解环境的分类维度:完全可观察/部分可观察、确定性/随机性、静态/动态、离散/连续、单智能体/多智能体,以及这些维度如何影响算法选择。
- 掌握四种智能体程序结构:简单反射智能体、基于模型的反射智能体、基于目标的智能体、基于效用的智能体,以及它们的适用场景。
- 理解学习在智能体中的作用:学习智能体如何通过经验改进行为,以及学习组件的各个组成部分。
1.2 为什么第二章如此重要
智能体概念是本书的组织原则。全书的每一个算法——搜索、规划、推理、学习——都可以理解为构建理性智能体的不同组件。没有智能体的框架,这些算法就是孤立的工具;有了智能体的框架,它们形成了一个统一的设计体系。
更重要的是,智能体框架提供了一种系统化的设计方法论。当你面对一个AI问题时,PEAS框架帮助你精确定义问题,环境分类帮助你选择合适的算法,智能体结构帮助你组织代码。这种从抽象到具体的设计能力,是AI工程师的核心素养。
1.3 核心问题引导
在阅读本章时,请带着以下问题思考:
理性是否等于最优? 在计算资源有限的情况下,一个"足够好"的决策是否比"最优但计算不完"的决策更理性?
为什么部分可观察的环境如此重要? 现实世界中几乎没有完全可观察的任务,这对AI系统的设计有什么根本性影响?
简单反射智能体有什么致命缺陷? 在什么情况下,基于条件的动作规则不足以产生理性行为?
效用函数如何影响智能体的行为? 设计不当的效用函数会导致什么意外行为?
学习智能体与固定智能体的本质区别是什么? 在动态变化的环境中,学习能力是否不可或缺?
二、核心概念详解
2.1 智能体与环境
2.1.1 智能体的定义
智能体是任何能够通过传感器感知其环境并采取行动以影响该环境的实体。这个定义非常宽泛——一个恒温器是智能体,一只狗是智能体,一个自动驾驶汽车也是智能体。
关键概念区分:
- 智能体(Agent):做出决策的实体
- 环境(Environment):智能体所处的外部世界
- 感知(Percept):智能体在某一时刻从环境中接收到的信息
- 感知序列(Percept Sequence):智能体迄今为止接收到的所有感知的完整历史
class Agent:
def __init__(self):
self.percept_history = []
def act(self, percept):
self.percept_history.append(percept)
action = self.program(percept)
return action
def program(self, percept):
raise NotImplementedError
class Environment:
def __init__(self, agent):
self.agent = agent
self.state = None
def generate_percept(self):
raise NotImplementedError
def apply_action(self, action):
raise NotImplementedError
def step(self):
percept = self.generate_percept()
action = self.agent.act(percept)
self.apply_action(action)
return percept, action
def run(self, steps=10):
history = []
for _ in range(steps):
percept, action = self.step()
history.append((percept, action))
return history2.1.2 智能体函数与智能体程序
智能体函数是一个数学抽象:它描述了从感知序列到行动的映射关系 f: P* → A。这个函数完整地刻画了智能体的行为。
智能体程序是智能体函数的具体实现:它在某种物理架构上运行,将当前的感知转化为行动。智能体函数是抽象的,智能体程序是具体的。
# 智能体函数:抽象的映射
def agent_function(percept_sequence):
if percept_sequence[-1] == 'dirty':
return 'suck'
elif percept_sequence[-1] == 'A':
return 'right'
else:
return 'left'
# 智能体程序:具体的实现
class SimpleAgentProgram:
def __init__(self):
self.percepts = []
def __call__(self, percept):
self.percepts.append(percept)
location, status = percept
if status == 'dirty':
return 'suck'
if location == 'A':
return 'right'
return 'left'2.2 理性(Rationality)
2.2.1 理性的精确定义
在本书的框架中,理性有精确的含义:对于每个可能的感知序列,理性智能体选择的行动应使其性能度量(performance measure)的期望值最大化。
关键要点:
- 理性关注的是期望性能,而不是实际性能。在不确定环境中,理性决策可能产生坏结果,但这不意味着决策本身不理性。
- 理性依赖于感知序列。智能体只能基于它已经感知到的信息做出决策,不能利用它无法获得的信息。
- 理性要求自主性:智能体应该通过自己的感知来学习,而不是依赖设计者预先植入的知识。
import random
class RationalityDemo:
def __init__(self):
self.performance_history = []
def performance_measure(self, state):
return state.get('reward', 0)
def expected_utility(self, action, belief_state):
total = 0.0
for state, prob in belief_state.items():
outcome = self.simulate(action, state)
total += prob * self.performance_measure(outcome)
return total
def simulate(self, action, state):
new_state = state.copy()
if action == 'invest_high':
new_state['reward'] = random.choice([100, -50])
elif action == 'invest_low':
new_state['reward'] = random.choice([30, 10])
return new_state
def choose_rational_action(self, belief_state, actions):
best_action = None
best_eu = float('-inf')
for action in actions:
eu = self.expected_utility(action, belief_state)
if eu > best_eu:
best_eu = eu
best_action = action
return best_action, best_eu
demo = RationalityDemo()
belief = {'state1': 0.6, 'state2': 0.4}
action, eu = demo.choose_rational_action(belief, ['invest_high', 'invest_low'])
print(f"理性选择: {action}, 期望效用: {eu:.2f}")2.2.2 完美理性 vs 有限理性
完美理性假设智能体有无限的计算能力,能够精确计算每个行动的期望效用。但在现实中,计算资源是有限的,这就是 Herbert Simon 提出的有限理性(Bounded Rationality)概念。
有限理性智能体不追求最优,而是追求满意(Satisficing)——找到足够好的解决方案。这在实践中往往比追求完美理性更实际。
class BoundedRationalAgent:
def __init__(self, max_computation_steps=100, aspiration_level=50):
self.max_computation_steps = max_computation_steps
self.aspiration_level = aspiration_level
def decide(self, actions, belief_state):
evaluated = 0
for action in actions:
utility = self.evaluate(action, belief_state)
evaluated += 1
if utility >= self.aspiration_level:
return action, evaluated
if evaluated >= self.max_computation_steps:
break
return self.best_so_far(actions, belief_state), evaluated
def evaluate(self, action, belief_state):
return sum(prob * self.utility(state)
for state, prob in belief_state.items())
def utility(self, state):
return state.get('value', 0)
def best_so_far(self, actions, belief_state):
return max(actions, key=lambda a: self.evaluate(a, belief_state))2.3 PEAS 描述框架
PEAS 是描述智能体任务环境的标准框架:
- P(Performance Measure):性能度量——如何评价智能体的成功?
- E(Environment):环境——智能体在什么环境中运行?
- A(Actuators):执行器——智能体能做什么?
- S(Sensors):传感器——智能体能感知什么?
class PEASDescription:
def __init__(self, name, performance, environment, actuators, sensors):
self.name = name
self.performance = performance
self.environment = environment
self.actuators = actuators
self.sensors = sensors
def describe(self):
print(f"=== {self.name} ===")
print(f"性能度量: {', '.join(self.performance)}")
print(f"环境: {', '.join(self.environment)}")
print(f"执行器: {', '.join(self.actuators)}")
print(f"传感器: {', '.join(self.sensors)}")
# 自动驾驶汽车的 PEAS 描述
self_driving = PEASDescription(
"自动驾驶汽车",
performance=["安全性", "到达目的地", "合法性", "舒适性", "燃油效率"],
environment=["道路", "其他车辆", "行人", "交通信号", "天气"],
actuators=["方向盘", "油门", "刹车", "转向灯", "喇叭"],
sensors=["摄像头", "激光雷达", "GPS", "速度计", "加速度计"]
)
self_driving.describe()
# 医疗诊断系统的 PEAS 描述
medical_diagnosis = PEASDescription(
"医疗诊断系统",
performance=["诊断准确率", "患者健康", "治疗成本", "法律风险"],
environment=["患者", "医院", "医疗设备", "医学知识库"],
actuators=["屏幕显示", "打印报告", "推荐治疗方案"],
sensors=["患者输入", "医疗记录", "检验结果", "影像数据"]
)
medical_diagnosis.describe()2.4 环境的分类
2.4.1 完全可观察 vs 部分可观察
如果智能体的传感器在每一时刻都能提供与环境状态相关的全部信息,环境就是完全可观察的。否则是部分可观察的。
注意:环境完全可观察不意味着智能体完全可观察——如果传感器有噪声,即使环境状态完全可知,智能体也只能获得部分信息。
2.4.2 确定性 vs 随机性
如果环境的下一个状态完全由当前状态和智能体的行动决定,环境就是确定性的。否则是随机性的。
2.4.3 静态 vs 动态
如果环境在智能体"思考"时保持不变,环境是静态的。否则是动态的。
2.4.4 离散 vs 连续
如果环境的状态空间、感知空间和行动空间都是有限的、可数的,环境是离散的。否则是连续的。
# 环境属性分析器
class EnvironmentAnalyzer:
def __init__(self):
self.environments = {}
def register(self, name, observable, deterministic, static, discrete, agents):
self.environments[name] = {
'observable': observable,
'deterministic': deterministic,
'static': static,
'discrete': discrete,
'agents': agents
}
def analyze(self, name):
env = self.environments[name]
complexity = 0
if env['observable'] == 'partial': complexity += 2
if env['deterministic'] == 'stochastic': complexity += 2
if env['static'] == 'dynamic': complexity += 1
if env['discrete'] == 'continuous': complexity += 1
if env['agents'] == 'multi': complexity += 2
return complexity
def recommend_approach(self, name):
env = self.environments[name]
recommendations = []
if env['observable'] == 'partial':
recommendations.append("需要状态估计(如卡尔曼滤波、粒子滤波)")
if env['deterministic'] == 'stochastic':
recommendations.append("需要概率推理(如贝叶斯网络)")
if env['static'] == 'dynamic':
recommendations.append("需要实时决策能力")
if env['discrete'] == 'continuous':
recommendations.append("需要连续优化方法")
if env['agents'] == 'multi':
recommendations.append("需要考虑多智能体交互(博弈论)")
return recommendations
analyzer = EnvironmentAnalyzer()
analyzer.register("国际象棋", "fully", "deterministic", "semi-dynamic", "discrete", "multi")
analyzer.register("自动驾驶", "partial", "stochastic", "dynamic", "continuous", "multi")
analyzer.register("数独", "fully", "deterministic", "static", "discrete", "single")
for name in ["国际象棋", "自动驾驶", "数独"]:
print(f"\n{name}:")
print(f" 复杂度评分: {analyzer.analyze(name)}")
print(f" 推荐方法: {analyzer.recommend_approach(name)}")2.5 智能体程序的四种类型
2.5.1 简单反射智能体
简单反射智能体根据当前感知直接选择行动,忽略感知历史。它使用条件-动作规则(if-then 规则)来工作。
class SimpleReflexAgent:
def __init__(self, rules):
self.rules = rules
def interpret_input(self, percept):
return percept
def match_rule(self, state):
for condition, action in self.rules:
if condition(state):
return action
return None
def act(self, percept):
state = self.interpret_input(percept)
return self.match_rule(state)
# 真空清洁的简单反射智能体
vacuum_rules = [
(lambda s: s['status'] == 'dirty', 'suck'),
(lambda s: s['location'] == 'A', 'right'),
(lambda s: s['location'] == 'B', 'left'),
]
simple_agent = SimpleReflexAgent(vacuum_rules)
print(simple_agent.act({'location': 'A', 'status': 'dirty'}))局限性:简单反射智能体只能在完全可观察的环境中工作。如果环境部分可观察,它会产生无限循环或次优行为。
2.5.2 基于模型的反射智能体
基于模型的反射智能体维护一个内部状态来跟踪环境的当前状态。它使用转换模型来更新状态。
class ModelBasedReflexAgent:
def __init__(self, rules, transition_model, sensor_model):
self.rules = rules
self.transition_model = transition_model
self.sensor_model = sensor_model
self.state = None
def update_state(self, state, action, percept, transition_model, sensor_model):
new_state = transition_model(state, action)
observation_info = sensor_model(new_state, percept)
return observation_info
def act(self, percept):
self.state = self.update_state(
self.state, self.last_action, percept,
self.transition_model, self.sensor_model
)
for condition, action in self.rules:
if condition(self.state):
self.last_action = action
return action
self.last_action = 'noop'
return 'noop'
# 带有内部状态的真空清洁智能体
def vacuum_transition(state, action):
new_state = state.copy() if state else {'A': 'unknown', 'B': 'unknown'}
if action == 'suck':
new_state[state.get('location', 'A')] = 'clean'
return new_state
def vacuum_sensor(state, percept):
location, status = percept
updated = state.copy() if state else {}
updated[location] = status
return updated2.5.3 基于目标的智能体
基于目标的智能体不仅跟踪当前状态,还考虑未来会怎样。它通过搜索或规划来找到能达到目标的行动序列。
class GoalBasedAgent:
def __init__(self, goal_test, actions_fn, transition_fn):
self.goal_test = goal_test
self.actions_fn = actions_fn
self.transition_fn = transition_fn
def search_plan(self, initial_state):
from collections import deque
frontier = deque([(initial_state, [])])
visited = set()
while frontier:
state, plan = frontier.popleft()
state_key = str(state)
if state_key in visited:
continue
visited.add(state_key)
if self.goal_test(state):
return plan
for action in self.actions_fn(state):
next_state = self.transition_fn(state, action)
frontier.append((next_state, plan + [action]))
return None
# 八数码问题的目标智能体
def puzzle_goal(state):
return state == (1, 2, 3, 4, 5, 6, 7, 8, 0)
def puzzle_actions(state):
blank = state.index(0)
row, col = blank // 3, blank % 3
actions = []
if row > 0: actions.append('up')
if row < 2: actions.append('down')
if col > 0: actions.append('left')
if col < 2: actions.append('right')
return actions
def puzzle_transition(state, action):
state = list(state)
blank = state.index(0)
row, col = blank // 3, blank % 3
if action == 'up':
swap = (row - 1) * 3 + col
elif action == 'down':
swap = (row + 1) * 3 + col
elif action == 'left':
swap = row * 3 + (col - 1)
else:
swap = row * 3 + (col + 1)
state[blank], state[swap] = state[swap], state[blank]
return tuple(state)
agent = GoalBasedAgent(puzzle_goal, puzzle_actions, puzzle_transition)
initial = (1, 2, 3, 4, 0, 6, 7, 5, 8)
plan = agent.search_plan(initial)
print(f"求解计划: {plan}")2.5.4 基于效用的智能体
基于效用的智能体不仅追求达到目标,还追求最大化效用。它使用效用函数来评估不同状态的"好"程度。
class UtilityBasedAgent:
def __init__(self, utility_fn, transition_fn, actions_fn, discount=0.9):
self.utility_fn = utility_fn
self.transition_fn = transition_fn
self.actions_fn = actions_fn
self.discount = discount
def value_iteration(self, states, max_iter=100, tolerance=0.01):
values = {s: 0 for s in states}
for _ in range(max_iter):
new_values = {}
delta = 0
for state in states:
if not self.actions_fn(state):
new_values[state] = self.utility_fn(state)
continue
best = float('-inf')
for action in self.actions_fn(state):
next_state = self.transition_fn(state, action)
value = self.utility_fn(state) + self.discount * values.get(next_state, 0)
best = max(best, value)
new_values[state] = best
delta = max(delta, abs(new_values[state] - values[state]))
values = new_values
if delta < tolerance:
break
return values
def best_action(self, state, values):
best_action = None
best_value = float('-inf')
for action in self.actions_fn(state):
next_state = self.transition_fn(state, action)
value = values.get(next_state, 0)
if value > best_value:
best_value = value
best_action = action
return best_action
# 网格世界的效用智能体
def grid_utility(state):
x, y = state
if (x, y) == (3, 3): return 1.0
if (x, y) == (2, 3): return -1.0
return -0.04
def grid_actions(state):
x, y = state
actions = []
if y > 0: actions.append('up')
if y < 3: actions.append('down')
if x > 0: actions.append('left')
if x < 3: actions.append('right')
return actions
def grid_transition(state, action):
x, y = state
if action == 'up': return (x, y - 1)
if action == 'down': return (x, y + 1)
if action == 'left': return (x - 1, y)
return (x + 1, y)
states = [(x, y) for x in range(4) for y in range(4)]
agent = UtilityBasedAgent(grid_utility, grid_transition, grid_actions)
values = agent.value_iteration(states)
print(f"从(0,0)出发的最佳行动: {agent.best_action((0, 0), values)}")2.6 学习智能体
学习智能体是最高级的智能体类型。它不仅能执行任务,还能通过经验改进自己的性能。
class LearningAgent:
def __init__(self):
self.critic = None
self.learning_element = None
self.problem_generator = None
self.performance_element = None
def perceive(self, percept):
feedback = percept.get('feedback', None)
if self.critic and feedback:
self.learning_element.update(feedback)
def act(self):
return self.performance_element.choose_action()
class SimpleLearningAgent:
def __init__(self, initial_policy=None):
self.policy = initial_policy or {}
self.experience = []
self.learning_rate = 0.1
def act(self, percept):
state = percept.get('state')
if state in self.policy:
return self.policy[state]
return 'explore'
def update(self, percept, action, reward, next_state):
self.experience.append((percept, action, reward, next_state))
state = percept.get('state')
if state not in self.policy:
self.policy[state] = action
else:
old_value = self.policy.get(state, 0)
if isinstance(old_value, str):
old_value = 0
self.policy[state] = old_value + self.learning_rate * (reward - old_value)
def get_statistics(self):
total_reward = sum(r for _, _, r, _ in self.experience)
return {
'total_experiences': len(self.experience),
'total_reward': total_reward,
'avg_reward': total_reward / max(1, len(self.experience))
}
# 简单的强化学习智能体
agent = SimpleLearningAgent()
import random
random.seed(42)
state = 'A'
for step in range(20):
percept = {'state': state}
action = agent.act(percept)
if action == 'explore':
action = random.choice(['left', 'right'])
reward = random.choice([-1, 0, 1, 2])
next_state = random.choice(['A', 'B', 'C'])
agent.update(percept, action, reward, next_state)
state = next_state
print(f"学习统计: {agent.get_statistics()}")
print(f"学到的策略: {agent.policy}")三、实战案例
3.1 案例一:完整的智能体模拟系统
下面构建一个完整的智能体模拟系统,比较不同类型智能体在真空清洁环境中的性能。
import random
class VacuumEnvironment:
def __init__(self, n_rooms=4):
self.rooms = {i: random.choice(['clean', 'dirty']) for i in range(n_rooms)}
self.agent_pos = 0
self.steps = 0
self.max_steps = 50
self.total_reward = 0
def get_percept(self):
return (self.agent_pos, self.rooms[self.agent_pos])
def apply_action(self, action):
self.steps += 1
reward = -1
if action == 'suck' and self.rooms[self.agent_pos] == 'dirty':
self.rooms[self.agent_pos] = 'clean'
reward = 10
elif action == 'left' and self.agent_pos > 0:
self.agent_pos -= 1
elif action == 'right' and self.agent_pos < len(self.rooms) - 1:
self.agent_pos += 1
self.total_reward += reward
return reward
def is_done(self):
return self.steps >= self.max_steps or all(
s == 'clean' for s in self.rooms.values()
)
class ReflexVacuumAgent:
def __init__(self):
self.visited = set()
def act(self, percept):
pos, status = percept
self.visited.add(pos)
if status == 'dirty':
return 'suck'
if pos < 3:
return 'right'
return 'left'
class ModelBasedVacuumAgent:
def __init__(self, n_rooms=4):
self.room_status = {i: 'unknown' for i in range(n_rooms)}
self.n_rooms = n_rooms
def act(self, percept):
pos, status = percept
self.room_status[pos] = status
if status == 'dirty':
return 'suck'
for i in range(self.n_rooms):
if self.room_status[i] == 'dirty':
if i > pos:
return 'right'
return 'left'
for i in range(self.n_rooms):
if self.room_status[i] == 'unknown':
if i > pos:
return 'right'
return 'left'
return 'left'
def simulate(agent_class, n_trials=100, n_rooms=4):
total_rewards = []
for _ in range(n_trials):
env = VacuumEnvironment(n_rooms)
agent = agent_class()
while not env.is_done():
percept = env.get_percept()
action = agent.act(percept)
env.apply_action(action)
total_rewards.append(env.total_reward)
return sum(total_rewards) / len(total_rewards)
random.seed(42)
reflex_avg = simulate(ReflexVacuumAgent)
model_avg = simulate(ModelBasedVacuumAgent)
print(f"简单反射智能体平均奖励: {reflex_avg:.1f}")
print(f"基于模型智能体平均奖励: {model_avg:.1f}")3.2 案例二:多智能体环境中的博弈
import random
class PrisonersDilemma:
def __init__(self, rounds=100):
self.rounds = rounds
self.payoffs = {
('cooperate', 'cooperate'): (3, 3),
('cooperate', 'defect'): (0, 5),
('defect', 'cooperate'): (5, 0),
('defect', 'defect'): (1, 1),
}
def play(self, agent1, agent2):
history1, history2 = [], []
score1, score2 = 0, 0
for _ in range(self.rounds):
a1 = agent1.decide(history1, history2)
a2 = agent2.decide(history2, history1)
p1, p2 = self.payoffs[(a1, a2)]
score1 += p1
score2 += p2
history1.append((a1, a2))
history2.append((a2, a1))
return score1, score2
class TitForTatAgent:
def __init__(self):
self.first_move = 'cooperate'
def decide(self, my_history, opp_history):
if not opp_history:
return self.first_move
return opp_history[-1][0]
class AlwaysDefectAgent:
def decide(self, my_history, opp_history):
return 'defect'
class RandomAgent:
def decide(self, my_history, opp_history):
return random.choice(['cooperate', 'defect'])
game = PrisonersDilemma(rounds=100)
agents = {
'Tit-for-Tat': TitForTatAgent(),
'Always Defect': AlwaysDefectAgent(),
'Random': RandomAgent(),
}
for name1, a1 in agents.items():
for name2, a2 in agents.items():
if name1 < name2:
s1, s2 = game.play(a1, a2)
print(f"{name1} vs {name2}: {s1} - {s2}")四、本章小结
本章建立了智能体的完整框架,这是理解全书后续内容的基础。我们从智能体的基本定义出发,引入了理性、PEAS描述、环境分类等核心概念,然后详细讲解了四种智能体程序结构。
核心知识点回顾:
- 智能体通过传感器感知环境,通过执行器作用于环境
- 理性意味着在给定信息下最大化期望性能度量
- PEAS框架是描述AI任务的标准方法
- 环境属性(可观察性、确定性、静态性、离散性)决定算法选择
- 四种智能体结构从简单到复杂,适用于不同场景
- 学习智能体通过经验改进行为,是最高级的智能体类型
五、思考与练习
思考题
性能度量的设计:为以下系统设计 PEAS 描述,并讨论性能度量中可能存在的冲突:(a) 自动翻译系统,(b) 智能推荐系统,(c) 无人机配送系统。
有限理性的权衡:考虑一个实时交易系统,它需要在毫秒内做出决策。在这种情况下,完美理性和有限理性之间的权衡是什么?设计一个能够在时间约束下做出"足够好"决策的智能体。
多智能体的挑战:当环境中存在多个智能体时,环境从单智能体变为多智能体。这对环境的确定性、可观察性有什么影响?博弈论如何帮助我们设计多智能体系统?
编程练习
实现基于效用的真空清洁智能体:为 4 房间的真空清洁环境实现一个基于效用函数的智能体,比较它与简单反射智能体和基于模型智能体的性能。
实现学习智能体:在囚徒困境博弈中实现一个 Q-learning 智能体,让它与 Tit-for-Tat 和 Always-Defect 智能体对战,观察学习过程。
环境分类器:编写一个程序,给定一个AI任务的描述(如"下棋"、"自动驾驶"、"垃圾邮件分类"),自动分析其环境属性并推荐合适的智能体结构。