Lecture 4: Heuristic Search on Graphs

10:30 AM, Feb 3, 2009

Contents

1 Overview 1

2 Examples 3

3 Optimality 5

4 Back to Trees 6

4.1 PathMax .......................................... 6

4.2 IDA∗ Search ......................................... 8

1 Overview

A tree is a special case of a graph in which there is exactly one path to every node. Indeed an arbitrarygraph can contain multiplepaths to anygiven node. To extendheuristic searchalgorithms designed for trees to graphs, we maintain a closed list of nodes already visited, in addition to the openlist of nodesyet tobe visited. Inbest-h search, nodes areinsertedinto the openlist as they are encountered,if they are notyet on eitherthe openorthe closedlists. In addition,inbest-g search, nodes on the open list are promoted if they are re-encountered with lower priority. In addition, in A∗ search, nodes on the closed list are re-opened if they are re-encountered with lower priority.

Best-h Search on Graphs Itis straightforward toextendbest-h search tographs: wemaintain a closedlist anddo not open any nodes that already appear on either the openlist or the closedlist. Since theheuristic h is afunction(i.e.,each node’svalueisunique),nodesareneverre-encountered with lower priority. Thus, open nodes need not ever be promoted and closed nodes need not be re-inserted into the open list. The best-h search algorithm for graphs is depicted in Table 1.

Best-g Search on Graphs The best-g search algorithm is outlined in Table 2. Like best-h search, best-g search maintains a closed list and never re-opens any nodes on this list: since nodes are visitedinpriority order, closed nodes can neverbe re-encountered withlowerpriority. But open nodes could be re-encountered with lower priority, since there can be multiple paths to a single node in a graph. Best-g search promotes nodes on the open list if ever they are re-encountered with lower priority.

A∗ Search on Graphs Table 3 describesA∗ search through arbitrarygraphs. Likebest-g search, A∗ search ongraphspromotes nodes onthe openlist whenever such nodes are re-encountered with lower priority. In addition, A∗ search on graphs maintains a closed list and re-opens nodes on this

Best-h(X,S,G,δ,c,h)

Inputs search problem
heuristic function h
Output (pathto) optimal goal node
Initialize O = S is the list of open nodes
C = ∅ is the list of closed nodes

while (O is not empty) do

  1. delete node n ∈ O s.t. h(n)is minimal
  2. if n ∈ G, return(pathto) n
    1. for all m ∈ δ(n)
      1. compute h′ (m)
      2. if m �∈ C and m �∈ O
    2. i. insert m into O with priority h(m)= h′(m)
  3. insert n into C

fail

Table 1: Best-h Search on Graphs.

Best-g(X,S,G,δ,c)

Inputs search problem
Output (pathto) optimal goal node
Initialize O = S is the list of open nodes
C = ∅ is the list of closed nodes

while (O is not empty) do

  1. delete node n ∈ O s.t. g(n)is minimal
  2. if n ∈ G, return(pathto) n
    1. for all m ∈ δ(n)
      1. g ′ (m)= g(n)+c(m,n)
        1. if m �∈ C and m �∈ O
          1. insert m into O with priority g(m)= g ′ (m) else if m ∈ O and g ′ (m)<g(m)
          2. promote m in O to priority g(m)= g ′ (m)
  3. insert n into C

fail

Table 2: Best-g Search on Graphs.

2

listif everthey are are re-encountered withlowerpriority. How canclosed nodesbere-encountered with lower priority? Recall that A∗ search is guided by the evaluation function f = g + h: i.e., nodes are visited in roughly f-order. Unlike in best-g search, where nodes are visited in g-order, a closed node in A∗ search can be re-encountered with lower a g-cost, and thus a lower f-priority than its f−priority during any previous encounters.

A∗(X,S,G,δ,c,h)

Inputs search problem
heuristic function h
Output (pathto) optimal goal node
Initialize O = S is the list of open nodes
C = ∅ is the list of closed nodes

while (O is not empty) do

  1. delete node n ∈ O s.t. f(n)is minimal
  2. if n ∈ G, return(pathto) n
    1. for all m ∈ δ(n)
      1. compute h′ (m)
      2. g ′ (m)= g(n)+c(m,n)
      3. f′(m)= g ′ (m)+h′(m)
      4. if m �∈ C and m �∈ O insert m into O with priority f(m)= f′(m) else if m ∈ O and f′(m)<f(m) promote m in O to priority f(m)= f′(m)

else if m ∈ C and f′(m)<f(m) re-open m in O with priority f(m)= f′(m)

4. insert n into C with priority f(n)

fail

Table 3: A∗ Search on Graphs. Note that f′(m)<f(m)iff g ′(m)<g(m), sinceh(m)is constant, for all nodes n.

Examples

A sample graph appears in Figure 1. The start state is S and the uniquegoal nodeis G, but there are two paths to this goal node: SACG and SBCG.

Best-h Search Best-h search maintains the following sequence of priority queues in its search through this graph: S0, A1B2 , C0 B2, G0 B2, goal! The path returned is suboptimal: SACG.

Figure 1: Sample DAG: edges are labeled with costs g; nodes are labeled with heuristic values h. The start state is S and the uniquegoal nodeis G. The optimal path to this goal is SBCG. Only best-g search and A∗ search return this optimal path; best-h search returns the suboptimal path SACG.

Best-g Search Best-g search maintains the following sequence of priority queues in its search through this graph: S0, A1B5 , B5 C21 , C10 , G10 , goal! The path returned is optimal: SBCG.

Best-g search first encounters node C via node A,traversing apath of cost21. Later,it encounters node C again via node B, traversing a path of cost only 10. At this point, node C’s priority is promoted from 21 to 10.

A∗ Search A∗ search maintains the following sequence of priority queues in its search through the graph in Figure 1: S0, A2 B7, B7 C21 , C10 , G10 , goal! The path to the goal that is returned is optimal: SBCG.

Figure 2(LHS):

  • Best-g: S0, D1 A1, A1G11 , B2G11 , C3G11 , G11 , goal! (After C3 ispopped, G13 is notpushed, since G11 is already on the queue.) The optimal path SDG is returned.
  • Best-h: S0 , A3 D10 , B2D10 , C1D10 , G0 D10 , goal! The path returned, namely SABCG, is not optimal.
  • A∗ : S0, A4D11 , B4 D11 , C4 D11 , D11 G13 , G11 , goal! (After D11 is popped, the node G13 is promoted to G11 .) The optimal path SDG is returned.

Figure 2(RHS):

h=10

h=1

h=0

Figure 2: Sample DAGs: edges are labeled with costs g; nodes are labeled with heuristic values h. The start state is S and the uniquegoalis G.(LHS)The optimalpath tothegoalis SDG. (RHS) The optimal path to the goal is SDCG.

  • Best-g: S0, D1 A1, C1A1 , A1G11 , B2G11 , C3 G11 , G11 , goal! (After C3 is popped, G13 is not pushed, since G11 is already on the queue.) The optimal path SDCG is returned.
  • Best-h: S0 , A3 D10 , B2D10 , C1D10 , G0 D10 , goal! The path returned, namely SABCG, is not optimal.
  • A∗ : S0, A4D11 , B4 D11 , C4D11 , D11 G13 , C2 G13 , G11 , goal! (After C with cost 4 is closed, it is re-opened with cost 2. And after C2 is popped, the node G13 is promoted to G11 .) The optimal path SDCG is returned.

Optimality

Theproof of optimality ofbest-g search relies on the fact that nodes are visited in g-order(that is, in order of nondecreasing g-costs). Hence,the firstgoal nodethatbest-g reaches is necessarily one of minimal g-cost. A∗ search, on the other hand, need not visit nodes in f-order(thatis,in order of nondecreasing f-costs). (See Figure 3.)

Thus, the first goal node that A∗ reaches is not obviously one of minimal f-cost. A property called consistency, however, ensures that f is a monotonically, nondecreasing function of depth: i.e., f(m)≥ f(n), for all m ∈ δ(n), sothatA∗ search does visit nodes in f-order. Consequently, assuming consistency, the first goal node that is reached is one of minimal f-cost.

Aheuristicfunctionhis called consistent iff(i) h(n)≤ c(n,m)+h(m),for allnodes n with successor

∗

nodes m ∈ δ(n); and(ii) h(n ∗)= 0, for all goal nodes n . The first part of this definition can be

1 100

S A G

h=10 & f=10 h=0 & f=1 h=0 & f=101

Figure 3: Simple search tree: S is the start node; A is an intermediate node; G is the goal node. Edges are labeled with costs c. Nodes are labelled with h-costs and f-costs. Note that f is not monotonically nondecreasing in depth, but that A∗ search visits nodesin order ofdepth. Thus,A∗ search does not visit nodes in f-order.

understood as a form of the triangle inequality, which stipulates that the length of a side of a triangle cannot exceed the sum of the lengths of the two other sides.

Lemma: If h is consistent, then f is monotonically, nondecreasing in depth: i.e., for all n ∈ X, for all m ∈ δ(n),f(m)≥ f(n).

Proof: Note that g (andtherefore f)is not a function. For each value of g(n),

f(n)= g(n)+h(n) = g(m)−c(n,m)+h(n) ≤ g(m)+h(m) = f(m)

⋄

This lemma implies that A∗ search with a consistent heuristic is optimal: since nodes are visited in nondecreasing order of their f-costs, the first goal node visited will be one of minimal f-cost. Since the h-value at goal nodes is 0, this node will also be a goal of minimal g-cost.

Theorem: A∗ search(ontrees andgraphs) with a consistentheuristicis optimal.

4 Back to Trees

Consistency implies admissibility, but the converse is not true. Still, most admissible heuristics thatareusefulinpracticehappentoalsobeconsistent. Thatsaid,ifyouever findyourselftrying to solve a search problem on a tree and your heuristic is not consistent, it can be made consistent using thepathmaxoptimization routine. Doing sowouldguarantee monotonicity, sothatA∗ would visit nodes in nondecreasing order according to their f-costs.

4.1 PathMax

The pathmax equation is an optimization routine applicable to heuristic search through a tree in which the heuristic h is inductively updated to yield a new heuristic hˆ: beginning at the root node n, hˆ(n)= h(n); then, for all successor nodes m ∈ δ(n), hˆ(m)= max{hˆ(n)−c(n,m),h(m)}. Equivalently, by adding g(m)to both ˆh(n)−c(n,m)and h(m),fˆ(m)= max{fˆ(n),f(m)}. (Check: hˆ(n)− c(n,m)+ g(m)= hˆ(n)+ g(n)= fˆ(n).) For example, if g(n) =1 and h(n) = 10, while g(m) =3 and h(m) = 3 for some child m of n, then n’s f-cost 11, while m’s f-cost is 6. The

A∗(X,S,G,δ,c,h)

Inputs search problem
heuristic function h
Output (pathto) optimal goal node
Initialize O = S is the list of open nodes

C = ∅ is the list of closed nodes

while (O is not empty) do

  1. delete node n ∈ O s.t. f(n)is minimal
  2. if n ∈ G, return(pathto) n
    1. for all m ∈ δ(n)
      1. compute h′ (m)
      2. g ′ (m)= g(n)+c(m,n)
      3. f′(m)= g ′ (m)+h′(m)
      4. if m �∈ C and m �∈ O insert m into O with priority f(m)= f′(m)

else if m ∈ O and f′(m)<f(m) promote m in O to priority f(m)= f′(m)

4. insert n into C

fail

Table 4: A∗ Search on Graphs with a Consistent Heuristic.

pathmax equation increases m’s f-cost to 11 because m is on the same path to a goal as n, so m’s

costs cannot be any less than n’s. Lemma: Applying the pathmax operation beginning at the root of a tree and working down towards the leaves yields a consistent heuristic: i.e., for all n,m ∈ δ(n),hˆ(n)≤ hˆ(m)+c(n,m).

Proof: Two cases arises. Either hˆ(m)= h(m)≥ hˆ(n)−c(n,m), orhˆ(m)= hˆ(n)−c(n,m). ⋄

Corollary: Pathmaximplies monotonicity(i.e.,A∗ visits nodes in nondecreasing order according to their f-costs). Lemma: The pathmax operation preserves admissibility: i.e., if h is admissible, then hˆ(m)=

max{h(m),h(n)−c(n,m)}, for all n and for all m ∈ δ(n)is also admissible. Proof: Fix an arbitrary n ∈ X. To show hˆ(m) ≤ h∗(m), for all m ∈ δ(n), it suffices to show:

(i) h(m) ≤ h∗(m), and(ii) h(n)−c(n,m) ≤ h∗(m). Condition(i) followsimmediatelyfromthe admissibility of h. Condition(ii) breaksdownintotwo cases: h(n)−c(n,m)≤ h∗(n)−c(n,m)= h∗(m), ifm is on a path from n to its nearest goal; otherwise, h(n)−c(n,m)≤ h∗(n)−c(n,m)< h∗(m). Therefore, h(n)−c(n,m)≤ h∗(m).

Theorem: A∗ search with pathmax is optimal.

4.2 IDA∗ Search

Iterative deepening A∗ (IDA∗) is a search algorithm with the performance properties of A∗—it is complete and optimal—and the space requirements of DFS—(essentially) linear in depth. The mainidea ofiterativedeepeningA∗ (oriterative deepeningBest-g orBest-h)is to repeatedlysearch in depth-first fashion, over subgraphs with f-cost(or g-cost or h-cost) less than α, less than 2α, less than 3α, and so on, until agoalisfound, where α is a lower bound on the cost between nodes and their successors: i.e.,0 <α ≤ c(n,m), for alln,m ∈ δ(n). IDA∗ is usually implemented with pathmax, so that the algorithm visits nodes in contours of greater and greater f-costs.

Recall that the space complexity of ID is O(bd), whered is the depth of the goal node. Similarly,

∗

the space complexity of IDA∗ is O(bg∗/α), whereg is the optimal cost. The time complexity of IDA∗, however, can exceed that of A∗ . In particular, in search spaces where the f-cost is different at every state, only one additional state is expanded during each iteration. In such a search space, if A∗ expands N nodes, IDA∗ expands 1+ ... + N = O(N2 ) nodes. The typical solution to this problem is to fix an increment β >α such that several nodes n have cost fi <f(n) ≤ fi + β, where fi is the ith incremental value of the f-cost. This strategy reduces search time, since the total number of iterations is proportional to 1/β < 1/α, and returns solutions that are at worst β-optimal: i.e., if the algorithm returns m ∗, then g(m ∗)<g ∗ + β.

IDA∗(X,S,G,β,c,h)

Inputs search problem
admissible heuristic h
Output (pathto) optimal goal node
Initialize β is the increment
i = 0 is the cut-off f-value
O = S is the priority queue of open nodes

while (1)do

1. while (O is not empty) do

(a)
delete first node n ∈ O
(b)
if n ∈ G, return(pathto) goal n
(c)
for all m ∈ β(n)

i. compute h(m)

ii. g(m)= g(n)+c(m,n)

iii. f(m)= g(m)+h(m)

iv. if f(m)≤ i, insert m in front of O

2. increment i by β, O = S

Table 5: Iterative Deepening A∗ .