1 Overview

Our next series of lectures is concerned with search algorithms for solving constraint satisfaction andconstraintoptimizationproblems. Constraintsatisfactionproblems(e.g., n-queens and map coloring) are problems in which the search is for a feasible solution, that is, one which satisfies all theconstraints. Optimizationproblems(e.g.,traveling salespersonproblem) areproblemsinwhich the search is for an optimal solution. Constraint optimization problems are optimization problems in which the search for an optimal solution is confined to the set of feasible solutions.

Specifically, we study:

1. satisfiability: systematic search algorithms, namely Davis–Putnam and Davis, Putnam, Logemann, and Loveland, and local search algorithms, namely GSAT, and WALKSAT
2. constraintsatisfactionproblems(CSPs):backtracking,variableand valueorderingheuristics, consistency techniques, and constraint propagation
3. mathematical programming: linear and integer linear programming

By definition, constraint satisfaction involves “hard” constraints—constraints that either pass or fail, with no middle ground. Nonetheless, constraint satisfaction problems are often solved as optimization problems with “soft” constraints. Here, the objective is to maximize the number of constraints satisfied; or equivalently, to minimize the number of constraints violated.

Ontheotherhand,optimizationproblemscanbesolved using routinesdesigned tosolve constraint satisfaction problems. In a maximization (minimization) problem, add the constraint that the value of the solution must be greater (less) than k; solve; if a feasible solution exists, increment (decrement) k by ǫ> 0 and repeat; otherwise, return k. The procedure returns an ǫ-optimal solution: i.e., one that is within ǫ of the optimal.

Thetopic of thislectureisconstraint satisfactionproblems(CSPs). Examples ofCSPs,inaddition to the n-queens problem and map coloring, include cryptoarithmetic puzzles and crossword puzzles. In Lectures 3 and 4, we discussed ways to extend blind search methods with domain-specific heuristics to yield heuristic search techniques. In this lecture, we show how blind search methods can be extended with CSP-specific heuristics.

3.1 Variable Ordering

If abranch ofbacktracking searchisdestined tofail,itispreferableforittofail soonerratherthan later. Hence, asensibleway toordervariablesisfrommost-constrained toleast-constrained. There are two popular heuristics that achieve this ordering. The first is the degree heuristic, which simply counts up the number of constraints that each variable is a part of, and chooses one for which this count is maximal. The second is the minimum remaining values (MRV) heuristic, which counts up the size of each variable’s domain during each recursive call, and chooses one for which this count is minimal. This heuristic is also called fail first because failure is detected immediately if ever there is a variable whose domain becomes empty.

TheMRV(orfail-first) heuristicis often usedin conjunction with forward checking. With each variable assignment, forward checkingdeletes anyinconsistent valuesin thedomains of neighboring variables in the constraint graph. A value vjk for variable xj is inconsistent with a value vik for variable xi if assigning vjk to xj and vik to xi violates a constraint. A subroutine to check for and then remove any inconsistencies is shown in Table 3. Note that this subroutine is asymmetric: it assumes that variable xi is assigned a value and checksfor and then removes anyinconsistent values in the domain of variable xj . Forward checking is called in between steps 1 and 2 in the recursive backtracking pseudocode.

3.2 Value Ordering

Once a variable is chosen, the next obvious question is: what value should it be assigned? That is, in what order should valuesbe assigned to variables? A solution to aCSPis morelikely tobefound if the domains of subsequent variables are bigger rather than smaller. Hence, it is reasonable to BackTracking(X,D,C) Inputs CSP Output satisfying assignment or failure

RecursiveBT(X,D,C,{})

RecursiveBT(X,D,C,A)

Inputs	CSP
	assignment A
Output	satisfying assignment or failure

1. if A is complete, return A
2. select unassigned variable xi
3. order domain values Di
4. for all vik ∈ Di

(a) if vik is consistent with A according to C

i. add {xi → vik}to A

ii. Z = RecursiveBT(X,D,C,A)

iii. if Z �fail, then return Z

=

iv. delete {xi → vik}from A

5. fail Table 2: Recursive Backtracking for CSPs.

RestrictDomain(X,D,C,vik ,xj )

Inputs	CSP
	assignment A
	variable xj
	value vik
Output	CSP

1. for all values vjk ∈ Dj

(a) if vjk is inconsistent with vik according to C

i. Dj = Dj \{vjk}/* deletevjk from xj ’s domain */

2. return (X,D,C)

Table 3: Inconsistency Check.

order valuesforthecurrent variablefromleast-constraining to most-constraining. Thisisprecisely what the least-constraining value (LCV) heuristic does: it chooses a value that rules out the fewest values for neighboring variables in the constraint graph. See Table 5.

ForwardChecking(X,D,C,A,xi )

Inputs CSP current assignment A most-recently-assigned variable xi

Output CSP

1. for all unassigned variables xj whose values are constrained by the value of xi

(a) (X,D,C)= RestrictDomain(X,D,C,A(xi ),xj )

2. return (X,D,C)

Table 4: Forward Checking.

LCV(X,D,C,A,xi )

Inputs CSP current assignment A about-to-be assigned variable xi

Output sorted list of values in domain Di Initialize for all 1 ≤ k ≤ ki, rk =0

1. for all vik ∈ Di

(a)

let A ^′ = A∪{xi �→ vik}

(b)

for all unassigned variables xj whose value is constrained by the value of xi

i. (X,D^′ ,C)= RestrictDomain(X,D,C,A ^′ (xi),xj )

ii. rk = rk + |Dj |−|D_j ^′ |

2. return viks sorted in nondecreasing order by rk values

Table 5: Least-Constraining Value.

3.3 Constraint Propagation (Rough Draft)

Backtracking can be further enhanced with constraint propagation techniques, such as validating node, arc, andpath consistency. Inthis section, we restrict our attentiontoCSPs with only unary and binary constraints, so called binary CSPs. (Fact: Any CSP with n-ary constraints can be converted to a binary CSP.)

A graph is said to be node consistent iff for all variables xi, any unary constraints on the values of Di are satisfied. Unary constraints are satisfied by simply restricting domains.

Agraph is said to be arc consistent iff for all variables xi,xj for which there exists(directed) arc (xi,xj ), for all d ∈ Di, there exists d ^′ ∈ Dj that is consistent with the constraint represented by the arc(xi,xj ).

BackTracking*(X,D,C) Inputs CSP Output satisfying assignment or failure

RecursiveBT*(X,D,C,{})

RecursiveBT*(X,D,C,A)

Inputs	CSP
	assignment A
Output	satisfying assignment or failure

1. if A is complete, return A
2. select unassigned variable xi /* e.g., usingMRV heuristic */
3. order domain values Di /* e.g., usingLCV heuristic */
4. for all vik ∈ Di

(a) if vik is consistent with A according to C

i. let A ^′ = A∪{xi → vik}: i.e., let D ^′ =

i ^{vik^}

ii. (X,D^′′ ,C)= ForwardChecking(X,D^′ ,C,A ^′ ,xi)

iii. Z = RecursiveBT*(X,D^′′ ,C,A ^′ )

iv. if Z �fail, then return Z

=

5. return fail

Table 6: Recursive Backtracking with Forward Checking.

ArcConsistency(X,D,C,A)

Inputs	CSP
	current assignment A
Output	CSP
Initialize	queue Q	of edges among unassigned variables

while Q is not empty

1. delete edge(xi,xj )from Q
1. for all values vik ∈ Di
1. CONSISTENT := false
1. for all values vjk ∈ Dj
2. i. if vik is consistent with vjk, then CONSISTENT = true and break out of this loop
2. if CONSISTENT == false

i. Di = Di \{vik}/* deletevik from xi’s domain */

ii. for all unassigned variables xj whose value is constrained by xi, insert edge(xj ,xi)into Q, ignoring duplicates

return (X,D,C)

Table 7: Arc Consistency.

BackTracking**(X,D,C) Inputs CSP Output satisfying assignment or failure

1. (X,D,C)= NodeConsistency(X,D,C)
2. (X,D,C)= ArcConsistency(X,D,C,{})
3. RecursiveBT**(X,D,C,{})

RecursiveBT**(X,D,C,A)

Inputs	CSP
	assignment A
Output	satisfying assignment or failure

1. if A is complete, return A
2. select unassigned variable xi /* e.g., usingMRV heuristic */
3. order domain values Di /* e.g., usingLCV heuristic */
4. for all vik ∈ Di

(a) if vik is consistent with A according to C

i. let A ^′ = A∪{xi → vik}: i.e., let D ^′ =

i ^{vik^}

ii. (X,D^′′ ,C)= ForwardChecking(X,D^′ ,C,A ^′ ,xi)

iii. (X,D^′′′ ,C)= ArcConsistency(X,D^′′ ,C,A ^′ )

iv. Z = RecursiveBT**(X,D^′′′ ,C,A ^′ )

v. if Z �fail, then return Z

=

5. return fail

Table 8: Recursive Backtracking with Node and Arc Consistency.