The First Half-Semester

In the first half of the semester the models that are introduced are the logic circuit, the deterministic and nondeterministic finite-state machine, the random access machine (RAM), and the deterministic and nondeterministic Turing machine.

While it is unconventional to begin a theory course with the logic circuit, it is the most fundamental computational model and can be used to express computations by models of machines with memory. For example, a T-step computation by an FSM can be simulated by a circuit constructed from Tcopies of a circuit for the next-state/output function of the FSM.

When this simple idea is extended to nondeterministic TMs it forms the basis for identifying the first NP-complete problem, namely, CIRCUIT SATISFIABILITY, the description of one-output circuits whose inputs can be chosen so that their output is satisfied (has value 1).

The simulation of the RAM by circuits demonstrates that some problems are unsolvable given the resources available. This is captured by a computational inequality of the type given below where S and T are the storage capacity and time used by the RAM to compute a function f and $C_{\Omega}(f)$ is its circuit complexity, namely, the size of the smallest circuit for f using gates from the basis $\Omega$. Here k > 0 is a constant.

$$C_{\Omega}(f) \leq k ST$$

This inequality demonstrates that the space-time product must be sufficiently large relative to the circuit complexity of a function in order for it to be computed.

The important role of circuit complexity is also demonstrated by the following computational inequality for the deterministic TM.

$$C_{\Omega}(f) = O(T^2)$$

If it can be shown that the recognition of an NP-complete language has exponential circuit size $C_{\Omega}(f)$ on strings of length n, this inequality shows that the time for this recognition problem on the TM is also exponential. If this result holds, the complexity classes P and NP are different.

These results, which are relatively easy to derive, demonstrate that many fundamental ideas are made accessible by beginning a first theory course with logic circuits. This material not only grounds theory in a concrete model, it builds on the presumed background that students have in computer organization.

The functional programming language Scheme is used for slightly less than half of the assignments. The Scheme programming assignments are used to cement understanding of circuits as graphs of straight-line programs as well as the simulation of time- and space-limited machines with memory by circuits. These assignments are also used to help them understand the reductions used in showing that CIRCUIT SATISFIABILITY is an NP-complete problem.

We define nondeterministic machines with memory to be deterministic machines with an external input associated with a choice agent that makes the choices to disambiguate next states in the nondeterministic machine. This agent must help the user to accept the input string, if that is possible. Thus, a nondeterministic FSM has two inputs, the standard input and a new one associated with the choice agent. The nondeterministic TM has a choice input on its control unit. This simple pedagogical device greatly simplifies the introduction of nondeterminism.