# Lecture 2

- Search Tree Implementation
- Header for Decision Tree Implementation
- Dynamic Programming
- Summary of Lectures 1-2
- Exercises

## 1. Search Tree Implementation

- The first element is selected from the still to be considered items
- If there is room for that item in the knapsack, a node is constructed that reflects the consequence of choosing to take that item.
- Then explore the consequences of not taking that item.

- The process is then applied
**recursively**to non-leaf children - Finally, chose a node with the highest value that meets constraints

### 1.1. Computational Complexity

- Time based on number of nodes generated
- Number of levels is number of items to choose from
- Number of nodes at level
**i**is$2^i$ - So, if there are
**n**items, the number of nodes is- $\sum_{i=0}^{i=n}2^i$, i.e., $O(2^{i+1})$

- An obvious optimization: don’t explore parts of tree that violate constraint

## 2. Header for Decision Tree Implementation

```
def maxVal(toConsider, avail):
if toConsider == [] or avail == 0:
result = (0, ())
elif toConsider[0].getUnits() > avail:
result = maxVal(toConsider[1:], avail)
else:
nextItem = toConsider[0]
withVal, withToTake = maxVal(toConsider[1:], avail - nextItem.getUnits())
withVal += nextItem.getValue()
withoutVal, withoutToTake = maxVal(toConsider[1:], avail)
if withVal > withoutVal:
result = (withVal, withToTake + (nextItem,))
else:
result = (withoutVal, withoutToTake)
return result
```

**toConsider**: Those items that nodes higher up in the tree (corresponding to earlier calls in the recursive call stack) have not yet considered**avail**: The amount of space still available- Does not actually build search tree, local variable result records best solution found so far

## 3. Dynamic Programming

**Optimal substructure**: a globally optimal solution can be found by combining optimal solutions to local subproblems- For
`x > 1`

,`fib(x) = fib(x - 1) + fib(x – 2)`

- For
**Overlapping subproblems**: finding an optimal solution involves solving the same problem multiple times- Compute
`fib(x)`

or many times

- Compute

### 3.1. Recursive Implementation of Fibonacci

```
def fib(n):
if n == 0 or n == 1:
return 1
else:
return fib(n - 1) + fib(n - 2)
# fib(120) = 8,670,007,398,507,948,658,051,921
```

#### Call Tree for Recursive Fibonacci(6) = 13

### 3.2. To avoid repeat work

- Create a table to record what we’ve done
- Before computing
`fib(x)`

, check if value of`fib\x)`

already stored in the table- If so, look it up
- If not, compute it and then add it to table

- Called
**memoization**

- Before computing

#### Use a Memo to Compute Fibonacci

```
def fastFib(n, memo = {}):
"""Assumes n is an int >= 0,
memo used only by recursive calls Returns Fibonacci of n"""
if n == 0 or n == 1:
return 1
try:
return memo[n]
except KeyError:
result = fastFib(n-1, memo) + fastFib(n-2, memo)
memo[n] = result
return result
```

### 3.3. Take another sample for knapsack

- Since
`a`

and`b`

have the same calories, the rest calculations of the two which marked by red box will be the same. This will be a**overlapping subproblems**which can be optimized.

#### Modify maxVal to Use a Memo

- Add memo as a third argument:
- def fastMaxVal(toConsider, avail, memo = {}):

- Key of memo is a tuple
- take (items left to be considered, available weight) as the key, which will be
`memo[(len(toConsider), avail)]`

. - Items left to be considered represented by
`len(toConsider)`

- The value will always be the same.

- take (items left to be considered, available weight) as the key, which will be
- First thing body of function does is check whether the optimal choice of items given the the available weight is already in the memo
- Last thing body of function does is update the memo
- sample code

## 4. Summary of Lectures 1-2

- Many problems of practical importance can be formulated as
**optimization problems** **Greedy algorithms**often provide adequate (though not necessarily optimal) solutions- Finding an optimal solution is usually
**exponentially hard** - But
**dynamic programming**often yields good performance for a subclass of optimization problems—those with optimal substructure and overlapping subproblems- Solution always correct
- Fast under the right circumstances