Lecture 2

1. Search Tree Implementation

  • The first element is selected from the still to be considered items
    • If there is room for that item in the knapsack, a node is constructed that reflects the consequence of choosing to take that item.
    • Then explore the consequences of not taking that item.
  • The process is then applied recursively to non-leaf children
  • Finally, chose a node with the highest value that meets constraints

1.1. Computational Complexity

  • Time based on number of nodes generated
  • Number of levels is number of items to choose from
  • Number of nodes at level i is2i2^i
  • So, if there are n items, the number of nodes is
    • i=0i=n2i\sum_{i=0}^{i=n}2^i, i.e., O(2i+1)O(2^{i+1})
  • An obvious optimization: don’t explore parts of tree that violate constraint

2. Header for Decision Tree Implementation

def maxVal(toConsider, avail):
    if toConsider == [] or avail == 0:
        result = (0, ())
    elif toConsider[0].getUnits() > avail:
        result = maxVal(toConsider[1:], avail)
        nextItem = toConsider[0]
        withVal, withToTake = maxVal(toConsider[1:], avail - nextItem.getUnits())
        withVal += nextItem.getValue()
        withoutVal, withoutToTake = maxVal(toConsider[1:], avail)

        if withVal > withoutVal:
            result = (withVal, withToTake + (nextItem,))
            result = (withoutVal, withoutToTake)
    return result
  • toConsider: Those items that nodes higher up in the tree (corresponding to earlier calls in the recursive call stack) have not yet considered
  • avail: The amount of space still available
  • Does not actually build search tree, local variable result records best solution found so far

3. Dynamic Programming

  • Optimal substructure: a globally optimal solution can be found by combining optimal solutions to local subproblems

    • For x > 1, fib(x) = fib(x - 1) + fib(x – 2)
  • Overlapping subproblems: finding an optimal solution involves solving the same problem multiple times

    • Compute fib(x) or many times

3.1. Recursive Implementation of Fibonacci

def fib(n):
    if n == 0 or n == 1:
        return 1
        return fib(n - 1) + fib(n - 2)

# fib(120) = 8,670,007,398,507,948,658,051,921

Call Tree for Recursive Fibonacci(6) = 13


3.2. To avoid repeat work

  • Create a table to record what we’ve done
    • Before computing fib(x), check if value of fib\x) already stored in the table
      • If so, look it up
      • If not, compute it and then add it to table
    • Called memoization

Use a Memo to Compute Fibonacci

def fastFib(n, memo = {}):
    """Assumes n is an int >= 0, 
        memo used only by recursive calls Returns Fibonacci of n"""
    if n == 0 or n == 1:
        return 1
        return memo[n]
    except KeyError:
        result = fastFib(n-1, memo) + fastFib(n-2, memo)
        memo[n] = result
        return result

3.3. Take another sample for knapsack


  • Since a and b have the same calories, the rest calculations of the two which marked by red box will be the same. This will be a overlapping subproblems which can be optimized.

Modify maxVal to Use a Memo

  • Add memo as a third argument:
    • def fastMaxVal(toConsider, avail, memo = {}):
  • Key of memo is a tuple
    • take (items left to be considered, available weight) as the key, which will be memo[(len(toConsider), avail)].
    • Items left to be considered represented by len(toConsider)
    • The value will always be the same.
  • First thing body of function does is check whether the optimal choice of items given the the available weight is already in the memo
  • Last thing body of function does is update the memo
  • sample code

4. Summary of Lectures 1-2

  • Many problems of practical importance can be formulated as optimization problems
  • Greedy algorithms often provide adequate (though not necessarily optimal) solutions
  • Finding an optimal solution is usually exponentially hard
  • But dynamic programming often yields good performance for a subclass of optimization problems—those with optimal substructure and overlapping subproblems
    • Solution always correct
    • Fast under the right circumstances

5. Exercises

results matching ""

    No results matching ""