Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[lp_intro] update suggestions #474

Merged
merged 2 commits into from
Jul 3, 2024
Merged

[lp_intro] update suggestions #474

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
18952a9
[lp_intro] update suggestions
longye-tian Jun 26, 2024
2c2187f
[lp_intro] update notations
longye-tian Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 28 additions & 32 deletions lectures/lp_intro.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,14 @@ Linear programs come in pairs:

* an associated **dual** problem.

If a primal problem involves **maximization**, the dual problem involves **minimization**.
If a primal problem involves *maximization*, the dual problem involves *minimization*.

If a primal problem involves **minimization**, the dual problem involves **maximization**.
If a primal problem involves *minimization**, the dual problem involves **maximization*.

We provide a standard form of a linear program and methods to transform other forms of linear programming problems into a standard form.

We tell how to solve a linear programming problem using [SciPy](https://scipy.org/) and [Google OR-Tools](https://developers.google.com/optimization).

We describe the important concept of complementary slackness and how it relates to the dual problem.

Let's start with some standard imports.

```{code-cell} ipython3
Expand All @@ -56,7 +54,7 @@ Let's start with some examples of linear programming problem.



## Example 1: Production Problem
## Example 1: production problem

This example was created by {cite}`bertsimas_tsitsiklis1997`

Expand Down Expand Up @@ -144,9 +142,9 @@ In this example, the optimal set is the point $(2.5, 5)$.



### Computation: Using OR-Tools
### Computation: using OR-Tools

Let's try to solve the same problem using the package *ortools.linear_solver*
Let's try to solve the same problem using the package `ortools.linear_solver`.



Expand All @@ -157,7 +155,7 @@ The following cell instantiates a solver and creates two variables specifying th
solver = pywraplp.Solver.CreateSolver('GLOP')
```

Let's us create two variables $x_1$ and $x_2$ such that they can only have nonnegative values.
Let's create two variables $x_1$ and $x_2$ such that they can only have nonnegative values.

```{code-cell} ipython3
# Create the two variables and let them take on any non-negative value.
Expand All @@ -182,34 +180,32 @@ Let's specify the objective function. We use `solver.Maximize` method in the cas
solver.Maximize(3 * x1 + 4 * x2)
```

Once we solve the problem, we can check whether the solver was successful in solving the problem using it's status. If it's successful, then the status will be equal to `pywraplp.Solver.OPTIMAL`.
Once we solve the problem, we can check whether the solver was successful in solving the problem using its status. If it's successful, then the status will be equal to `pywraplp.Solver.OPTIMAL`.

```{code-cell} ipython3
# Solve the system.
status = solver.Solve()

if status == pywraplp.Solver.OPTIMAL:
print('Objective value =', solver.Objective().Value())
x1_sol = round(x1.solution_value(), 2)
x2_sol = round(x2.solution_value(), 2)
print(f'(x1, x2): ({x1_sol}, {x2_sol})')
print(f'(x1, x2): ({x1.solution_value():.2}, {x2.solution_value():.2})')
else:
print('The problem does not have an optimal solution.')
```

## Example 2: Investment Problem
## Example 2: investment problem

We now consider a problem posed and solved by {cite}`hu_guo2018`.

A mutual fund has $ \$ 100,000$ to be invested over a three year horizon.
A mutual fund has $ \$ 100,000$ to be invested over a three-year horizon.

Three investment options are available:

1. **Annuity:** the fund can pay a same amount of new capital at the beginning of each of three years and receive a payoff of 130\% of **total capital** invested at the end of the third year. Once the mutual fund decides to invest in this annuity, it has to keep investing in all subsequent years in the three year horizon.
1. Annuity: the fund can pay a same amount of new capital at the beginning of each of three years and receive a payoff of 130\% of total capital invested at the end of the third year. Once the mutual fund decides to invest in this annuity, it has to keep investing in all subsequent years in the three year horizon.

2. **Bank account:** the fund can deposit any amount into a bank at the beginning of each year and receive its capital plus 6\% interest at the end of that year. In addition, the mutual fund is permitted to borrow no more than $20,000 at the beginning of each year and is asked to pay back the amount borrowed plus 6\% interest at the end of the year. The mutual fund can choose whether to deposit or borrow at the beginning of each year.
2. Bank account: the fund can deposit any amount into a bank at the beginning of each year and receive its capital plus 6\% interest at the end of that year. In addition, the mutual fund is permitted to borrow no more than $20,000 at the beginning of each year and is asked to pay back the amount borrowed plus 6\% interest at the end of the year. The mutual fund can choose whether to deposit or borrow at the beginning of each year.

3. **Corporate bond:** At the beginning of the second year, a corporate bond becomes available.
3. Corporate bond: At the beginning of the second year, a corporate bond becomes available.
The fund can buy an amount
that is no more than $ \$ $50,000 of this bond at the beginning of the second year and at the end of the third year receive a payout of 130\% of the amount invested in the bond.

Expand Down Expand Up @@ -279,9 +275,9 @@ $$



### Computation: Using OR-Tools
### Computation: using OR-Tools

Let's try to solve the above problem using the package *ortools.linear_solver*.
Let's try to solve the above problem using the package `ortools.linear_solver`.

The following cell instantiates a solver and creates two variables specifying the range of values that they can have.

Expand All @@ -290,7 +286,7 @@ The following cell instantiates a solver and creates two variables specifying th
solver = pywraplp.Solver.CreateSolver('GLOP')
```

Let's us create five variables $x_1, x_2, x_3, x_4,$ and $x_5$ such that they can only have the values defined in the above constraints.
Let's create five variables $x_1, x_2, x_3, x_4,$ and $x_5$ such that they can only have the values defined in the above constraints.

```{code-cell} ipython3
# Create the variables using the ranges available from constraints
Expand Down Expand Up @@ -388,7 +384,7 @@ c = \begin{bmatrix} c_1 \\ c_2 \\ \vdots \\ c_n \\ \end{bmatrix}, \quad
x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \\ \end{bmatrix}. \quad
$$

The standard form LP problem can be expressed concisely as:
The standard form linear programming problem can be expressed concisely as:

$$
\begin{aligned}
Expand All @@ -408,15 +404,15 @@ It is useful to know how to transform a problem that initially is not stated in

By deploying the following steps, any linear programming problem can be transformed into an equivalent standard form linear programming problem.

1. **Objective Function:** If a problem is originally a constrained **maximization** problem, we can construct a new objective function that is the additive inverse of the original objective function. The transformed problem is then a **minimization** problem.
1. Objective function: If a problem is originally a constrained *maximization* problem, we can construct a new objective function that is the additive inverse of the original objective function. The transformed problem is then a *minimization* problem.

2. **Decision Variables:** Given a variable $x_j$ satisfying $x_j \le 0$, we can introduce a new variable $x_j' = - x_j$ and substitute it into original problem. Given a free variable $x_i$ with no restriction on its sign, we can introduce two new variables $x_j^+$ and $x_j^-$ satisfying $x_j^+, x_j^- \ge 0$ and replace $x_j$ by $x_j^+ - x_j^-$.
2. Decision variables: Given a variable $x_j$ satisfying $x_j \le 0$, we can introduce a new variable $x_j' = - x_j$ and substitute it into original problem. Given a free variable $x_i$ with no restriction on its sign, we can introduce two new variables $x_j^+$ and $x_j^-$ satisfying $x_j^+, x_j^- \ge 0$ and replace $x_j$ by $x_j^+ - x_j^-$.

3. **Inequality constraints:** Given an inequality constraint $\sum_{j=1}^n a_{ij}x_j \le 0$, we can introduce a new variable $s_i$, called a **slack variable** that satisfies $s_i \ge 0$ and replace the original constraint by $\sum_{j=1}^n a_{ij}x_j + s_i = 0$.
3. Inequality constraints: Given an inequality constraint $\sum_{j=1}^n a_{ij}x_j \le 0$, we can introduce a new variable $s_i$, called a **slack variable** that satisfies $s_i \ge 0$ and replace the original constraint by $\sum_{j=1}^n a_{ij}x_j + s_i = 0$.

Let's apply the above steps to the two examples described above.

### Example 1: Production Problem
### Example 1: production problem

The original problem is:

Expand All @@ -442,9 +438,9 @@ $$



### Computation: Using SciPy
### Computation: using SciPy

The package *scipy.optimize* provides a function ***linprog*** to solve linear programming problems with a form below:
The package `scipy.optimize` provides a function `linprog` to solve linear programming problems with a form below:

$$
\begin{aligned}
Expand All @@ -456,7 +452,7 @@ $$
$$

```{note}
By default $l = 0$ and $u = \text{None}$ unless explicitly specified with the argument 'bounds'.
By default $l = 0$ and $u = \text{None}$ unless explicitly specified with the argument `bounds`.
```

Let's now try to solve the Problem 1 using SciPy.
Expand Down Expand Up @@ -488,7 +484,7 @@ else:

The optimal plan tells the factory to produce $2.5$ units of Product 1 and $5$ units of Product 2; that generates a maximizing value of revenue of $27.5$.

We are using the *linprog* function as a **black box**.
We are using the `linprog` function as a *black box*.

Inside it, Python first transforms the problem into standard form.

Expand All @@ -499,12 +495,12 @@ Here the vector of slack variables is a two-dimensional NumPy array that equals
See the [official documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html#scipy.optimize.linprog) for more details.

```{note}
This problem is to maximize the objective, so that we need to put a minus sign in front of parameter vector c.
This problem is to maximize the objective, so that we need to put a minus sign in front of parameter vector $c$.
```



### Example 2: Investment Problem
### Example 2: investment problem

The original problem is:

Expand Down Expand Up @@ -710,7 +706,7 @@ $$
# Instantiate a GLOP(Google Linear Optimization Package) solver
solver = pywraplp.Solver.CreateSolver('GLOP')
```
Let's us create two variables $x_1$ and $x_2$ such that they can only have nonnegative values.
Let's create two variables $x_1$ and $x_2$ such that they can only have nonnegative values.

```{code-cell} ipython3
# Create the two variables and let them take on any non-negative value.
Expand Down
Loading