Optimal Electricity Supply Bidding by Markov Decision Process

1. Optimal Electricity Supply Bidding by Markov Decision Process Presentation Review By: Feng Gao, Esteban Gil, & Kory Hedman IE 513 Analysis of Stochastic Systems Professor Sarah Ryan February 14, 2005 Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren

2. Outline • Introduction • Purpose • Problem Formulation • Model Overview • Summary

3. Introduction • Electric Industry has Transitioned from Regulated to Deregulated • Regulated: Vertically Integrated, Monopolistic Market • Deregulated: Ideally, Perfect Competition Market • Decision Analysis: Based on Profit & Competition

4. Introduction Cont’d • Traditional Power System • Generation, Transmission, Distribution, Consumption

5. Introduction Cont’d • Structure of Power Market • Optimize Resources With Competition

6. Introduction Cont’d • Day Ahead Market Considered • Inelastic Demand • Generation Companies (GenCos) are Risk-Neutral • GenCos Bid a Price (P) & Quantity (Q) • Bids are Chosen from Cheapest to Most Expensive • Market Clearing Price: P from the Highest Chosen Q • (Similar to the New Zealand & Great Britain Electricity Markets)

7. Purpose • Overall Objective: Maximize Expected Profit over a Planning Horizon of 7 Days • For all States, Determine Optimal Bidding Strategy. Depends on: • Competitors’ Bidding • Load Forecasting • Remaining Time Horizon (given day i, 7 – i) • Production Limit (Max Available Supply Remaining over Planning Horizon) • Accumulated Data over Time (Past Load & Price Data)

8. Problem Formulation • States are Defined by 7 Variables: • Peak Load & Peak Price (2) • Off-Peak Load & Off-Peak Price (2) • Current Production Limit for the Remaining Planning Horizon (1) • Load Forecast for the Following Day (2) • Aggregation Limits Number of States • P & D Broken into High, Medium, & Low • Illogical States Ignored: (High P & Low Q, etc.)

9. Problem Formulation Cont’d • Transition Probability Depends on: • Current State i, Subsequent State j, Decision a • Pr (i, j, a) • Decision Maker Receives Reward • R (i, j, a) • State of the Market Defines the Competitors’ Bids & Decision Maker’s Bidding Options • Bid Prices are Determined Using a Staircase Supply Fn for Varying MW

10. Problem Formulation Cont’d • MDP Algorithm Considers: • Rewards Based on Load Forecast • Decisions of a State • Competitors’ Bidding Characteristics • Decision Options Affect Transition Probabilities & Rewards • Competitors’ Bids are Independent • Scenarios (s) are exclusive

11. Model Overview • Probability of Scenario s: • Pr (i, n, k): Probability that Supplier n (n ~= m) Chooses Option k in State i • m is decision maker • Competitors’ Bids are Independent • Remaining Production Limit: • q (i, s, t) is the Q used in period t for scenario s. • Spot Price for Scenario s: SP(i, s, t)

12. Model Overview Cont’d • Probability to Move from State i to j: • Pr (i, LF (j, t) = Probability that Load Forecast for Day After Tomorrow is LF (j, t) Given Present State i • Reward for Decision Maker, r (i, s):

13. Model Overview Cont’d • Reward for Transition from i to j & Decision A is Sum of Rewards Weighted by Conditional Probabilities: • V (i, T+1): Total Expected Reward in T+1 Remaining Stages from State i • Solved by Value Iteration

14. Summary • Introduction • Electric Market is now Competitive • GenCos Bid on Demand • Purpose • MDP Used to Determine Optimal Bidding Strategy • Problem Formulation • Transition Probability Determined by Current State, Subsequent State, & Decision Made • 7 Variables to Define a State • Aggregation Used to Limit Dimensionality Problems • Model Overview • 7 Day Planning Horizon • Objective is to Maximize Summation of Expected Reward • Value Iteration

15. Questions???

16. References • Song, H.; Liu, C.-C.; Lawarree, J.; Dahlgren, R.W, “Optimal Electricity Supply Bidding by Markov Decision Process,” IEEE Transactions. Power Systems, Vol. 15, no. 2, pp 618-624, May 2000. • http://www.acclaimimages.com/_gallery/_pages/0037-0409-0607-4216.html