The objective of this research is to develop methods to design and control heterogeneous portfolios of energy storage devices for the power grid. The approach of this research is to use approximate dynamic programming to develop optimal control policies, that will then be used to understand the economic value of different technologies in the context of a complete power grid. Intellectual merit The intellectual merit of the project is the development of scalable algorithmic technologies for solving high-dimensional stochastic optimization problems arising in energy storage. We propose to use the framework of approximate dynamic programming coupled with tools from machine learning and convex optimization. We exploit convexity which makes it possible to construct effective approximations that scale to handle large numbers of storage devices. Broader impacts The broader impacts of the research will be: 1) The research will guide the design of storage devices so that they meet the specific needs of the power grid in the presence of large supplies of intermittent energy such as wind and solar. 2) Renewable energy, coupled with appropriately designed storage, should dramatically reduce the need for coal. 3) The research, including the approximate dynamic programming models and algorithms, will be made available using a special website with datasets, software, published research and working papers, and downloadable presentations. The results will be integrated in courses at Princeton University, and presented at conferences and workshops to a broad community spanning energy systems and economics, as well as the algorithmic communities.
We have developed a near-optimal algorithm for controlling multiple (dozens, hundreds, even thousands) of storage devices around the grid in the presence of highly volatile sources of energy from wind and solar. The method, based on the modeling and algorithmic framework of approximate dynamic programming, exploits the natural convexity in the value of energy in storage by fitting piecewise linear value function approximations. These can handle time-dependent patterns of wind and solar, as well as electricity prices and loads. The logic can be used by an independent system operator to coordinate storage around the grid to minimize congestion. The system can also be used to simulate the behavior of an ISO to study the impact of growing penetrations of renewables, and to study the ability of storage to smooth the loads on the grid. We have developed an integrated system that simultaneously performs economic dispatch (the tuning of generators that are already operating) while also issuing charge/discharge instructions to the storage devices. We have shown that the logic can reduce grid congestion, helping ISOs to reduce the need for expensive transmission investments. The research represents a fundamental advance in the solution of high-dimensional resource allocation problems. We have shown that the method dramatically outperforms a wide range of approximation strategies based on the work in reinforcement learning. It also outperforms a powerful technique known as the "stochastic dual decomposition procedure" (SDDP) which is based on multidimensional cutting planes. SDDP has two main limitations: the multidimensional cutting plane approach does not scale past 10 or 20 storage devices. Also, SDDP has to work with a small sample of "scenarios" which dramatically simplifies the uncertainties that these large grids have to manage.
