This proposal explores an online reinforcement learning framework that can provide high capacity rating and scheduling of many end user-level flexible resources such as swimming pools. In sharp contrast with conventional approaches of statically and uniformly treating end user loads with small capacity rating and scheduling them via heuristics based algorithms, the proposed framework will provide a theoretically rigorous and practically scalable approach for learning the unknown parameters of end user loads and adaptively controlling them with provable guarantees.
Intellectual Merit:
(i) This proposal will illustrate the possibility of substantial increasing of capacity credit from end user demand response in provision of spinning reserves via scalable real-time estimation and control as opposed to the conventional heuristic based scheduling algorithms. (ii) This proposal will introduce a learning and adaptive control algorithm using the framework of online reinforcement learning to address the operational problems when the consumer specific parameters are unknown. (iii) This proposal will introduce an index-based learning and scheduling algorithm that scales only linearly with the number of end users. (iv) This proposal will test a data-driven optimal scheduling that jointly maximize the profit for the aggregator and track the required reserve provision trajectory from the collection of even a small number of flexible users. The proposed research is generalizable towards many resource scheduling problems with uncertainty that arise in the context of transportation, communication, and other engineering dynamical systems.
Broader Impacts:
Once successful, this project will provide a systematic approach for obtaining spinning reserve at much less cost from flexible end user resources in a provably reliable and environmentally sustainable way. This team will introduce new course modules on the topic of data-driven online learning in dynamical systems, which closely integrates reinforcement learning, dynamical control, and optimization for more than 200 undergraduate and graduate students currently enrolled in related areas courses at Texas A&M. This team will continue the strong track record of engaging undergraduate students for research, in particular the under-representative groups.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
