Multi-Agent Learning in Smart Power Grids for Efficiently Acquiring and Distributing Electricity

Abstract

In smart power grids, the demand and supply of electricity can vary signi?cantly, even over short periods of time. Similarly, the pricing of electricity is dynamic and varies with the changes in demand and supply. The needs and behavior of consumers regarding the electricity consumption change from individual to individual and from time to time. Similarly, the preference of daily tasks and comfort levels differ according to weather conditions and from individual to individual. It is necessary to meet the energy requirements of users with the comfort level they choose. To achieve this, the consumers should be able to con?gure and teach their devices to run their daily tasks according to their needs and wants. Arti?cial learning algorithms can potentially be used by intelligent devices in smart grids to provide such capabilities. They can learn to effectively schedule the daily loads in smart grids. This will help consumers to determine when to use an appliance in a cost effective way. Eventually, it will result in reducing the peak demand, as consumers will learn to shift their non-urgent energy consumption to off-peak periods to get lower electricity bills. In this research, we analyzed the performance of arti?cial learning algorithms in smart power grids when multiple learning agents interact with each other. Specifically, the emphasis was to test the ability of these algorithms to ef?ciently acquire and distribute energy resources in new-age electricity markets. Our study shows that Q-Learning consumers are not able to collectively achieve the social optimum behavior, and, hence, are not able to maximize the social welfare of the market in our smart gird problem. This is due to non-stationary of the environment, ?uctuations in supply from renewable sources, and a large state space. We then reformulated the learning problem to better understand the fundamentals of learning in smart grids. Our results show that R-lopars and GIGA-WoLF perform the best on average against other associates. GIGA-WoLF takes the least time to learn but does not converge to a good solution. R-lopars performs the best in self play and achieves the closest payoff to the Nash bargaining solution out of the selected algorithms.

Date of Award	Dec 2011
Original language	American English
Supervisor	Jacob Crandall (Supervisor)