A game theoretic investigation for high interaction honeypots

Honeypots are traps designed to resemble easy-to-compromise computer systems in order to deceive botmasters. Such security traps help security professionals to collect valuable information about botmasters' techniques and true identities. Depending on the complexity of services provided by honeypots, botmasters might be able to detect these traps by performing a series of tests. In particular, to detect honeypots, botmasters can command compromised machines to perform specific actions such as targeting sensor machines controlled by them. If honeypots were designed to completely ignore these commands, then they can easily be detected by the botmasters. On the other hand, full participation by honeypots in such activities has its associated costs and may lead to legal liabilities. This raises the need for finding the optimal response strategy needed by the honeypot in order to prolong its stay within the botnet without sacrificing liability. In this paper, we address the problem of honeypot detection by botmasters. In particular, we present a Bayesian game theoretic framework that models the interaction between honeypots and botmasters as a non-zero-sum noncooperative game with uncertainty. The game solution illustrates the optimal response available for both players. Simulation results are conducted to show the botmasters' behavior update and possible interactions between the game players. The obtained results can be utilized by security professionals to determine their best response to these kind of probes by botmasters.