I’m a computer science and artificial intelligence student and I’m interested in creating an AI system capable of good natural language understanding and representation. What my advisor and I think is that IF could be a great source of data and learning material for such system. Why? IF stories or games usually include some sort of feedback to the player which is something that could possibly help the system learn useful representations of natural language. The dream scenario would be to let the system train on a set of IF games and then test it on games that it has never seen before and see it perform well (or at least human-like in some ways).
There are already two papers that do something similar , except they simply test their systems on the same games they have been trained on – which makes very little sense to me, as this makes the task virtually trivial.
Now I have a couple of questions about the availability of IF games with specific features which might help (they might or might not be necessary) us create the system described above:
Are there any games (or perhaps servers) with user input data available? More specifically, it would be extremely useful to know how exactly users play IF games (simply put, see their input and actions) or in other words – how well they play. Using this data, it might be possible to bootstrap the system to behave at least partially human-like. Even more importantly, it would be great to have such data for evaluation, so that we could see how the AI system’s peformance compares to that of human players.
Some (perhaps most?) games have different endings. In that case, the endings can usually be associated with some kind of reward. The problem is that the reward is specified in the form of words, perhaps a couple of sentences. To evaluate the human or AI players, though, it would be very much needed to assign numeric values to the different endings in different games. Are there any (sets of) games that actually have their endings annotated with numbers (score)?
Are there any random games? The randomness can either come from: A. a random description of the same inner world state (e.g. ‘You see a bird on the window’ and ‘There is a bird on the window’); or B. random transitions between the world states (one action from a given world state can lead to two or more different world states, based on some random distribution).
Ideally, I’d like to have a set of games with all the properties above but I realise such dataset (or in this case, such collection of IF games) may not be available at all. At any rate, any help or feedback is greatly appreciated!
As a final note, I do know about MUD games that even offer interaction with human players. The problem is that the system needs to have a very fast simulation of the environment (in the case of IF games, perhaps something like a simple HTML page with hyperlinks) which is something that the server-based MUD games don’t really allow.
Thank you very much