Performance issues with objects not in the location

Authoring Inform 7
1

My application has lots of objects… lots and lots. I had to up the memory settings to let the program compile, etc. This isn’t really too much of a problem in and of itself, not yet anyway, but I wonder if there is something else I’m missing that could help. Basically, I understand that it will take longer to load the game in the first place, that’s entirely understandable. However, what confuses me is that the more objects there are, the slower the performance even if most of those objects are not in the current room the player occupies.

I made sure I don’t have any loops which are running through all things, or anything like that… I’m not explicitly (at least to my knowledge and on purpose) running any functions against objects that are not visible. Yet, some actions seem to bog down performance the more objects that are in the game, regardless of if they are in the location.

I do have definition phrases and have functions which use lists of objects that are triggered by commands… is the game considering all objects in the game when sorting through those definitions and lists instead of only working with items that are in the current location? Is there a way to control this if so, or control whatever it might be that is having the game seemingly consider everything in the game all the time even though I don’t have any loops I intentionally coded that way?

2

A very simple I7 game should not show this. If you just create a room and a hundred objects not in the room, the out-of-scope objects will never be considered.

However, many I7 statements do deal with objects that are out of scope. Nearly any statement that refers to “all things” (or “all lit things”, etc) will iterate through all things. (“All ” will use a more efficient loop that iterates just through objects of that kind.)

It’s hard to tell what’s going on in your game, because this can happen from lots of causes. You might have to muck around with the profiling tools, which are not friendly, I’m afraid.

3

That was going to be my follow-up actually… is there a way for me to see a log file of each session of play? It is hard to track down what is taking so long, if it isn’t obvious already by manually reading the code, without some text that is saying what it is doing at the moment… profiling tools?

4

The first step is probably more inspection. Are some actions notably slower than others? If so, what do they have in common?

5

If you’re doing lots of dynamic descriptions manipulating fairly long strings of indexed text, those operations would be my first suspect.

6

First of all, I’m assuming that we’re talking I7 and not I6.
Second of all, that depends on what you want the log file to contain. There are debug commands in I7 that can trace and list property values, relations, rules, actions, scenes, instantations, verb information, and object scopes. If you want real lowlevel stuff, try the “trace 1-6” command. I doubt that even “trace 6” contains any information about memory usage, though. “showheap” is for I7 devs only, but I guess that could contain something useful about memory.
…or do you simply want a simple playtest transcript? In that case I7 has a built in one, and the parsers’ transcript can be turned on with “transcript on”.

7

On a fairly basic level, there’s the built-in “rules” or “rules all” command, which you type while the game is running; it’ll print out a list of the rules that are running. (Up to a point; I don’t think it prints things like the basic accessibility rule.) …although, on a quick test with a time-consuming rule, it seems like it waits to print out its output until all the rules have run. Still, it’ll let you know what’s running.

8

Ok, so I ran trace 6, and what I found out was that every object in the game seems to be considered for every action. If I try to “look bob,” even though “bob” is a very clear-cut answer for the parser as far as I can see, it DSA decide scope on every object in the game, starting with “the north” and running through everything… instead of just deciding on things in the location. So, the more things, the worse performance gets, regardless of where they are.

I tried redefining the examining command, just to troubleshoot ,because the fact that a visible thing needs to be able to be seen doesn’t seem to restrict the parser to looking only in the current location for scope. I tried to define a new property “present” but this is giving me an error that an action can only apply to things or kinds of value?

A thing can be present.

Definition: something is present if it is in the location.
Definition: something is not present if it is not in the location.

Understand the command "look" as something new.

f-examining is an action applying to one present thing.

Understand "look [something]" as f-examining.

Instead of f-examining:
	do-examining;

Instead of someone f-examining:
	do-examining instead;
	
Instead of asking someone to try f-examining:
	do-examining instead;

To do-examining:
        say "Examining [the noun].";
9

The directions are in scope in all rooms, but objects out of play shouldn’t be.

This for instance does not suggest that anything but the twelve directions, yourself and Bob are in scope:

The Place is a room.

Bob is a man in place.
A sword is a kind of thing.
There are 15 swords.
Bill is a man.
Excalibur is a sword.
There is a grue.

Test me with "trace 6 / x bob"
10

Ok, some more conclusive info here… I ripped things apart with comments and code band-aids until I narrowed down the lines of code that were causing the behavior. It was coming from the conversation extensions. The ones by Eric Eve add some … recursive (not sure if that is the right word) scope checking, but it’s not that major. However, if I have Threaded Conversation by Chris Conley included, the whole thing blows up.

I’m not knocking his extension, I have no idea why it’s doing it… it may just not have been built with a crazy assembly-happy game with hundreds of objects like mine in mind? Maybe it works without issue for more “normal” I7 games? I’m just not sure why, but it seems to either be looping through every item in the game, or looping through the same items over and over when this extension is included though. Things go from maybe 1 or 2 screens worth of trace 6 lines to a gigantic amount where the scrollbar icon is as thin as a fingernail when doing anything with this extension.

Simple answer… don’t use it, but from what I read about it, it looked interesting. I wonder why it is blowing me up?

11

Here’s all the debugging commands of Inform 7. Just make a separate extension out of it, start the game, and type “options”. (I’ve updated the code since, but it should work just fine.)

[spoiler][code]
Chapter - Requesting options

Understand “options” or “settings” as requesting options.

Requesting options is an action out of world.

Report requesting options (this is the standard options report rule):
Say “[line break][bold type]Default options[roman type][paragraph break]”;
Say “For the player, there are only two options affecting gameplay:[paragraph break]”;
Say “Room description printings, that are typically only set to establish the scenario and aren’t meant to contain anything practically useful, can be set by the following commands:[line break]”;
Say “[bold type][fixed letter spacing]verbose[variable letter spacing][roman type] - Long room descriptions will always be printed (even when you re-enter rooms). This is intended for immersion, and is the default setting.[line break]”;
Say “[bold type][fixed letter spacing]brief[variable letter spacing][roman type] - Long room descriptions are only printed upon the first visit of a room, or if you type ‘[fixed letter spacing]look[variable letter spacing]’ to look around. This is the default setting of older IF games.[line break]”;
Say “[bold type][fixed letter spacing]superbrief[variable letter spacing][roman type] - Long room descriptions will never be printed (even if you type ‘[fixed letter spacing]look[variable letter spacing]’). This is intended for people with short attention spans.[paragraph break]”;
Say “[bold type][fixed letter spacing]notify on[variable letter spacing][roman type]/[bold type][fixed letter spacing]off[variable letter spacing][roman type] - Turns on/off score notifications (such as ‘[fixed letter spacing][bracket]Your score has just gone up by one point.[close bracket][variable letter spacing]’).”.

Report requesting options (this is the playtesting options report rule):
Say “Playtesters may also find the following options useful:[paragraph break]”;
Say “[bold type][fixed letter spacing]version[variable letter spacing][roman type] - Gives the full banner text associated with the game, including title, author, release number, IFID, and other bibliographical data; it follows this with a list of the included extensions.[line break]”;
Say “[bold type][fixed letter spacing]pronouns[variable letter spacing][roman type] - Announces to the player what the game is currently understanding as the antecedents of ‘him’, ‘her’, ‘it’, and ‘them’. This is often useful during testing, but sometimes also during play.[line break]”;
Say “[bold type][fixed letter spacing]transcript on[variable letter spacing][roman type]/[bold type][fixed letter spacing]off[variable letter spacing][roman type] - Turns on/off logging of the playthrough (if supported by your interpreter). (Works in Windows Frotz, but not inside Inform 7.)[line break]”;
Say “[bold type][fixed letter spacing]verify[variable letter spacing][roman type] - Verifies the story file for errors by examining checksums. This is a legacy command that is rarely useful with modern mediums.[paragraph break]”.

[All of these options are present in the release.]

Chapter - Developer options - Not for release

Understand “dev” or “debug” or “cheats” as requesting developer options.

Requesting developer options is an action out of world.

Report requesting developer options (this is the developer testing commands report rule):
Say “[line break][bold type]Run-time testing commands[roman type][paragraph break]”;
Say “(Note that these commands may change in future releases of Inform 7. These are all found to work in Build 6G60.)[paragraph break]”;
Say “[bold type][fixed letter spacing]showme [variable letter spacing]x[roman type] - Lists the properties (including any relations) of an object anywhere in the model world. Typing it alone will list all objects within the current room.[line break]”;
Say “[bold type][fixed letter spacing]test [variable letter spacing]x[roman type] - Execute a test script. Typing it alone will list which test scripts are available.[paragraph break]”;
Say “[bold type][fixed letter spacing]purloin [variable letter spacing]x[roman type]- Immediately places an object in the player’s inventory, no questions asked. This is not limited to physical things, so typing ‘[fixed letter spacing]purloin all[variable letter spacing]’ is not recommended.[line break]”;
Say “[bold type][fixed letter spacing]abstract [variable letter spacing]x[fixed letter spacing] to [variable letter spacing]y[roman type] - Similar to purloin, but gives the first object to the second object, in a sensible way (such as placing things inside containers).[line break]”;
Say “[bold type][fixed letter spacing]gonear [variable letter spacing]x[roman type] - Teleports the player to the room that contains the object.[line break]”;
Say “[bold type][fixed letter spacing]scope [variable letter spacing]x[roman type] - Shows the scope from an object. Typing it alone will show the scope from the player.[line break]”;
Say “[bold type][fixed letter spacing]random[variable letter spacing][roman type] - Sets the random-number generator to a predictable seed value (making the generator predictable inbetween plays).[line break]”;
Say “[bold type][fixed letter spacing]relations[variable letter spacing][roman type] - Lists the current state of any mutable relations created in the source code.[paragraph break]”;
Say “[bold type][fixed letter spacing]actions on[variable letter spacing][roman type]/[bold type][fixed letter spacing]off[variable letter spacing][roman type] - Actions tracing on/off. Lists every action as it happens, and what its outcome is.[line break]”;
Say “[bold type][fixed letter spacing]scenes on[variable letter spacing][roman type]/[bold type][fixed letter spacing]off[variable letter spacing][roman type] - Scene-change tracing on/off. Also lists the scenes currently happening.[line break]”;
Say “[bold type][fixed letter spacing]rules on[variable letter spacing][roman type]/[bold type][fixed letter spacing]off[variable letter spacing][roman type]/[bold type][fixed letter spacing]all[variable letter spacing][roman type] - Rules tracing on/off. Lists the name of rules before they execute. The ‘[fixed letter spacing]all[variable letter spacing]’ setting will include even the rules [italic type]considered[roman type].[paragraph break]”;
Say “[bold type][fixed letter spacing]showverb [variable letter spacing]x[roman type] - Shows I6-level information about a verb: Synonyms, arguments, and whether the arguments are reversed for a particular line.[line break]”;
Say “[bold type][fixed letter spacing]tree [variable letter spacing]x[roman type] - Prints the I6 object tree, showing the instantiation and the containment relation of an object. Typing it alone will show all instantiations, indented according to the containment relation. Typing ‘[fixed letter spacing]tree all[variable letter spacing]’, will list all objects in detail.[line break]”;
Say “[bold type][fixed letter spacing]trace [variable letter spacing]x[roman type] - Sets the level of parser tracing, listing very low-level parsing information. Levels range from level 0 (off), through level 1 (on) to level 6 (the most detailed).[line break]”;
Say “[bold type][fixed letter spacing]showheap[variable letter spacing][roman type] - Shows the heap of managed memory used by the story file (if used). (Meant for Inform 7 developer debugging only.)[paragraph break]”.

[The testing commands, are all listed here: http://inform7.com/sources/src/i6template/Woven/B-testt.pdf ]
[There is also a mention of a Glulx-only command, called GLKLIST. I should switch to Glulx, and test if it’s there.]

[/code][/spoiler]

12

Thank you for sharing the debugging extension code, Andreas.

As I mentioned in my previous post, performance of my application skyrocketed when I took the Threaded Conversation extension out.

Even after doing this though, I am noticing some increase in performance degradation the more objects there are, even if they aren’t in the location. It is much less than it was with that extension installed, but it’s still there.

Trace 6 doesn’t show any additional items being considered for adding more items not in the location, but there’s still more lag detectable by my human senses the more objects that are in the game… I’m not sure what debug command I’d use to figure that out…

Is it just that the game is having to “look through” the list of all objects to find the ones it wants to actually parse, so even determining what is in the location or what is visible is causing the slowdown?

13

Out of the box, no. But if you’ve added some rule or condition that does that, then yes.

14

Do definition phrases have the ability to cause this kind of behavior?

I looked through everything and took out all extensions except Plurality… because taking that out would be a nightmare… and I don’t have any loops that reference “all things” or anything like that, but there are several definitions. I can’t think of anything else… but what’s really weird is that the parser isn’t saying that it’s considering anything not in the location, so…

15

Sure, depending on how they’re written. I mean you could write

Definition: a thing is pitchy if the number of lit things is zero.

…and that would have to iterate through every thing in the game.

The “trace 6” command doesn’t trace every loop in the game. It just shows you the parser’s object-selection loop. There may be other stuff going on that doesn’t do trace output.

16

I found that problem in Threaded Conversation quite a while ago and have fixed it in my personal copy: I think it was an after deciding the scope rule without a “make no decision” interacting badly with other extensions. I’ll post the code when I’m back at my main computer.

17

Is there a tool or debug command that shows how long each rule or phrase is taking to process? I mean, I’m commenting things out, trying to find the culprit, but so many things are interdependent in my application it’s hard to isolate things. If I knew how long each rule was running, it might help.

18

You could try Benchmarking by Danii Willis, although I don’t know if that can handle individual rules.

19

Ok, I may have found the culprit of the remaining slowdown… I had to rebuild the application piece by piece to find it:

It is the underlying and overlying relationships defined in the layered clothing example and improvements by Shadow Wolf:

Underlying relates various garment-elements to various garment-elements with fast route-finding. The verb to underlie (it underlies, they underlie, it is underlying, it is underlaid) implies the underlying relation. The verb to be under implies the underlying relation.

Definition: a garment-element is uppermost if it is not under something.

Overlying relates various garment-elements to various garment-elements with fast route-finding. The verb to overlie (it overlies, they overlie, it is overlying, it is overlaid) implies the overlying relation.

I can start a new thread if it makes more sense… but I think its the fast route-finding that is killing performance… however, if that is taken away, then the interaction of these lines with later rules that actually make use of these relationships makes the game all but crash, taking seemingly “infinite” time to process.

I’m using this because I have lots of equipment in the form of clothes, armor, and accessories in mind for the real game I hope to morph my learning project into, and lots of body-parts which act as “slots” to put the equipment into. Many of the other rules regarding managing this implementation rely on these overlying and underlying relationships… but boy is it hard on performance. Why does the game have to consider every garment-element in the game, seemingly, to deal with parsing a command against garment-elements in the location?

20

You must have a rule in there that checks for only uppermost garments.

I don’t really see the reason for these relations when you’re using simple “clothing slots”. I’d recommend the code from Counterfeit Monkey for your purposes.