Memory allocation error (z-code) vs blank story tab (Glulx)

I’m most of the way done with my first real text adventure, and it’s been going good so far.

Unfortunately, when I added one more new item, I started getting this message after I make the first couple of moves in the game:

Memory is constantly being allocated and deallocated, as needed, to store
indexed text, lists and other values which can stretch to contain large
amounts of data. If you see this problem, then the story file has needed
more memory at once than was available.

If I remove the last thing I added, and don’t add any new rooms or items, then everything works fine, but if I add anything at all, I can only make one or two moves, before the game crashes. I tried increasing the dynamic memory allocation, but it made no difference.

I decided that maybe I’m exceeding the limits of z-code, so I went into settings and changed the Story File Format to Glulx, but now when I compile and run, I get a blank Story tab. The Results tab shows no error, but the Story tab is completely blank. I have also tried loading various demos and I always get the blank Story tab, when using the Glulx format.

To provide a little more information: I’m running Inform for Windows 10.1.1.20220821. My game has 16 rooms and 54 things.

Can anyone help me figure how to get Glulx format to work or else how to avoid the memory allocation problem in z-code?

Cheers,
C. Scott Davis

In recent months we have seen reports of problems with over zealous anti-virus checkers quarantining “glulxe.exe” the executable that by default drives the Story tab on Windows, when Glulx is the output mode. To check this, you can open Windows Explorer, go to where Inform is installed (usually “C:\Program Files (x86)\Inform” and look in the “Interpreters” directory. There should be three files in there: “frotz.exe” (interpreter for Z-code), “glulxe.exe” (default interpreter for Glulx) and “git.exe” (alternate interpreter for Glulx).

If one of these is missing, especially if its “glulxe.exe”, that’s likely to be the problem. You’ll need to see if your anti-virus has blocked that executable or hidden it somewhere. Or, if you can’t be bothered with that, and “git.exe” is still there, go to the Preferences dialog, Advanced tab, and set the “Glulx interpreter” dropdown to “Git”, which should cause the front-end to use Git to interpret Glulx games.

3 Likes

McAfee had quarantined both glulxe.exe and git.exe. I moved them out of quarantine and added them to McAfee’s exclusion list, so it wouldn’t quarantine them again.

Now when I compile and run my game, I get this message, instead of a blank tab:

Playing the Game - Failed
The application compiled and ran your source text, but the resulting game
caused the interpreter to fail. This should never happen.

If you can reproduce this problem, please report it to David Kinder, the
maintainer of this Windows build of Inform 7.

Sorry for the inconvenience.

This was followed shortly by a McAfee pop-up stating that “We caught something trying to hijack one of your apps.”

Setting the Glulx interpreter to “Git” has the same result.

Ah, lovely McAfee. Even by the standards of the anti-virus software industry, they’re impressively rubbish.

Unfortunately there isn’t anything I can do about this. Nothing in the interpreters is trying to hijack anything, whatever that means. If you can turn off whatever witless check McAfee is doing, then the system will work.

3 Likes

back to the root cause of Scott Davis’s problem, if you compile from the command line using the powerful i6 setting ‘$OMIT_UNUSED_ROUTINES=1’ you will gain a substantial elbow space in .z8

I use Linux and compile .z8 code under inform 10 with this nifty bash script:

inform/inform7/Tangled/inform7 -release -project $1.inform/ -format=Inform6/16

inform/inform6/Tangled/inform6 -v8 -w -~D '$OMIT_UNUSED_ROUTINES=1' $1.inform/Build/auto.inf -o $1.inform/Build/$1.z8

With this script and an empty lab, I get a 335,360 bytes .z8 story file

Best regards from Italy,
dott. Piergiorgio

I’m confused by my z-code memory problem. It compiles and runs with no problems, and I can play it all the way through, but if I add just one simple object of any kind, the second attempt to move will result in:

*** Run-time problem P49: Memory allocation proved impossible.

The Console tab shows about 80k free in z-machine.

Remove the object, everything’s fine. Add one, it crashes on the second attempt to move.

One thing to try may be “rules all” as an out-of-world command before an actual move. That shouldn’t activate the turn rules, but you may be able to see which rule the compiler crashes on, and that may help.

The Z-code problem is about RAM: the Z-Machine only provides 64k of it, and there’s no way to make that bigger. Inform has some code in it that simulates a dynamic memory heap, which works by putting a part of that RAM aside to act as a heap. But when that is all used up, you get this error.

The advice to add OMIT_UNUSED_ROUTINES is unhelpful - that will do nothing to make more RAM in the Z-Machine available to your game. Generally when you hit such problems you either need to start digging into the Inform template layer, or give up and go to Glulx. If I were you I would just uninstall McAfee and do that :slight_smile: .

3 Likes

I tried the “rules all” suggestion and this was the result:

>rules all
Rules tracing now switched to "all". Type "rules off" to switch it off again.

[Rule "parse command rule" applies.]
>n
[Rule "declare everything initially unmentioned rule" applies.]
[Rule "generate action rule" applies.]

*** Run-time problem P49: Memory allocation proved impossible.

Unfortunately, I’m not really sure what it’s telling me.

Every object in the game consumes some amount of virtual machine RAM (to store its properties, its location, etc), and it seems you’ve hit the point where there’s not enough RAM left for the dynamic memory heap. This means that as soon as Inform requests any memory from the heap, the game crashes.

The only solution is to use less RAM (use fewer objects, fewer variables, etc) or switch to Glulx. I recommend switching to Glulx, and turning off the malfunctioning antivirus that keeps killing the Glulx interpreters.

3 Likes

Quick background:

  • The “free space” reported by the compiler output refers to the maximum size of the game expressed as “packed memory” – which I don’t want to really get into but is a multiple of the 64K mentioned by DavidK. For Z8, the multiplier is 8x, so 512K.
  • Routines and fixed text can be parked in “high memory” but certain constructs (such as modifiable text, lists and some forms of relation) must go into the “low memory” of the first 64K.
  • Part of the low memory is set aside as a run-time heap for use by those constructs. The amount set aside is 8K by default but can be automatically raised to 16K (the functional maximum) or manually raised using the Use dynamic memory allocation... option that you tried. (This likely had no effect because it had already been increased automatically.) You can use the debugging command >SHOWHEAP to see how large of a heap has been allocated and its current usage.

You must be making heavy use of the heap for it to run out of space on move 2. I have to admit that I’m curious about what you are doing that would cause this. There may be a memory leak affecting your game, but it’s also likely that you could do things differently to reduce the heap memory footprint.

Are you very committed to sticking with the Z-Machine?

I’m not committed to the z-machine, as such, but I’m concerned that I might be doing something that is unnecessarily gobbling up memory and I’m also not super thrilled at having to disable my virus scanner in order to use glulx.

When the game starts, SHOWHEAP reports 8186 of 8208 free. After I open the door, SHOWHEAP still reports 8186 of 8208 free. When I try to go north, the game crashes with the error “*** Run-time problem P49: Memory allocation proved impossible.”

If I remove just one object from the game, there is no crash, and after 57 moves, SHOWHEAP still reports 8186 of 8208 free. If I add a single object definition of any kind, the game crashes on the second move again.

It’s telling you that the memory error occurs somewhere in the small amount of code between the “generate action rule” commencing, and this rule then calling the “standard set going variables” rule.

The “generate action rule” does try to dynamically allocate memory on the memory stack for any action variables needed by the going action before calling the “standard set going variables” rule, and presumably this allocation is what is failing.

I can’t really look into this any further without being able to trace what’s going on in your code precisely, which would require you sending the full source…

I don’t know how to trace what’s going on in my code, but I do know that if I remove one object, SHOWHEAP reports “8186 of 8208 bytes free”, RULES ALL shows a very long list of rules, and there is no crash, even after 57+ moves.

If I add just one object, even a very simple one, this is the result, when I make my second move:

>showheap
8186 of 8208 bytes free.
>rules all
Rules tracing now switched to "all". Type "rules off" to switch it
off again.

[Rule "parse command rule" applies.]
>n
[Rule "declare everything initially unmentioned rule" applies.]
[Rule "generate action rule" applies.]

*** Run-time problem P49: Memory allocation proved impossible.

I’m sure there are lots of things that I’m not doing the best or most efficient way, but I’m very confused by how the addition or removal of one object is causing such a drastic difference between working perfectly and crashing on the second move.

Like Dr. Bates said, at this point it’s not likely anyone will be able to help without source code that reproduces the problem. Usually we ask for the shortest, simplest code that does so, but in this case it may be very hard to reproduce if you remove things.

This is sounding like an Inform 7 bug.

Per drpeterbatesuk’s suggestion stack frame allocation might be where it’s failing, you can try to add:

Include (-

[ StackFrameCreate size new;
	new = I7SFRAME - WORDSIZE*size;
	print "^blockv_stack: ", blockv_stack;
	print "^new: ", new;
	if (new < blockv_stack) { RunTimeProblem(RTP_HEAPERROR); @quit; }
	I7SFRAME = new;
	"^finished!";
];

-) replacing "StackFrameCreate".

and let us know what output results.

EDIT: OK, some interesting things while following up on this. Note that all of the below is for 6M62 and may have changed a bit in 10.1.

First: I think the Use dynamic memory allocation of... option has only one effect, which is to override the default value of 8192 in the declaration of constant DynamicMemoryAllocation.

! Use option:
 Constant DynamicMemoryAllocation = 8192; ! value overridden if 'Use dynamic memory allocation...' provided

8192 bytes is coincidentally also the default size of the Flex heap (which is the automatically-sized heap I mentioned above), but there is no direct connection as I had previously believed (and as seems to be implied in WWI 20.2 Memory limitations based on the description of behavior; perhaps this information is out-of-date?).

Second: The DynamicMemoryAllocation constant has one and only one use in the rest of the code, which is to set the value of constant BLOCKV_STACK_SIZE.

#ifdef TARGET_ZCODE;
Constant BLOCKV_STACK_SIZE = 224;						! note invariant size in Z-machine
#ifnot;													! for TARGET_GLULX
Constant BLOCKV_STACK_SIZE = DynamicMemoryAllocation/4; ! clearer to divide by WORDSIZE?
#endif;

...

Array blockv_stack --> BLOCKV_STACK_SIZE;				! number of words, not bytes -- 448 bytes for Z

The blockv_stack array is used as a stack for the block value subsystem, and it is separate from the Flex heap which is also used extensively by the same system. So, for Glulx (and Glulx only), DynamicMemoryAllocation is used to set the number of entries in the block value stack based on the dictated size in bytes (rounded down to the nearest whole word).

Third: From what I can see, the only place an RTP_HEAPERROR run-time error can be displayed without additional information accompanying it is via StackFrameCreate(). The other two routines from which it can be triggered [which are FlexError() and BlkValueError()] have preceding print statements that should have shown up in your output.

The StackFrameCreate() routine is so simple that it’s hard to imagine a bug. The implication is either that an unexpectedly large size parameter is being fed to it, or (much more likely) that the I7SFRAME pointer is at its lower bound when the call is made. The latter implies a recursion error in your code in which an infinite number of stack frames are being generated. Many things use stack frames (e.g. rules, phrases, block value manipulation), so it’s hard to know what the precise problem is without more information.

If the addition of a new object is the proximate cause, and going north is the trigger, I would look for some interaction between rules governing the going action and any phrases that you’ve defined – such as a rule calling a phrase that directly or indirectly calls that same rule.

EDIT 2: For example, this reproduces your error:

To p1:
	let b1 be "temp1";
	p2.

To p2:
	let b2 be "temp2";
	p1.

Instead of going north:
	p1.

Does any of your logic change its behavior based on the total number of things, or the number of things in a room, or the number of things in player inventory, etc.?

5 Likes

I don’t do anything in my code for going north specifically or going in general, except:

Understand "go to" as going. Understand "go home" as leaving.

I don’t do anything related to numbers of things at all.

I placed your Include code at the top of my code and these are the results:

Without adding an object, I get:

>n

blockv_stack: 32171
new: 32619
finished!

blockv_stack: 32171
new: 32619
finished!

blockv_stack: 32171
new: 32619
finished!

and the game works properly.

If I add one simple object definition, I get:

>n

blockv_stack: 32346
new: -32742
*** Run-time problem P49: Memory allocation proved impossible.

at which point the game crashes.

By the way, I have no problem sharing my full code, if that is allowable (and if it will help).

Much thanks to everyone for trying to help me figure this out!

EDIT - Here is the object that I’m adding for test purposes that makes it crash:

The thingamabob is an object.

but it doesn’t seem to matter what the object actually is. It’s as if I’m at some kind of threshold and one more object tips it over.

You have reached what seems to be a hard-wired Z-machine threshold whereby this error is triggered if the stack frame pointer rises above 32767. That’s roughly where you would expect it to be for a Z-machine game compiled in 10.1.2 with ~70 total objects (each of which adds ~170 bytes to the sub-stack memory).

Compiling for 9.3 ( [Settings] tab → [Language Version] → choose ‘Inform 9.3 (6M62), December 2015’ ) would give you a lot more headroom (~256 total objects) before hitting this limit- each additional object appears to use only ~100 bytes in 9.3 and other (non-object) usage of sub-stack memory is also considerably less than what 10.1.2 appears to consume.

EDITED for a clearer description of the issue in view of Otis’s insight below.

3 Likes

What happens if you include the following (and remove the previous replacement)?

Include (-

[ StackFrameCreate size new;
	new = I7SFRAME - WORDSIZE*size;
	if (UnsignedCompare(new,  blockv_stack) == -1) { RunTimeProblem(RTP_HEAPERROR); @quit; }
	I7SFRAME = new;
];

-) replacing "StackFrameCreate".

It looks like the “top” of the block value stack is above the max signed integer, so the existing test is confused. (Here “top” means the high end from a memory address perspective.)

2 Likes

Ah yes. I see now that it’s more subtle than I thought.

The problem as you point out is that the start in memory of the block value stack array (blockv_stack) has got pushed up in memory so that the end of the stack (where the stack pointer I7SFRAME starts from, moving back from there through the stack array, i.e. downwards in memory) is above the 32767 byte (max signed integer) boundary, so the comparison test intended to check that we’re not about to move the stack frame pointer back beyond the start of the block value stack array misfires, triggering a spurious memory allocation error.

So it’s a longstanding bug in Inform 7 exposed by the memory-hunger of Ver 10, not a limitation of the Z-machine, and your fix should fix it :grinning:

2 Likes