I’ve been using lists for constant arrays in my project, but now I’m thinking they use up a lot of extra memory.
If I have a bunch of things like this:
list1 is always {1, 2, 3}.
list2 is always {“hello”, “hi”, “goodbye”}.
(etc.)
It seems like I hit thresholds where the variable/array memory jumps by 150k or so.
Is this a bad way to do constant arrays? Should I use tables or something?
(This is using build 6L38.)
Inform isn’t great about dynamically-resizable things being made constant: all lists still go on the heap, even if they’re compile-time constants. Which can increase your memory usage a lot, but more importantly can end up being many times slower than it really should be.
Tables, on the other hand, are literally just blocks of RAM (with fixed size known at compile-time). And RAM is super cheap at this point.
Ugh, I had thought that the whole point of allowing constant lists was to let Inform know they wouldn’t be dynamically resizable. Guess I was wrong. (In the old days I had a project that blew through z-machine memory limits with one room and nine things, because I had a bunch of tables with a list column).
The docs (21.2) even say you can change them. Calling them “constant” lists is a misnomer, it should call them list literals.
Inform recognises that “{1, 2, 3, 4}” is a list because of the braces, and looks at the entries inside, sees that they are numbers, and deduces that it is a constant whose kind of value is “list of numbers”. L is then a temporary list variable and we can add to it, remove things, and so on as we please - {1, 2, 3, 4} is merely its initial value.
I like to use tables a lot. One of the big things is that they are so well organized visually. I also like to put tables in a separate header file, which doesn’t save memory, but it has helped me organizationally.
I know for one of my games I had a list of random text because that seemed like the way to go, but I changed it to a one-column table that was easier to sort (automatically or with a script) and weed out duplicates. This has meant much less of a headache for me. Another bonus was, if an item threw an error, I could see which one instead of seeing “Oops, there’s an error in one of these 100 entries! Now which did I add recently?”
If tables get very big, you may have to change compiler constants to accomodate them, but that’s a minor nuisance.
ETA @MattW: I’ve had bigger games that did not trip the memory limit. Was the problem that some of the strings were very long, or there were a lot of table entries? I know I didn’t have a problem adding a lot of tables to my 2017 IFComp entry, but it mostly used variables instead of long strings.
I have to admit I don’t know what adds a lot to the memory. I’ve done some testing with a switch Zarf showed me …
For those who don’t know, adding this tells you when you hit z-machine limits, and you can add/subtract code as needed to see what takes up a ton of space.
Include (- Switches z; -) after "ICL Commands" in "Output.i6t".
'Twas definitely the number of table entries that were lists; zarf explained to me that every single list had to have memory space reserved for it on the memory heap (if I’ve got that terminology right). The game more or less entirely consisted of tables that had a list-valued column; I think there were fifty or sixty lists all told.
@Dannii: I see that 21.2 has some pretty misleading language about “let” vs. “now”:
On the other hand, writing
let M be { };is fine provided that M already exists, and then does the obvious thing - empties M.
This is only fine if M was created as a temporary variable in the same code block. If M is a global, then you’ll run into the usual issue where Inform tries to create a separate temp variable named “M” (and in this case, it will fail because it doesn’t know what kind of value is in the list). In general, for whoever is listening and needs to know this, setting the value of an existing variable should always be done with “now” and not “let.”
Thanks.
To clarify, are tables the only way to do fixed-length arrays?
They’re the standard one in I7. I6 has standard one-dimensional fixed-length arrays like in C, which are equivalent to tables but accessed differently; usually these aren’t used except in pure I6 code (like the parser).
This, in part, is an unfortunate consequence of compiling down to Inform 6, which puts all user-defined arrays in Z-machine RAM.
Infocom put a lot of theirs in Z-machine ROM, which ZIL supports with the “pure” flag on table definitions.
I thought I7 had some optimizations to avoid the standard memory management for constant heap values, though – maybe those are only for some types? Or maybe they don’t solve this problem, because the constants are copied into the regular heap as soon as they’re assigned to a variable? (Copy-on-write would really help here.)
The code for the heap and the kinds that use it has terrible performance because it’s still trying to work on the z machine.
If it switched to using Glulx’s malloc then it wouldn’t need to worry about discontinuous blocks, and could be much faster. Flexible lists would still be slower than a fixed size array though.