tables & topics & indexed text -- Oh My! (memory limits)

I could not for the life of me get part of the player’s command -> snippet -> indexed text -> text to compare to a column of table (long story, code snippets below). I could possibly have renamed the table column “topic” but I really didn’t want to do that (partly because it’s NOT a topic!). And I’ve been the value route, and didn’t really want to do that either – if I could avoid it.

So I bit the bullet and changed that first column of all my tables to indexed text. Then I hit memory limits – to the tune of having to raise memory limits tremendously to get to compile. Now my table-coping/populating routines caused a much longer delay than before (30 seconds on the Glulx welcome screen before the game start screen loads), and I’m just worried that I may have done something Very Wrong™ or unnecessary…

So here’s my current memory limits:

Use MAX_OBJ_PROP_COUNT of 256. Use MAX_PROP_TABLE_SIZE of 2000000. Use MAX_STATIC_DATA of 5000000. Use MAX_OBJECTS of 4000. Use MAX_ARRAYS of 4000. [wasn't sure I needed this, but it also looked like a safe bet to add it...]

A small sample table (I have longer ones with up to 6 columns – only the 1st column being indexed text on any of them…I may be able to do cross-reference tables on some of the other columns of information in the future but I’m trying to get it running to proof-of-concept for the tables before fussing with that):

Table of Node Stats stat (indexed text) rating activation is-attack is-program "rating" 0 false false false "matrix initiative bonus" 0 false false false "matrix initiative pass bonus" 0 false false false "response" 1 false false false "firewall" 1 false false false "system" 1 false false false "signal" 1 false false false "persona limit" 1 false false false "processor limit" 1 false false false "analyze" 0 false false true "rigged" 0 false false false "encrypted" 0 false false false with 6 blank rows

And one of the rules – the one I had to index the first column of the tables for – and the definition I created to change the snippet (the topic understood) into an indexed text:

[spoiler][code]

To decide what indexed text is the stattext (oldsnippet - a snippet) (this is fixingtext):
let N be indexed text;
now N is “[oldsnippet]”;
replace the text “’” in N with “”;
decide on N;

Understand “activate [text]” as activating a program.
Activating a program is an action applying to one topic.
Carry out an actor activating a program:
let thistopic be fixingtext applied to the topic understood;
if thistopic is a stat listed in the stat table of the actor:
if activation entry is true:
say “You remember that [the topic understood] is already running.”;
otherwise:
if the rating entry is greater than 0:
now activation entry is true;
if the actor is the player, reset player successes;
say “[1 ca][The topic understood] is now running.”;
otherwise:
say “You don’t have that program.” instead;
otherwise:
say “[The topic understood] is not a [if actor is a technomancer]complex form[otherwise]program[end if].”;
[/code][/spoiler]

(I can change things to Check…Carry Out…Report after I know each segment works…so it’s all in Carry Out because I couldn’t tell what was failing without everything being in one block so I could showme snippets and work out how to parse the text…)

So do I really have to change “the topic understood” to indexed text, hence change all the tables to indexed text, to find the topic understood in the ‘stat column’ of the table? I tried many permutations to try to get the text & table to match up before I found the example with naming Fido in the manual and decided to try indexed text. It seems to be a memory hog and to really slow things down a whole lot, though.

I’m getting pretty tired of re-writing everything to get it right…I know it’s the wise thing to do in the long run but I’ve been rewriting full-time for at least 2 weeks…It’s taking the excitement and fun out of the project. :frowning: At least I haven’t spent 3 days on one bug in a while… :unamused:

As far as I can tell, this seems to be how you gotta phrase things.

[code]Table of Node Stats
[declare stats to be topics]
stat (topic) rating activation is-attack is-program
“rating” 0 false false false
“matrix initiative bonus” 0 false false false
“matrix initiative pass bonus” 0 false false false
“response” 1 false false false
“firewall” 1 false false false
“system” 1 false false false
“signal” 1 false false false
“persona limit” 1 false false false
“processor limit” 1 false false false
“analyze” 0 false false true
“rigged” 0 false false false
“encrypted” 0 false false false
with 6 blank rows

Carry out an actor activating a program:
if the topic understood is a topic [***not ‘a stat’!!!***] listed in the stat table of the actor:
if activation entry is true:
say “You remember that [the topic understood] is already running.”;
otherwise:
if the rating entry is greater than 0:
now activation entry is true;
say “[The topic understood] is now running.”;
otherwise:
say “You don’t have that program.” instead;
otherwise:
say “[The topic understood] is not a program.”;[/code]

Why Inform refuses to compile with “if the topic understood is a stat listed in the stat table of the actor”, I don’t know. If it’s not a bug, it’s a pretty annoying feature.

EDIT: The code above fails, too (though it compiles). To make it work you mustn’t call the topics column anything but ‘topic’.

Table of Node Stats [***topics column has to be called "topic"***] topic rating activation is-attack is-program "rating" 0 false false false

For what it’s worth, Inform seems to recognize the reference to topics “by any other name” as part of the description of the action.
It accepts:

Carry out an actor activating a program stat listed in the stat table of the actor:

I understand all of this as a protection versus what the player might type on the command line, but now it’s ridiculous in the programming direction…

First, I want to thank you for your help! I really do appreciate it, because I never would have thought to change stat->topic.

However, when I changed (indexed text) to (topic) in the tables, other functions broke. :frowning: Most of the time the “stat” column is not something said by the player. There’s the activate and deactivate commands, to change one truth state, because that’s the only thing in the player’s control. So there might be a better way for me to go about this than to change every table to a topic column (I know I can change the whole column to values, but that also seems like a waste in this case, and means all the values are now in the entire program’s namespace which I don’t like). Perhaps this is best handled with an interim “Table of Things Actors Can Activate” table simply there for these 2 functions to catch the phrases/topics said by the player then do the real table lookup elsewhere (a table with a topic column + a text column that by some coincidence :wink: completely match both columns…). [Actually, the same table could also store a few more tidbits of information that are the same across all tables…]

Problem is see WI 15.13: “There is really only one operation allowed with topic columns, the “…listed in…” construction, but fortunately it is the one most often needed.”

In this case, it’s NOT the only operation I will ever need. Because it’s not always something said by the player. :frowning: When it is something said by the player, now it compiles. When it’s something I need referenced internally by the system, now it breaks.

Example: This is a simple debugging not-for-release function that broke:

Table-dumping is an action applying to one thing. Understand "tablecheck [a visible object]" as table-dumping. Carry out table-dumping something visible: if the noun is a person and the noun is not a technomancer: say "Current Matrix Boxes: [current matrix boxes of the noun][line break]"; repeat through the stat table of the noun: say "[stat entry]: ([rating entry]) [activation entry] [if is-program entry is true]program[end if][line break]";

This isn’t a make-or-break function, but shouldn’t I be able to print the column?

Changing it to “topic entry” didn’t work (exact same error), and the error I’m getting is odd. It seems you can’t simply print a topic column? [On this one, I got it past this compile error with my snippet-to-indexed text function (had to change input type from snippet to topic!) but there were other errors (see below).]

Error:

[spoiler][code]
Problem. You wrote ‘say “[stat entry]: ([rating entry]) [ac […] s true]program[end if][line break]”’ , and in particular ‘stat entry’: but this asked to say something of a kind which can’t be said, or rather, printed. Although this problem can arise when you use complicated text substitutions which come in variant forms depending on the kinds of value used, far more often what this means is just that you tried to use a substituted value (e.g., in ‘say “The dial reads [V].”’) of a kind which could not be printed out. For instance, if V is a number or a piece of text, there is no problem: but if V is a parsing topic, say an entry in a ‘topic’ column of a table, then this problem will arise.
I was trying to match one of these phrases:

  1. say “stat entry - text”
  2. say “[stat entry - number]”
  3. say “[stat entry - unicode character]”
  4. say “[stat entry - sayable value]”
  5. say “”: (" - text"
  6. say “rating entry - text”
  7. say “[rating entry - number]”
  8. say “[rating entry - unicode character]”
  9. say “[rating entry - sayable value]”
  10. say “”) " - text"
  11. say “activation entry - text”
  12. say “[activation entry - number]”
  13. say “[activation entry - unicode character]”
  14. say “[activation entry - sayable value]”
  15. say “” " - text"
  16. say “[if is-program entry is true - a condition]”
  17. say “if is-program entry is true - text”
  18. say “[if is-program entry is true - number]”
  19. say “[if is-program entry is true - unicode character]”
  20. say “[if is-program entry is true - sayable value]”
  21. say ““program” - text”
  22. say “[end if]”
  23. say “end if - text”
  24. say “[end if - number]”
  25. say “[end if - unicode character]”
  26. say “[end if - sayable value]”
  27. say “[line break]”
  28. say “line break - text”
  29. say “[line break - number]”
  30. say “[line break - unicode character]”
  31. say “[line break - sayable value]”
    This was what I found out:

stat entry = a table entry, holding a topic
rating entry = a table entry, holding a number
activation entry = a table entry, holding a truth state
is-program entry is true = a condition
if is-program entry is true = something unrecognised
end if = something unrecognised
line break = something unrecognised
[/code][/spoiler]

But other rules/actions/functions also broke…

[spoiler][code]
Problem. In the sentence ‘if stat entry is a topic listed in the new table begin’ , it looks as if you intend ‘stat entry is a topic listed in the new table’ to be a condition, but that doesn’t work because you use the listed in relation (between a snippet and a table name) with a topic instead of a snippet.
I was trying to match this phrase:

if (stat entry is a topic listed in the new table - a condition):
This was what I found out:

stat entry is a topic listed in the new table = a condition
[/code][/spoiler]

So now I have to use a function to translate to a snippet every time I want to look up something in the table, and a function every time I want to say the topic? I’m sure this will be kinder on the memory use of the game, but it’s not kinder on the programmer or readability of the code.

So I’m going to make that interim table for juggling all this.

Also thank you for your new post! :slight_smile: This is frustrating (some) but helpful (much more so!).

I won’t pretend to understand the internals, but as I understand it, you can’t print a topic column. Topics aren’t just texts but complicated functions involving texts; for instance “[any door]” and “[carried thing]” compile in topic columns. Or “monica/phoebe”, which matches either “monica” or “phoebe”. But you can’t just print either “monica” or “phoebe”, and I don’t think it’s represented internally with the slash intact. So there’s no straightforward way to print an arbitrary topic from a topic column.

I would like to reiterate something that capmikee suggested in the other thread but has perhaps gone overlooked. However, I will do so in somewhat stronger terms: Do not use texts / topics for this. Create a new kind of value for programs. Why?

  1. kovs are automatically parsable --no indexed text conversion stuff needed;
  2. unlike topics, kovs can be printed;
  3. this method is every bit as extensible as using texts; in this case maybe moreso.

You’re driving yourself batty unnecessarily – something I have often done with Inform myself. :slight_smile:

Here’s an example of what I mean:

[code]A program is a kind of value.
rating, matrix initiative bonus, matrix initiative pass bonus, response, and firewall are programs.

A person has a table name called the stat table.
The stat table of yourself is Table of Node Stats.

Activating is an action applying to one program.
Understand “activate [program]” as activating.
Understand “activate [text]” as a mistake (“You can only activate programs.”).

Check an actor activating (this is the can’t activate disallowed programs rule):
unless the program understood is a stat listed in the stat table of the actor:
say “The [program understood] program is not available to you.” instead.

Check an actor activating (this is the can’t activate running programs rule):
choose row with stat of program understood in the stat table of the actor;
if activation entry is true:
say “You remember that [the program understood] is already running.” instead.

Check an actor activating (this is the can’t activate programs with zero rating rule):
choose row with stat of program understood in the stat table of the actor;
unless the rating entry is greater than 0:
say “You don’t have that program.” instead.

Carry out an actor activating (this is the default activating rule):
choose row with stat of program understood in the stat table of the actor;
now activation entry is true;
[whatever you want that doesn’t print stuff here.]

Report an actor activating (this is the default report activating rule):
say “The [program understood] program is now running.”.

[Note that I declared response as a program above, but deleted it from the table below:]
Table of Node Stats
stat rating activation is-attack is-program
rating 0 false false false
matrix initiative bonus 0 false false false
matrix initiative pass bonus 0 false false false
firewall 1 false false false

test me with “activate boo / activate firewall / g / activate response / activate matrix / activate matrix initiative pass bonus”.

Lab is a room.[/code]

Note that there are still some issues: “activate matrix” is unrecognized as opposed to leading to disambiguation (which you probably run into with the existing implementation anyway) and you have to watch capitalization. These can both be fixed, I’m just not sure off the top of my head the best way yet. Also, I’m not convinced that tables are the way to go with this either, but I’m not a real fan of using tables in this manner, so I’m probably biased.

It can be done with values, with tables, etc. If the only place the tables breaks is when I have to parse what someone is saying, I just made a 2-column table where one column is topic, the other is text. The nice thing – that you point out, is that I can put some types of disambig in the topic column. [The player can’t “activate matrix” on any quality anyway – that’s an activity reserved for things with an is-program of true. If they heal, they use a program for that “medic me” or “heal me” which is an action that invokes the medic program.]

There are kinds of people and kinds of rooms with these stats. And some types of people /rooms can only have some of these stats. It becomes a true headache if you have to say “A node is a room.” and assign the same values used by people, when only a few of those 30-50 values overlap between people and rooms. So even if it ended up being vaules, it would probably end up in a table anyway. However for other housekeeping internal to the game, all these “stats” in one table makes it easier to program. Yeah, I could use values with values – but like I said I didn’t like the interference with the namespace in the rest of the program. When someone says “analyze joe” as a verb versus “run analyze” (the program) versus me programming what happens when the actor analyzes joe etc. – it got messy. I started having to come up with creative words in the program, rendering it confusing and less readable.

I think that the extra lookup table will make a big difference, and I’ve got a single place to keep the information that is stable no matter whom has the quality in question (i.e. a program is a program, whether a person or a “room”/node has it). What I needed was to understand why it had that problem.

And if I ended up having to switch to values – argh. But I’d live, I guess.

Do you have an example source text? I couldn’t get it to not compile, though it does trip over bug 707:[spoiler][code]There is a room.
Yourself has a table name called the stat table.
The stat table of yourself is the Table of Node Stats.
To reset player successes: do nothing.
To say 1 ca: do nothing.
Definition: a person is a technomancer if true is false.

Table of Node Stats
stat (indexed text) rating activation is-attack is-program
“rating” 0 false false false
“matrix initiative bonus” 0 false false false
“matrix initiative pass bonus” 0 false false false
“response” 1 false false false
“firewall” 1 false false false
“system” 1 false false false
“signal” 1 false false false
“persona limit” 1 false false false
“processor limit” 1 false false false
“analyze” 0 false false true
“rigged” 0 false false false
“encrypted” 0 false false false
with 6 blank rows

To decide what indexed text is the stattext (oldsnippet - a snippet) (this is fixingtext):
let N be indexed text;
now N is “[oldsnippet]”;
replace the text “’” in N with “”;
decide on N;

Understand “activate [text]” as activating a program.
Activating a program is an action applying to one topic.
Carry out an actor activating a program:
let thistopic be fixingtext applied to the topic understood;
if the topic understood [thistopic] is a stat listed in the stat table of the actor:
if activation entry is true:
say “You remember that [stattext the topic understood in title case] is already running.”;
otherwise:
if the rating entry is greater than 0:
now activation entry is true;
if the actor is the player, reset player successes;
say “[1 ca][stattext the topic understood in title case] is now running.”;
otherwise:
say “You don’t have that program.” instead;
otherwise:
say “[stattext the topic understood in title case] is not a [if actor is a technomancer]complex form[otherwise]program[end if].”[/code][/spoiler]

Neither of these sound like normal consequences of using indexed text—there’s probably something else going on. I’m assuming that your source text is fairly large, but if you’re willing to copy it into another story, pare off the parts unrelated to those symptoms, and post it, that’d be great. If it turns out to be an Inform bug, I’d like to have it on Mantis, and even if not, there’s still a good chance that one or both of those problems can be dealt with.

I was referring to the situation where the stat column contained topics rather than indexed text.

This doesn’t compile:

[code]Table of Stats

stat (topic) rating activation is-attack is-program
“rating” 0 false false false
“system” 1 false false false

Activating a program is an action applying to one topic.
Understand “activate [text]” as activating a program.

Instead of activating a program: if the topic understood is a stat listed in the Table of Stats, say “Gotcha!”[/code]

Change the last line as below and it compiles, BUT the topic understood never matches the stat entry topics! (This affects the code I proposed upthread.)

Instead of activating a program: if the topic understood is a topic listed in the Table of Stats, say "Gotcha!"

This, however, works as expected:

[code]Table of Stats

stat (topic) rating activation is-attack is-program
“rating” 0 false false false
“system” 1 false false false

Activating a program is an action applying to one topic.
Understand “activate [text]” as activating a program.
Instead of activating a program: if the topic understood is a topic listed in the Table of Stats, say “Gotcha!”[/code]