Memory, Performance, and Maximum Objects in TADS 3

Authoring TADS
1

I’ve been working with Inform 7, and discovered that it is possible to hit a very human-reachable limit on the scope of how big I might program the game. Of course, there’s a limit to any application, and I don’t need “infinite” space, but I don’t want to work in a system where I am limited to what I might reasonably try to create. I’m looking at TADS 3 as an alternative, and it looks like it is far more difficult to learn and less well documented than Inform 7, so before I expend significant resources learning a new language/system, I want to make sure it will do what I am hoping to do.

Is it feasible in TADS to create an application with hundreds, maybe a few thousand, unique persons, and hundreds, if not tens of thousands, of unique things? Is it possible to do this without making the application have difficulty compiling, or where the performance of the application becomes unacceptable?

I may have reusable classes of things as well, but many things will need to have persistent records. From what I understand, TADS doesn’t keep all objects in memory all the time, and objects should be able to be created dynamically. However, I want to make sure that the objects that are not currently “in use” can still be altered when they are in use, and be recalled with the changes, not set to a “respawn” or “default” state. At a high level view, is this at all feasible in TADS 3?

1 Like
2

At that level of abstraction, it sounds feasible in TADS 3. So far as I’m aware there aren’t any built-in limits to the number of objects a TADS 3 game can have, so you’d basically be limited by the resources of the machine you were working on. That said, I’m not sure anyone has tried to write a game quite as large as you seem to be envisaging, so I don’t know whether there’s any practical experience of running up against limits in TADS 3. The only one I’ve noticed is that a game that has a lot of objects (100+ or so) in scope at once starts to slow down noticeably. So far as I’m aware this shouldn’t be a problem otherwise, but that might depend on your game code (e.g., if you have thousands of persons in your game and you write code that loops through every single person every turn, this might cause a bit of a slow-down; indeed, if all these persons have some kind of autonomous behaviour that has to be checked and executed each turn, this is almost bound to slow things down).

You certainly can create objects dynamically in TADS 3, and the ability to do so may obviate the need to have thousands of persons or other objects in the game at once.

Compiler speed in TADS 3 is helped by the fact that you can split your source code into separate modules and (apart from the occasions when you need to do a complete compile) the compiler only needs to recompile source files that have changed since the previous compile. In general this makes the compiling of a TADS 3 game/story quite a bit faster than that of an I7 game of comparable size.

As the author of much of the TADS 3 documentation I’m probably not the best person to comment on that. I hardly think TADS 3 is “less well documented” than Inform 7 if you’re thinking of the quantity of documentation available. The perception of quality seems to be more of a matter of taste. Whether TADS 3 is “more difficult to learn” than I7 in part depends where you’re starting from. If you’re already familiar with a C-like language then TADS 3 will seem a lot more intuitive than if you’re not. Again it’s not uncommon for people to remark that while I7 is easier to learn initially, that advantage tends to disappear once you want to do more advanced stuff - but again I think you’ll find opinions vary. I think it probably is the case, however, that TADS 3 is generally more approachable on Windows (where you can use the Workbench IDE) than on non-Windows systems where you have to use command-line tools. Again, whether that’s an issue depends on your tastes and previous experience.

One further thing I should point out is that if you’re thinking of learning TADS 3 you might want to take a look at adv3Lite, which is intended to be a bit easier to learn and use than the adv3 library that comes standard with TADS 3. It also has a number of I7-like features (such as Scenes and Regions) you won’t find in the standard adv3 library.

4 Likes
3

Thank you for your detailed reply.

I would like to follow up on a few things. I would never want to have anything even close to 100+ actors and thousands of items in scope at once. Even professional, modern game engines struggle with that and that would be far too complicated for one person to write anyway. I don’t intend to be releasing such large amounts of content in the alpha, or even 1.0 of my game, but what I’m looking at is that if I like the world I am building enough, I hope to be able to release modular expansions over time. I don’t have any delusions of grandeur about the final scope of my game, but I want to know that I have the freedom to work on it for the next 40 years if I so choose, as a novelist might on a series in a setting, and continue to expand it without fear of technical failure because I ran out of room.

Rather than having everything in scope, indeed I am wondering if it is possible to bring things in and out of scope, but retain information in data about the out of scope objects, so that when brought back into scope, they can reflect the changes that happened to them the last time they were in scope, instead of returning to a default state. I see that you can create dynamic objects, but the only example I’ve understood so far seems to create an instance of a class, a “new”, but I didn’t yet find an example I understood that applies data from an out of scope source file to update the parameters of the new object to present itself to the player as something they’d interacted with previously. I’m still not clear from your response above if that is doable.

The reason I say the documentation isn’t as good for TADS 3 may very well be that the entrance fee is steeper, but that later on it gets easier. That’s still a factor though. Using no library at all, TADS is a lot like C. I could figure out how to add another function to main(), with no library easily, but even after 30 minutes of reading and searching, I could not figure out how to make something happen at the start of play in adv3… the equivalent of “When play begins” in Inform 7. Generally, I would hope that figuring something like that out would take about 2 minutes with the searchable internet as a resource, but I still don’t know how to do it, and I program C# and Javascript for websites for a living. In any case, it’ll all be worth learning if it is capable of what I am outlining above, I’m just still not 100% convinced yet… I spent a year on Inform 7 only to find that it can’t do what I need it to, so I don’t want to repeat that experience.

Edit: I did finally figure out how to run custom functions at startup, but it was just almost impossible to search for that specific info. Had to sift through many dud search results to find it. It seems I was looking for “Pre-initialization objects”.

1 Like
4

I’m not sure what you want to accomplish, but take a look on the chapter 9 Beginnings and Endings in the Learning TADS 3. You can add code to the showIntro() method. Then you could also look into TADS 3 Library Reference Manual for class GameMainDef, where you can click to see actual source code of the library. There you’ll see that showIntro() is called from newGame() method and there are other methods nearby like restoreAndRunGame() or showGoodbye() which you could modify to do other things in addition to their normal behavior.

But probably more cool way is to use InitObject or PreinitObject classes whose are documented in chapter 8.5 Initialization and Pre-initialization of the same book.

2 Likes
5

I’m still not 100% sure I fully understand what you want to do. I also suspect we may be at cross-purposes since I was talking about scope in the Interactive Fiction sense (i.e. something the player character can interact with in his or her current location) while you may be thinking of it in the programming sense (e.g. the scope of a variable).

If you create a class instance via the new keyword, then you will indeed lose any data stored on that instance if the object goes out of scope in the programming sense (since it will be deleted by the garbage collector). The trick is to keep it in (programming) scope by keeping a reference to it somewhere, such as an object property. In outline, you’d could do something like this:


myDataStore: object
    widgets = static new Vector(100)

    addWidget()
    {
        local wid = new Widget;
        widgets.append(wid);
        return wid;
     }
;

Since myDataStore is a permanent object, it maintains a reference to the new Widget you’ve created in its widgets property, so the new Widget never goes out of (programming) scope, and any data changed on it is preserved (of course the code given is only a toy outline; in a real game you’d probably do something more complex).

Alternatively, a TADS 3 game can save data to a file and read it back again. (But I believe an I7/Glulx game can do this too).

As tomasb said, the information is there in the documentation and there are various ways of accomplishing it. The trouble is, it’s not the first thing most TADS 3 users need to know, since there’s no point learning how to change things at the start of play before you’ve learned what things there are to change and how to change them (i.e., rooms, things, and the basics of working with properties and methods).

2 Likes
6

Not to hijack this thread, but I personally find tads much easier than inform, and the documentation is great, imo.

I’ve got a fairly good sized game in the works right now, but it’s no where near the sizes you’re taking about. That said, I’ve never seen any limitations any time I wanted to do something with tads.

3 Likes
7

I am speaking of it being in scope in the programming sense. The problem I ran into with Inform was that every object in the game was always in programming scope. So, the more things you want in the story, the more things the game had to compile and allocate to memory at one time.

Assuming that the object you created in this example isn’t going to bog down the application when not being accessed… this is what I am trying to determine entirely. So, with Inform, again, if I create a “class” (not really Inform 7 lingo, but behind the scenes it’s the same) of object, and pull one in, assign it properties, and then reassign it other properties, the original properties are lost unless I store them in a table or some other object. What I came to realize is that the other object that is storing the data is just as taxing on the program as having the object be a persistent, static object to begin with. Nothing is gained, and instead I’m just shuffling around parameters for no reason at all.

I am not a professional game developer, so perhaps I’m finding it hard to ask the question in a concise way that makes sense… but the idea of storing data “outside of scope” is kind of what I’m getting at. Let’s say I have 100 static objects, or a pool of dynamic objects limited to 100 maximum allowed… whichever… but I want to make those APPEAR to be thousands of things to the player. If I use an example like yours above, am I going to need thousands of widgets that are persistently in scope and taking up memory allocation in order to keep records of the changes?

I guess the question is whether the data can be stored “navitely” to the application without being “in active scope”. With Inform, you can store the data externally, but you have to define a file reference for each table manually, and this information is then not able to be shared between “saves.” You’d have to literally have the user copy the entire game directory to a new folder to allow them to have more than one “save” of the game, because only one data file reference per game file is allowed.

Argh… I don’t know how to specifically ask what I’m asking, but maybe the rambling above makes more sense of it, I don’ know…

8

It seems that it’s quite hard question and maybe one of the reasons is that the question is quite abstract. We don’t know what the objects should represent in game and therefore it’s quite hard to imagine any solution. So I’ll try to explain some principles behind, but I’m not sure I’ll succeed with explanation.

When you want to track state of some object, that means the object and it’s properties should exist during all game, it must reside as a programming object in TADS’s virtual machine memory. That way the object is preserved in virtual machine, it is saved and restored in a game state and also that way it is automaticaly versioned in the sense of undo command. When player saves the game, basically the contents of internal memory is serialized in binary format to a file on hard drive.

So for an object to be part of this process, it should reside in TADS’s memory. To overcome this by saving some objects elsewhere (i.e. in disk file) would be really hard, because you must account for what to do when player saves and restores a game and what to do when player undos a move. I suspect that these problems can quickly show that it would be impractical to try do something like that.

On the other hand todays computer memory tend to be quite big and steadily improving, while on the other hand today players still have quite the same capacity of how big game world they could possibly navigate and understand. So it’s not easy for me to understand the problem keeping all objects in memory.

But there are already situations where it is desired to conserve memory and not to waste it without purpose. For example TADS’s library already implements multiple objects by one proxy in some situations - look at the MultiLoc or RoomParts which are largely shared between rooms and only pretend to be different objects on the different places. I can imagine a game using many NPCs which are esentialy same differing only in few details to be represented by some similar meta object which pretends to be multiple individuals only parametrizing/randomizing few properties. Of course this would require quite a good understanding of the TADS and it’s library and it would be a lot of work, but I think it shloudn’t be impossible (in contrast to storing objects outside of memory).

3 Likes
9

I might just have to learn a little more about TADS and run some experiments before I can ask a more answerable question.

I’m trying to create a test application now that creates a bunch of random rooms, actors, and things and places them all randomly. Then I’ll run around and change a bunch of stuff with simple actions, and see how that goes.

I already made an application with 5000 “dummies” (actors with only the base Actor class properties) and though that was taxing on performance, the application even still worked. I could even have 500 dummies in one room, and no performance impact seemed evident. This is actually already light years better than the limitations in Inform, although I don’t know if a test of Actors with nothing special about them is much of a conclusive test. Still, in Inform, any more than 600 base “persons” would crash the compiler, let alone what just under that would do to performance.

Maybe TADS is robust enough that there isn’t a limit that I will reach to begin with?

10

Nope.

11

Well, I don’t know if “crash” is the right word, but it is unable to compile. With default memory settings, even in Glulx, 500 things is fine, 600 will not compile.

You can then increase the memory settings, and get it about 3500 things or so, and as far as I can tell, increasing the memory settings higher has no effect… beyond this point, Inform cannot compile. I admit, perhaps I don’t know how to set the memory settings, but I’ve asked numerous times on the Inform board about how to maximize these settings, and for a few months have been trying to get more out of the application, but just can’t get over that hurdle.

I did a comparable test in TADS, creating 10000 base actors, and though at those high numbers things did slow down, everything compiled, and I could even place the PC in the room with the 10000 dummies and use “look” without it crashing.

I realize I could be mistaken, as I don’t really know Inform inside and out, but based on all of my experience, it isn’t able to handle thousands of objects, and doesn’t have a viable way to fake this without completely rewiring the language.

All I’m asking about TADS is the same question I started asking about Inform… what are the upper limits, and how can I test them.

This is also all well and good for a game that does nothing but have a bunch of base objects in it, in either TADS or Inform… it remains to be seen what happens in TADS when trying to do real things, but I had a large application going in Inform which got to the point where the performance for taking any action was slow, and adding even a single object to the game made it not compile… and I hadn’t even made any game content yet, that was just my “framework” code.

Maybe TADS can’t do it either, but it seems like it is going to handle more based on my initial impression and the fact that it seemed to handle 10000 things without too much trouble.

12

I don’t have any benchmark tests on this subject, and I rather doubt anybody does. My halfway educated guess, however, is that for the kind of project you’re envisioning, you may be better off in T3 than in I7.

Referring back to your original question, though – about a system that would allow you to add refinements to your game for 40 years – I don’t think that’s a realistic expectation. Computer technology is going to go through vast, sweeping changes in that period of time (assuming our entire civilization doesn’t crash and burn long before then). IF development systems are maintained by tiny, dedicated teams of volunteers, with the emphasis on “tiny.” Whether anybody will feel like updating either T3 or I7 twenty or thirty years from now so that it will operate in whatever machinery is in common use at the time … there are no guarantees at all.

Five years is a reasonable distance to look into the future. And my own experience writing IF suggests that the limiting factor in a five-year project will not be the number of objects an authoring system (or interpreter software) can support, but rather the amount of time you have to write and test the code.

In sum, I can do no better than quote Bobby McFerrin: “Don’t worry. Be happy.” I realize that’s not an answer to your question, except in the broadest sense. Just my two cents.

2 Likes
13

IIRC, the closest one was Finding Martin, which necessitated a change with the TADS interpreter at the time.

There used to be a web page featuring the game, with a note about using the TADS runtime with a particular memory-related setting. Eventually Mike Roberts folded this change into the subsequent TADS Player version. (It’s been ~9 years since the game was released, and I can’t find the page anymore through searching - most IF roads are leading to IFDB now, so to speak.)

14

I realize that this is an old thread and I’m not sure if this casts any light on the memory problem fix used for “Finding Martin,” but there is a thread about those technical problems at https://groups.google.com/forum/#!search/%22finding$20martin%22$20memory$20setting/rec.games.int-fiction/Zj2pbCSYTLk/AkYjo7EeiQ0J. The home page for Finding Martin is now at http://www.gkwmusic.com/fm/home.html.

15

Long post (sorry in advance - longtime computer programmer and lots of thoughts to share on this issue).
Been writing Tads 3 code for years - also tried Inform 7. With Tads 3 you do not need to worry so much about running out of memory. With Inform 7 you ABSOLUTELY DO need to worry about that stuff. Reason is - Inform 7 is actually Inform 6 (at the memory management level) which is outdated. This is similar to Tads 2 being like Inform 6 in that at some point back in the old days of coding in those languages you DID have to worry about memory management issues on larger games. This is why Tads 3 is superior to Inform 7 in that regards already and I have written massive game code in Tads 3 without ever a problem so far (though you won’t see my games typically published on the main IF websites as it’s usually on the TG/TF Games one under Cleo Kraft as the author). That said, I do keep offline games in progress so far unpublished which I’ve added to gradually over the years which have expanded to crazy scale sizes and Tads 3 does not at all have any problems with it so far, even with multimedia and all the rest and TONS of redundant spaghetti code in between.
I think I asked Mike Roberts about this same thing at one point and he replied, if I recall correctly, that there shouldn’t be issues. Tads 3 was written in C (I think) and has all that memory management stuff written in for you, so you don’t have to drop in weird lines of code at the start of your text adventures to handle this in your games in a Tads 3 game, unlike in Tads 2 or Inform 6 or Inform 7.
I hope this information helps clarify and resolve the “memory” issue you were worried about in your original post/question. Similar to Java - Tads 3 is just designed to handle it, so far as I’ve seen and understand. I mean - no “garbage handling” like Java per-se, that I know of, but it does it’s best, which means it’s machine limited, not code or software limited.
Another thing to look into is program jumping from one file to another. This used to have to happen a LOT in the old days of BASIC programming, for example, where you’d reach the end limit of one program, save the game sate, and load it all up and run things all into a new program. If you ever did come near to exceeding program size limit that old days version of jump code is an easy solution on how to handle it. Just run it in chunks from one file to the next and tie them in all together to “simulate” one giant program. I don’t think we have to worry about that these days though. It’s usually hardware limited now and from here on out - not disk space or anything like that. Conceivably, though, you could use that old jump scheme to bypass enormous amounts of memory issue concerns but if you do that you’re going to have to start using room numbers for your room locations vs. room object names if you don’t want to have to re-define large swaths of old room objects in each chunk load.

2 Likes
16

Inform 6’s memory management was completely rewritten in 2021. Most of the issues you’re thinking of are fixed.

I should say: the Inform issues mentioned in this 2014 thread have all been fixed.

2 Likes
17

This is VERY good to know. Thank you! You said “Inform 6” was updated? If that’s been updated then that’s a big deal in the IF community. I wish I’d known sooner then because Inform spans more platforms than Tads and I’ve been stuck on Tads3 with this issue for years so far.

18

Yes, that was Inform 6 version 6.36, released January 2022.

(Sorry, I said 2021 above – that was when I did the code changes. The next release after that was Jan 2022.)

19

TADS3 does have explicit garbage collection. Most of the time it stays more or less out of the implementor’s way, but you can manually force garbage collection via t3RunGC().

This usually isn’t necessary, but since under the hood T3 handles garbage collection via refcount you can run into situations where a firstObj()/nextObj() or forEachInstance() loop will either include or not include an otherwise-unreachable object depending entirely on when garbage collection was last run.

You also run into a lot of issues that are due to the design of the virtual machine (and interpreters) and not the underlying hardware, like brutally slow floating point performance. And there are a bunch of things that the compiler will complain about by default but which can be ignored via compiler flags (like having constant values larger than 16 bits).

There are also miscellaneous limits on e.g. object counts that (as far as I can tell) aren’t enumerated in the official documentation and produce unpredictable results when you get anywhere near them. These are in the ten thousand plus range, so the vast majority of games will never run into them, but they’re absolutely not limitations due to the underlying hardware.

3 Likes
20

“CLEO KRAFT” ?!

Welcome here ! and thanks for sharing the source code of Diabolical, IMVHO one of the best set of source code, and the very first IF with a postgame !

On “crazy scale size”, if diabolical is to be taken as reference, I think you’re not joking.

A tip on expanding: I have a long-term WIP, but I have committed to a timetable, placing some definite limits.

Back on topic… I confirm Zarf’s post, but, because you came from a definite coder’s background, I suspect that TADS 3 perhaps remain more amenable to you.

Best regards from Italy,
dott. Piergiorgio.

2 Likes