Dialog


#101

Perfect !

What are the pros / cons of the two different approaches ?

Please elaborate on “(though not necessarily a good idea in this situation)”.

I found a couple more variations that might be more useful :

(ShowAnyX $Obj)       
	(exhaust) {
                *($Part has parent $Obj)
               (line) $Part is part of $Obj
	  }

#thing1
(descr *)
     (ShowAnyX #thing1) 
%% --- or ----  
      (ShowAnyX *)   %% no need to repeat the obj name from within \(descr \* \)

(Björn Paulsen) #102

So I tried some variations of Cloak of Darkness (z5) on my Amiga. I’d expected the ones produced by I6 to be slower and bulkier, but was surprised to see Dialog and I6 neck-and-neck in terms of both file size (bit over 80 kB) and execution speed. More puzzling was that the original CoD file found at Roger Firth’s page (built under 6.21 with the 6/10 library) was 50 kB in size and ran slightly but noticeably faster.

This intrigues me because I was always annoyed by the sluggish pace of IF on the Amiga, and I’m trying to see where I can shorten the response loop. Dialog is promising, and I’d expected the turn loop to run quicker. Is I6 really that efficient, or does a minimal compile (such as Cloak) in Dialog really encode much more functionality than the I6 version? Note that I’m not throwing shade on either compiler; I’m merely looking for points where optimization can be done in a more trivial manner than rolling my own z-machine terp in Asm.


(Linus Åkesson) #103

It’s mostly a matter of readability. Moving a chunk of code into a separate predicate can sometimes be helpful, if the name of the predicate helps to clarify the purpose of the code. But the downside is that the reader has to jump to a different part of the file to see what the predicate does. In this case, especially with a placeholder name such as ‘(showAll)’, I just couldn’t see the benefit.

However: When you’ve got identical or very similar code in multiple places, it can be a good idea to put that in a separate, well-named predicate, just to avoid having to maintain multiple copies of the same thing. Your ‘(ShowAnyX $Obj)’ is a potentially much more useful predicate, although it probably needs a more descriptive name.

All of this is subjective. From a technical point of view, having multiple copies of the same code can make the story file bigger, but in this particular case it would be negligible.


(Linus Åkesson) #104

Thanks for this benchmark! I’ve been meaning to check how Dialog code performs on the Amiga, but so far I’ve been using Tethered on C64 as my go-to measurement, so I’ve really only looked at relative improvements across compiler versions.

A couple of points:

First of all, yes, I6 is really efficient. It’s a low-level language in a number of senses: What you write is structurally pretty close to what the Z-machine needs to do at runtime. There is no type-safety, array bounds aren’t enforced, so you can mess up the memory in ways that lead to mysterious bugs much later. When you are working this close to the machine, you get to decide how the various story elements are represented internally, and representations that are faster at runtime also tend to be more straightforward (easier) to implement.

On the flip side, the development process becomes less flexible. For instance, once you’ve decided to represent a property as an object flag, it might be difficult to adapt the code to some other representation, such as a three-way property, or a value that’s computed on the fly with a function call. In terms of its level of abstraction, Dialog is more like I7, and the goal is to achieve a good performance while allowing the author to focus more on the story, and less on the implementation details.

With that in mind, I’m quite happy to be on par with the performance of I6. But I’m not done yet. There have been performance improvements over the last half-year, and I’m sure there will be plenty more. Writing an optimizing compiler is a never-ending job, but it can be really fun! I think much can be gained by implementing a more thorough type inference, for instance.

Since your benchmark is based on a very small game, a large proportion of the result will be due to differences in the standard libraries. The parser in the Dialog library is more flexible than the parser of I6/I7. Not necessarily more powerful in a given situation, but more flexible for the programmer, and capable of disambiguation at the verb level, for instance. As for the jump from 50 kB to 80 kB you mention, I suspect that it’s mainly due to changes from I6 lib 6/10 to lib 6/12. A quick glance at the ChangeLog for 6/11 shows that a number of extra features were added.

One final possibility, and this is somewhat fuzzy and speculative: The Z-machine interpreter you’re using on the Amiga was presumably developed in the 90s or early 00s, when nearly every game released for the Z-machine was coded in I6 or ZIL. As a result, the interpreter may have been optimized (intentionally or not) to deal particularly well with I6- or ZIL-generated Z-code. When I was creating Zeugma for the C64, I really wanted to be able to run recent I7 games. One of the things I did was to have the interpreter check for a particular, often-called routine that’s present in every I7 game, and replace it with a native version coded in 6502 assembler. Eventually I had to admit defeat and give up on I7 support, but that feature is still in there. Now, I’m not saying that your Amiga interpreter contains something like that, tailored to a particular version of the I6 compiler, but I’m saying that its developers must have been fueled by the same desire to run recent games well. Hence, they would have spent a lot of effort on speeding up the kind of operations that dominate I6 code, while leaving the rest of the interpreter more or less unoptimized.


(Björn Paulsen) #105

That’s quite interesting. I have to admit I discounted the idea that a more flexible structure might force certain design choices. I’m excited to see where Dialog is going. Would types be exposed as a syntactic part of the language, or would it be an under-the-hood thing?

My pleasure. Honestly, though, I’d hardly dignify the comparison as a “benchmark”: I only attempted it on an A1200, had no concrete metrics, and the environment I used was not really what you’d call pristine or controlled. I’d call it an indication, no more.

As for optimization, I’ve been peeking through the Frotz sources, and while no obvious signs of fine-tuning leaped out (it was mainly platform-agnostic C, zero Asm includes so far), that only means the bottleneck might be due to the stack-passing overhead typical of C on the Amiga (which seems possible, given that each opcode is a function pointer). Conversely, it could also be down to expensive OS 1.x system calls, or the multitasking environment, or any of a dozen other factors. I may just have to bite the bullet and measure properly.


(Andrew Plotkin) #106

That’s all true. It’s also true, and I think more relevant here, that the I6 parser and world model are also written very close to the metal. They’re built to rely on Z-machine data structures and avoid any operation which is expensive (like managing dynamic lists).

The cost is that the I6 authoring environment is sharply constrained. Some authoring features just aren’t on the table.

When Graham went to I7, he consciously didn’t take “I7 should be as efficient as I6” as a goal. I don’t mean there’s no optimization in I7; there’s plenty. But I7 offers a much more flexible authoring model, and it runs slower than I6 in the best case. These facts are connected.


(Linus Åkesson) #107

Oh, I was referring to the built-in types (object, number, dictionary word, list, reference). It’s nice to be able to use any value (mostly) anywhere, so that, for instance, there can be a rule for ‘(the $)’ that takes a list and prints a description of the entire collection. But it’s probably the case that a lot of local variables only take on values of one particular kind, and the compiler might be able to deduce that. In those cases, it should be possible to eliminate certain runtime checks, and this in turn could lead to a speedup.


(Linus Åkesson) #108

Release notes for Dialog 0e/02, Library 0.21:

Library change: Objects at room boundaries are attracted.

Objects around the perimeter of a room, as defined by ‘(from $Room go $Dir to $Obj)’, are now automatically attracted into the room; they become floating objects. This used to be the case for doors, but now it is also true for any non-room, non-direction object mentioned in such a rule.

Library change: Scope

Objects around the perimeter of a room, including neighbouring rooms, are now only in scope if the player can see them. The current room is still always in scope (and can be referred to as dark/darkness when the player can’t see).

Library change: Disambiguation

Object-based disambiguation—when the library asks if the player meant to do a thing to any of a given list of objects—is handled differently: The answer from the player is now matched against the output from ‘(the full $)’, i.e. the actual words from the printed list. Furthermore, the answer is regarded as an elaboration of the previous, incomplete action, rather than as a new action. This makes UNDO behave more intuitively.

Library change: Internal performance tweaks

The library no longer maintains a list of objects in scope. The earlier approach had a performance advantage until 0e/01 introduced ‘(determine object $)’, but now it is faster to compute ‘($ is in scope)’ on the fly.

In contrast, the current visibility ceiling is now tracked using a global variable, and the ‘(player can see)’ predicate is a global flag. The library updates them as required.

Compiler: Minor upgrade.

Various performance improvements and bugfixes.


#109

Hej! So I only heard of this project yesterday and gave it a try last night after skimming the documentation. Looks very promising. Compiling in Windows 10 was a breeze, using the built-in linux sub-system with Ubuntu installed (essentially just sudo apt install gcc make, make;even apt-installed mingw and compiled the windows EXE… development in Windows have never been better than now).

The uuid warning was a bit annoying, but again apt came to the rescue with a quick “apt install uuid” and a one-line make-rule to create a uuid-file for each dg-file if it did not exist and automatically put that on the command-line for my %.z8 make-rule. 10 minutes and I think I have a reasonable Makefile already. No issues with the command-line interface so far.

My memories of Prolog are mixed. I took an undergraduate course in Prolog ~20 years ago. I thought it was really neat at first, but then when actually trying to implement anything it kind of turned into backwards LISP and having to constantly worry about exactly what the resolver was up to and adding cuts in the correct places. Eew. Was looking at some logic programming library for Clojure more recently (core.logic or something? based on some similar scheme library) and it was much more declarative, but probably with some other downsides. It looks from all the examples I saw so far that the higher-level Dialog code looks more like I would hope that logic programming would be, but the lower-level language stuff (that I am sure there is more of inside of the library) are more like my bad Prolog memories with carefully backwards-written code to get intended results (rather than “declare what you want to be true and let the computer resolve things” ideal)? It looks much more programmer-friendly than any of the Inform-versions I tried either way.

An emacs-mode would be nice. Anyone already have something like that, or know of some language that is similar enough that it would be a good starting point? I guess some automatic handling of indenting blocks somewhat similar to python-mode (but simpler) would be useful, but other than that not much would be needed? Maybe enable paredit or some similar minor mode.

One thing that immediately came to mind that looks like it could be built-in in some nice way is automatic testing. Ages ago when I last had an idea to write parser-based IF I wrote a perl-script wrapping frotz to play back walkthrough files with some lines being regular expressions that were matched against interpreter output in that location. (I think Inform 7 has something somewhat similar built-in?) It would be very neat if it was somehow possible to write tests that would execute lines of text and either just parse each line or (with some syntax) apply a line as a predicate to check the state of the game (or/and output). Maybe combine that with the debugger somehow. I saw that the debugger can already play-back saved scripts, but not sure if it can combine that with batch-running scripts and checking the results?

EDIT: I should add that I am not sure I want to go back to experiment with parser-based if, so maybe do not listen too much to my opinions, but I have looked for a sane way to compile choice-based games to the z-machine, and it looks like what would be needed to make that is already available in the Dialog language and that could be a fun project.

EDIT2: And now, a few minutes later, I have confirmed that it compiles and runs on my mac in OSX and on my raspberry pi in Raspbian as well. Very good work making it so portable and easy on the dependencies etc. No obvious issues making a runnable helloworld.z8 on either platform.


#110

RegTest is a generic tool for testing parser programs and will work fine for Dialog.


#111

I expected there to be something like that now, since it was 20+ years ago I wrote my perl-hack (and maybe there was something already I just did not find). But what I meant is that the Dialog design looks like it would be perfect to make something more integrated, making it possible to mix in Dialog predicates as tests for what state the interpreter ends up in, in addition to checking for substrings in the output. Have not looked at the debugger yet, but it looks like it runs on a level that would make it possible to do something like that there.


(Linus Åkesson) #112

I use scripts and makefiles for regression testing.

For game testing, I start with the transcripts from my testers and extract all the input lines (except save and restore). Then I pipe that into dumbfrotz and dgdebug, and store the output. After checking each output file manually, I copy it to a “blessed” file. Then, whenever I change the source code, I run all the tests and see how the output files differ from the blessed files (using meld). If it looks good, I type “make bless” to bless the new version of every output file.

The commandline version of dgdebug (i.e. not the windows version) checks whether its standard input and output streams are terminals, pipes, or files, and adapts. So if you tell it to read from a file and write to another file, it will do the right thing, deal with line endings and so on. Furthermore, it has a special “dumbfrotz quirks mode”, so you’ll get dumbfrotz-compatible output and random number generation. That way, you can keep a common set of blessed files for both backends. I run dumbfrotz with the options “-R mp -R lt -R ch1 -s 4242 -w 80 -h 999” and dgdebug with “-qD -w 80 -s 4242”. The “-q” tells dgdebug to terminate when the game quits, and “-D” enables the quirks mode. Note that these options assume a one-line status bar; your mileage may vary.

To save time, you don’t have to test with both backends each time. Dgdebug is faster, but it doesn’t truncate long dictionary words, and it won’t catch heap overflows that could occur when running on the Z-machine. So keep your transcripts dumbfrotz-compatible and run the full test suite every now and then.

What Inform7 integrates into its development environment is similar in spirit to the above. I personally prefer an open model based on text files, because it is easier to integrate with version control systems and other commandline tools, but that’s a matter of taste.


#113

Thanks! Those flags could be useful to keep around if I get around to actually write a story and good to know you considered this in the interface for the debugger. True that just checking all the output can be quite feasible, at least for other types of projects. I just recalled that is exactly what I do for gamebookformat for instance. Even if some bug slips through at least you get a nice history of saved outputs in version control if you need to bisect to find it later.


(Linus Åkesson) #114

Dialog 0e/03, Library 0.22 is out.

This release mostly features internal improvements to the compiler. The source code has been cleaned up, and a large part of the Z-machine backend has been rewritten. Several new optimizations have been implemented, so the generated Z-code tends to be a bit smaller and faster.

Library: Added ‘(them $)’ for printing a pronoun in the objective case (it/him/her/them) for a given object.