Dialog

#101

Perfect !

What are the pros / cons of the two different approaches ?

Please elaborate on “(though not necessarily a good idea in this situation)”.

I found a couple more variations that might be more useful :

(ShowAnyX $Obj)       
	(exhaust) {
                *($Part has parent $Obj)
               (line) $Part is part of $Obj
	  }

#thing1
(descr *)
     (ShowAnyX #thing1) 
%% --- or ----  
      (ShowAnyX *)   %% no need to repeat the obj name from within \(descr \* \)
(Björn Paulsen) #102

So I tried some variations of Cloak of Darkness (z5) on my Amiga. I’d expected the ones produced by I6 to be slower and bulkier, but was surprised to see Dialog and I6 neck-and-neck in terms of both file size (bit over 80 kB) and execution speed. More puzzling was that the original CoD file found at Roger Firth’s page (built under 6.21 with the 6/10 library) was 50 kB in size and ran slightly but noticeably faster.

This intrigues me because I was always annoyed by the sluggish pace of IF on the Amiga, and I’m trying to see where I can shorten the response loop. Dialog is promising, and I’d expected the turn loop to run quicker. Is I6 really that efficient, or does a minimal compile (such as Cloak) in Dialog really encode much more functionality than the I6 version? Note that I’m not throwing shade on either compiler; I’m merely looking for points where optimization can be done in a more trivial manner than rolling my own z-machine terp in Asm.

(Linus Åkesson) #103

It’s mostly a matter of readability. Moving a chunk of code into a separate predicate can sometimes be helpful, if the name of the predicate helps to clarify the purpose of the code. But the downside is that the reader has to jump to a different part of the file to see what the predicate does. In this case, especially with a placeholder name such as ‘(showAll)’, I just couldn’t see the benefit.

However: When you’ve got identical or very similar code in multiple places, it can be a good idea to put that in a separate, well-named predicate, just to avoid having to maintain multiple copies of the same thing. Your ‘(ShowAnyX $Obj)’ is a potentially much more useful predicate, although it probably needs a more descriptive name.

All of this is subjective. From a technical point of view, having multiple copies of the same code can make the story file bigger, but in this particular case it would be negligible.

(Linus Åkesson) #104

Thanks for this benchmark! I’ve been meaning to check how Dialog code performs on the Amiga, but so far I’ve been using Tethered on C64 as my go-to measurement, so I’ve really only looked at relative improvements across compiler versions.

A couple of points:

First of all, yes, I6 is really efficient. It’s a low-level language in a number of senses: What you write is structurally pretty close to what the Z-machine needs to do at runtime. There is no type-safety, array bounds aren’t enforced, so you can mess up the memory in ways that lead to mysterious bugs much later. When you are working this close to the machine, you get to decide how the various story elements are represented internally, and representations that are faster at runtime also tend to be more straightforward (easier) to implement.

On the flip side, the development process becomes less flexible. For instance, once you’ve decided to represent a property as an object flag, it might be difficult to adapt the code to some other representation, such as a three-way property, or a value that’s computed on the fly with a function call. In terms of its level of abstraction, Dialog is more like I7, and the goal is to achieve a good performance while allowing the author to focus more on the story, and less on the implementation details.

With that in mind, I’m quite happy to be on par with the performance of I6. But I’m not done yet. There have been performance improvements over the last half-year, and I’m sure there will be plenty more. Writing an optimizing compiler is a never-ending job, but it can be really fun! I think much can be gained by implementing a more thorough type inference, for instance.

Since your benchmark is based on a very small game, a large proportion of the result will be due to differences in the standard libraries. The parser in the Dialog library is more flexible than the parser of I6/I7. Not necessarily more powerful in a given situation, but more flexible for the programmer, and capable of disambiguation at the verb level, for instance. As for the jump from 50 kB to 80 kB you mention, I suspect that it’s mainly due to changes from I6 lib 6/10 to lib 6/12. A quick glance at the ChangeLog for 6/11 shows that a number of extra features were added.

One final possibility, and this is somewhat fuzzy and speculative: The Z-machine interpreter you’re using on the Amiga was presumably developed in the 90s or early 00s, when nearly every game released for the Z-machine was coded in I6 or ZIL. As a result, the interpreter may have been optimized (intentionally or not) to deal particularly well with I6- or ZIL-generated Z-code. When I was creating Zeugma for the C64, I really wanted to be able to run recent I7 games. One of the things I did was to have the interpreter check for a particular, often-called routine that’s present in every I7 game, and replace it with a native version coded in 6502 assembler. Eventually I had to admit defeat and give up on I7 support, but that feature is still in there. Now, I’m not saying that your Amiga interpreter contains something like that, tailored to a particular version of the I6 compiler, but I’m saying that its developers must have been fueled by the same desire to run recent games well. Hence, they would have spent a lot of effort on speeding up the kind of operations that dominate I6 code, while leaving the rest of the interpreter more or less unoptimized.

(Björn Paulsen) #105

That’s quite interesting. I have to admit I discounted the idea that a more flexible structure might force certain design choices. I’m excited to see where Dialog is going. Would types be exposed as a syntactic part of the language, or would it be an under-the-hood thing?

My pleasure. Honestly, though, I’d hardly dignify the comparison as a “benchmark”: I only attempted it on an A1200, had no concrete metrics, and the environment I used was not really what you’d call pristine or controlled. I’d call it an indication, no more.

As for optimization, I’ve been peeking through the Frotz sources, and while no obvious signs of fine-tuning leaped out (it was mainly platform-agnostic C, zero Asm includes so far), that only means the bottleneck might be due to the stack-passing overhead typical of C on the Amiga (which seems possible, given that each opcode is a function pointer). Conversely, it could also be down to expensive OS 1.x system calls, or the multitasking environment, or any of a dozen other factors. I may just have to bite the bullet and measure properly.

(Andrew Plotkin) #106

That’s all true. It’s also true, and I think more relevant here, that the I6 parser and world model are also written very close to the metal. They’re built to rely on Z-machine data structures and avoid any operation which is expensive (like managing dynamic lists).

The cost is that the I6 authoring environment is sharply constrained. Some authoring features just aren’t on the table.

When Graham went to I7, he consciously didn’t take “I7 should be as efficient as I6” as a goal. I don’t mean there’s no optimization in I7; there’s plenty. But I7 offers a much more flexible authoring model, and it runs slower than I6 in the best case. These facts are connected.

(Linus Åkesson) #107

Oh, I was referring to the built-in types (object, number, dictionary word, list, reference). It’s nice to be able to use any value (mostly) anywhere, so that, for instance, there can be a rule for ‘(the $)’ that takes a list and prints a description of the entire collection. But it’s probably the case that a lot of local variables only take on values of one particular kind, and the compiler might be able to deduce that. In those cases, it should be possible to eliminate certain runtime checks, and this in turn could lead to a speedup.

(Linus Åkesson) #108

Release notes for Dialog 0e/02, Library 0.21:

Library change: Objects at room boundaries are attracted.

Objects around the perimeter of a room, as defined by ‘(from $Room go $Dir to $Obj)’, are now automatically attracted into the room; they become floating objects. This used to be the case for doors, but now it is also true for any non-room, non-direction object mentioned in such a rule.

Library change: Scope

Objects around the perimeter of a room, including neighbouring rooms, are now only in scope if the player can see them. The current room is still always in scope (and can be referred to as dark/darkness when the player can’t see).

Library change: Disambiguation

Object-based disambiguation—when the library asks if the player meant to do a thing to any of a given list of objects—is handled differently: The answer from the player is now matched against the output from ‘(the full $)’, i.e. the actual words from the printed list. Furthermore, the answer is regarded as an elaboration of the previous, incomplete action, rather than as a new action. This makes UNDO behave more intuitively.

Library change: Internal performance tweaks

The library no longer maintains a list of objects in scope. The earlier approach had a performance advantage until 0e/01 introduced ‘(determine object $)’, but now it is faster to compute ‘($ is in scope)’ on the fly.

In contrast, the current visibility ceiling is now tracked using a global variable, and the ‘(player can see)’ predicate is a global flag. The library updates them as required.

Compiler: Minor upgrade.

Various performance improvements and bugfixes.

#109

Hej! So I only heard of this project yesterday and gave it a try last night after skimming the documentation. Looks very promising. Compiling in Windows 10 was a breeze, using the built-in linux sub-system with Ubuntu installed (essentially just sudo apt install gcc make, make;even apt-installed mingw and compiled the windows EXE… development in Windows have never been better than now).

The uuid warning was a bit annoying, but again apt came to the rescue with a quick “apt install uuid” and a one-line make-rule to create a uuid-file for each dg-file if it did not exist and automatically put that on the command-line for my %.z8 make-rule. 10 minutes and I think I have a reasonable Makefile already. No issues with the command-line interface so far.

My memories of Prolog are mixed. I took an undergraduate course in Prolog ~20 years ago. I thought it was really neat at first, but then when actually trying to implement anything it kind of turned into backwards LISP and having to constantly worry about exactly what the resolver was up to and adding cuts in the correct places. Eew. Was looking at some logic programming library for Clojure more recently (core.logic or something? based on some similar scheme library) and it was much more declarative, but probably with some other downsides. It looks from all the examples I saw so far that the higher-level Dialog code looks more like I would hope that logic programming would be, but the lower-level language stuff (that I am sure there is more of inside of the library) are more like my bad Prolog memories with carefully backwards-written code to get intended results (rather than “declare what you want to be true and let the computer resolve things” ideal)? It looks much more programmer-friendly than any of the Inform-versions I tried either way.

An emacs-mode would be nice. Anyone already have something like that, or know of some language that is similar enough that it would be a good starting point? I guess some automatic handling of indenting blocks somewhat similar to python-mode (but simpler) would be useful, but other than that not much would be needed? Maybe enable paredit or some similar minor mode.

One thing that immediately came to mind that looks like it could be built-in in some nice way is automatic testing. Ages ago when I last had an idea to write parser-based IF I wrote a perl-script wrapping frotz to play back walkthrough files with some lines being regular expressions that were matched against interpreter output in that location. (I think Inform 7 has something somewhat similar built-in?) It would be very neat if it was somehow possible to write tests that would execute lines of text and either just parse each line or (with some syntax) apply a line as a predicate to check the state of the game (or/and output). Maybe combine that with the debugger somehow. I saw that the debugger can already play-back saved scripts, but not sure if it can combine that with batch-running scripts and checking the results?

EDIT: I should add that I am not sure I want to go back to experiment with parser-based if, so maybe do not listen too much to my opinions, but I have looked for a sane way to compile choice-based games to the z-machine, and it looks like what would be needed to make that is already available in the Dialog language and that could be a fun project.

EDIT2: And now, a few minutes later, I have confirmed that it compiles and runs on my mac in OSX and on my raspberry pi in Raspbian as well. Very good work making it so portable and easy on the dependencies etc. No obvious issues making a runnable helloworld.z8 on either platform.

#110

RegTest is a generic tool for testing parser programs and will work fine for Dialog.

#111

I expected there to be something like that now, since it was 20+ years ago I wrote my perl-hack (and maybe there was something already I just did not find). But what I meant is that the Dialog design looks like it would be perfect to make something more integrated, making it possible to mix in Dialog predicates as tests for what state the interpreter ends up in, in addition to checking for substrings in the output. Have not looked at the debugger yet, but it looks like it runs on a level that would make it possible to do something like that there.

(Linus Åkesson) #112

I use scripts and makefiles for regression testing.

For game testing, I start with the transcripts from my testers and extract all the input lines (except save and restore). Then I pipe that into dumbfrotz and dgdebug, and store the output. After checking each output file manually, I copy it to a “blessed” file. Then, whenever I change the source code, I run all the tests and see how the output files differ from the blessed files (using meld). If it looks good, I type “make bless” to bless the new version of every output file.

The commandline version of dgdebug (i.e. not the windows version) checks whether its standard input and output streams are terminals, pipes, or files, and adapts. So if you tell it to read from a file and write to another file, it will do the right thing, deal with line endings and so on. Furthermore, it has a special “dumbfrotz quirks mode”, so you’ll get dumbfrotz-compatible output and random number generation. That way, you can keep a common set of blessed files for both backends. I run dumbfrotz with the options “-R mp -R lt -R ch1 -s 4242 -w 80 -h 999” and dgdebug with “-qD -w 80 -s 4242”. The “-q” tells dgdebug to terminate when the game quits, and “-D” enables the quirks mode. Note that these options assume a one-line status bar; your mileage may vary.

To save time, you don’t have to test with both backends each time. Dgdebug is faster, but it doesn’t truncate long dictionary words, and it won’t catch heap overflows that could occur when running on the Z-machine. So keep your transcripts dumbfrotz-compatible and run the full test suite every now and then.

What Inform7 integrates into its development environment is similar in spirit to the above. I personally prefer an open model based on text files, because it is easier to integrate with version control systems and other commandline tools, but that’s a matter of taste.

#113

Thanks! Those flags could be useful to keep around if I get around to actually write a story and good to know you considered this in the interface for the debugger. True that just checking all the output can be quite feasible, at least for other types of projects. I just recalled that is exactly what I do for gamebookformat for instance. Even if some bug slips through at least you get a nice history of saved outputs in version control if you need to bisect to find it later.

(Linus Åkesson) #114

Dialog 0e/03, Library 0.22 is out.

This release mostly features internal improvements to the compiler. The source code has been cleaned up, and a large part of the Z-machine backend has been rewritten. Several new optimizations have been implemented, so the generated Z-code tends to be a bit smaller and faster.

Library: Added ‘(them $)’ for printing a pronoun in the objective case (it/him/her/them) for a given object.

1 Like
(Linus Åkesson) #115

Release notes for Dialog 0f/01, Library 0.23:

Version 0f of the language introduces a number of new features:

Closures

A closure is an anonymous bit of code that can be placed in a variable and invoked later. Closures are defined inside curly braces where a value is expected. For instance:

        ($X = { Hello, world! })

        %% Nothing is printed yet, but $X is bound to the code in braces.
        %% Use '(query $)' to invoke it:

        (query $X) %% This will print "Hello, world!"

Earlier, a typical story would contain something like this:

        (space 5) (bold) \*\*\* You win! \*\*\* (roman)
        (game over) %% Launch the game-over menu.

In a story with multiple endings, that boilerplate code could end up being repeated in several places. But from now on, the library contains an additional predicate, ‘(game over $)’, defined like this:

(game over $Message)
        (par)
        (space 5)
        (bold) \*\*\* (query $Message) \*\*\* (unstyle)
        (game over)

In the story code, all that remains to do is this:

(game over { You win! } )

A closure captures the environment surrounding its definition. This means that you have access to the same local variables both inside and outside of the brace-expression:

(perform [attack $Obj])
        The attack is unsuccessful.
        (game over { You've been eaten by (a $Obj) } )

Finally, closures may take a parameter, accessed through a variable called $_. To query a closure with a parameter, use the built-in predicate ‘(query $Closure $Parameter)’. Multiple parameters can be passed in the form of a list.

Recall that the standard library provides a family of predicates for parsing object names, such as:

(understand $Words as object $Obj preferably worn)

and

(understand $Words as object $Obj preferably takable)

The ‘preferably’ part primarily controls how the word ALL is interpreted. There is now a new variant of the predicate:

(understand $Words as object $Obj preferably $Closure)

The closure takes a candidate object as parameter. This allows you to specify your own arbitrary conditions, such as:

(understand $Words as object $Obj preferably {(item $_) ~(edible $_)})

Divisions and styles

A new ‘(div $) …’ syntax is introduced, to help separate content from presentation. Divisions are rectangular areas of text, usually spanning the full width of the interpreter window, to which style hints can be applied. At the moment, the only supported style hints in the main text window are bold, italics, and top and bottom margins. More hints are supported in the status bar area; see below.

(intro)
        (div @quote) {
                This could be displayed in italics, for instance, if
                the corresponding style hint is defined for the @quote
                class.
        }

The parameter of ‘(div $)’ is the name of a style class. These names don’t end up in the game dictionary (unless they’re also used elsewhere, of course).

Styling is specified using a small subset of CSS. All CSS attributes are regarded as optional styling hints, so a backend or interpreter can pick and choose among them, or ignore them altogether. This is what the syntax looks like:

(style class @quote)
        font-style: italic;
        margin-bottom: 2 em;

The purpose of divisions and style hints is to allow the same story to run on many different platforms, with varying support for advanced presentation techniques. This is a first step towards a future where Dialog stories can be played in a web browser with full CSS support. But, critically, I want Dialog stories written for the web to also be playable (with excellent performance) on vintage computer systems, and to work well will screen readers. That’s why the style attributes are defined separately from the text and logic of the story, and treated as optional hints.

This is not a total break with the past. It is still possible to set the style explicitly at any time, using e.g. ‘(bold)’ and ‘(italic)’. To get back to the default style of the surrounding div, use the new predicate ‘(unstyle)’ instead of hardcoding a switch to ‘(roman)’ type.

The old ‘(par $)’ built-in predicate has been removed. The same effect, if absolutely necessary, can be achieved with an empty div with the desired margin.

Major overhaul of the status bar functionality

The old low-level predicates for drawing into the status bar, by explicitly moving the cursor, have been removed from the language. Instead, the layout of the status bar is controlled via style attributes.

The status bar is now a special kind of div, and it may contain other divs in turn. These internal divs can have a fixed width (expressed in em units) or occupy a percentage of the width of the parent division. They can float towards the left or right end of the parent division. The height of the entire status bar is specified in ems. The height of a floating div is always 100% of the status bar, and its contents in turn can be arranged vertically using ordinary, non-floating divs.

This is an example of a simple status bar with a score display in the upper right corner:

(style class @status)
        height: 1 em;

(style class @score)
        width: 20 em;
        float: right;

(redraw status bar)
        (current score $S)
        (current room $R)
        (status bar @status) {
                (div @score) {
                        Score: $S
                }
                (room header $R)
        }

A new built-in predicate, ‘(progress bar $ of $)’, draws a progress bar scaled to fit the width of the current div. It is rendered with character graphics on the Z-machine backend.

Miscellaneous

The built-in predicate ‘(unbound $)’, which probably wasn’t used outside of the library, has been replaced by ‘(bound $)’ with opposite semantics. This has allowed an optimization to be implemented in a cleaner way inside the compiler.

Documentation

For more details of these new features, please check out the new Input and Output chapter in the manual, as well as the new section on Closures.

This release also includes bugfixes and performance improvements to the standard library and to the optimizing compiler.

5 Likes
(Nicholas) #116

Very cool stuff here. I really like how clean the story code looks, its very readable. I’ll be playing around with this and will post some questions later.

1 Like
(Nicholas) #117

Thanks to everyone who shared examples, I’ve learned a lot from those. I’ve been playing around with porting examples from the Inform7 material. I like randomization and took a crack at porting example 132 which has a passerby npc with randomized traits and description.

Original Inform 7 example: http://inform7.com/learn/man/RB_3_9.html#e169
My Dialog port: https://pastebin.com/ayhrJkLX

One of the hardest things for me to wrap my brain around was setting random text. I have something that works but I get the feeling that I’m missing out on a feature or two of the language that would have made that easier to do. I suspect closures and lists might be what I’m looking for. Also, my code takes 25 seconds to compile so I thought I had managed to crash it, though it is the largest thing I’ve compiled so far.

1 Like
(Linus Åkesson) #118

Hi Elliot! Thanks for trying out Dialog!

From a quick glance, your code looks perfectly workable, but I see a couple of places where I would have taken a different approach. It’s a very nice example, and I’m going to see if I can reimplement it differently, just to show some of the other techniques that the language supports.

I am treating the 25-second compile time as a bug, which I’ll try to fix promptly. Compiling should be instant on modern hardware.

2 Likes
(Nicholas) #119

That would be great, I would love to see how someone else might have done it.

For me, the following bits felt the least elegant, though they all work:

  • The trait predicate system is one binary predicate per trait value as opposed to one predicate per trait with a list value. Very much a cumbersome brute force solution how I wrote it.
  • The 1 in X random structures I made work but there might have been a nicer way to write them.
  • The passerby description table in the description predicate works but might not be as readable as I would have hoped though I’m not sure how I could have done that differently. I have a wider monitor but I know that not everyone does so I made each entry two lines instead of one.
  • The walking scene should probably start immediately but that is an easy change with an intro predicate or some on entering rule.

One of my favorite parts was the dictionary predicate, as being able to dynamically change the trait words as they changed was awesome. The scene control stuff in Dialog is great and I very much appreciated the select predicates you built in as those kind of random effects were one of the first things I thought about how I might implement.

(Linus Åkesson) #120

Dialog 0f/02 fixes a bug that caused heavily nested conditional expressions to compile very slowly.

1 Like