Glk/Quixe/Emglken without side-effects (pure JS API)?

Technical Development Tools
1

@Dannii I couldn’t respond inline because new users are limited to three replies per topic.

Sorry I haven’t been clear. At the risk of providing too much information, here’s a fairly complete description of what I’m trying to do.

I’m writing a Node.js application that has chat capabilities. My server is long running and may handle many chat sessions concurrently. In each session, the user is chatting with a chatbot. Users do not chat with each other.

Each user is a customer who has bought something from an online marketplace. The purpose of my chatbot is to supply answers to users’ queries about the status of their order(s), and perhaps provide options to escalate their case.

One way that I could implement the chatbot is to parse incoming chat messages and scan for known keywords such as “HELP” or “ORDER STATUS”. This would be fairly typical, abstract, limited and bland.

Instead, I was hoping to use IF to enrich the experience. Rather than talking abstractly to a chatbot, the customer enters an IF storyline starting in a room called “Support Desk” (or similar). In the room are placards or posters, or maybe a support agent NPC, etc. I haven’t worked out all the details.

Since the IF story is fully contained, I was going to do string replacement on command output to insert things like real order status details. String replacement was going to be my bridge between the static story world and the dynamic real world.

To use IF in this way, for each customer I need to:

  1. instantiate the interpreter and load the story,
  2. supply text to the interpreter on behalf of the user (incoming chat messages),
  3. retrieve the text the story produced,
  4. perform string replacement,
  5. post the final string back to the user (chat response).

(In addition I’ll probably want to save and load story state to recover from hiccups, but one thing at a time.)

Since Quixe is already written in native JavaScript it is nearly ideal for my task, except that it comes with lots of browser-centric assumptions (makes use of the window, DOM, localStorage, etc.).

What I’m looking for is a contained, native JS interpreter that I can instantiate many times in the same parent process, without side-effects (no use of argv, stdin/stdout, window, filesystem, etc.) and which offers an asynchronous, text in/text out API.

I understand that others have approached similar problems by spinning up a child process and then sending data through pipes. I could certainly look into firing up frotz or Glulx on a per-user or even per-chat-message basis. But given that Glk seems to be designed to provide an IF API, and that Quixe already exists, this seems closer to what I wanted.

I hope this makes sense!

2

See my comment in the previous thread: Using Quixe without the display layer?

If you remove the glkote.js module, then the interpreter is no longer connected to the DOM.

(Putting several interpreters in the same Javascript environment may require some code tweaks. Everything is supposed to be modular and avoid use of window-global variables, but again, not well tested.)

3

I have a modified version of Quixe lying here (ignore the rest of the repository, it was an experiment I never had time to finish):

It is essentially Quixe without DOM manipulation, that receives and sends RemGlk/GlkOte events. See the README in the Quixe folder for usage. (You can also make a diff to see what I’ve changed). There may have been new releases of Quixe since I made this, so it may not be up to date.

Also, I haven’t extensively tested it, but it should work.

It’s possible to run multiple interpreters at the same time, but since the initial Quixe structure with closures (no classes) is kept, each time you create a story, new copies of all Quixe methods are created.

Hope it helps!

I don’t think it’s that easy, because the interpreter and glkote.js are a bit coupled. If you remove glkote.js, you have to modify Quixe to remove the glkote references from the file. (At least that’s what I had to do. Maybe I missed something.)

I still think it would be a good idea to rewrite Quixe to split the interpreter part from the DOM part and to use more modern JS (module imports/exports and classes instead of closures; webpack and such; and why not TypeScript while we are at it), but well, I understand there are other priorities.

1 Like
4

Did you leave glkapi.js in place? That should narrow the API surface a lot.

At that point I’d write a replacement GlkOte interface object – you basically have to implement init() and update(), plus log/warning/error calls. Everything goes in and out as JSONable data objects.

There are no DOM manipulation calls in quixe.js or even glkapi.js.

I recently updated the modules to use a (clunky) hybrid model. The original closures still exist, but (in a modern JS environment) they are pushed into exports so that the files can be loaded with require(). Lectrote now uses require().

I intend to do a more thorough updating at some point, but as you say, priorities.

5

Writing a minimalistic GlkOte implementation can be done in only about 200-300 lines. It’s not that complex a protocol. Especially if you’re just sending plain text to and from the chat bot, and can ignore formatting, character input, etc.

6

Pretty sure FyreVM-Web would work perfectly with this.

http://plover.net/~dave/cloakjs/# (inspect the FyreVM object)

2 Likes
7

I did a quick experiment in this direction.

In the standard Quixe setup, I replaced glkote.js with this stub file:

var interface = null;

var metrics = {
    "height": 800,
     "width": 800,
     "buffercharheight": 20,
     "buffercharwidth": 8,
     "buffermarginx": 0,
     "buffermarginy": 0,
     "graphicsmarginx": 0,
     "graphicsmarginy": 0,
     "gridcharheight": 20,
     "gridcharwidth": 8,
     "gridmarginx": 0,
     "gridmarginy": 0,
     "inspacingx": 0,
     "inspacingy": 0,
     "outspacingx": 0,
     "outspacingy": 0
};

var GlkOte = {
    version:  '0',

    init: function(iface) {
        interface = iface;
        interface.accept({ "type": "init", "gen": 0, "metrics": metrics });
    },
    
    update: function(dat) { console.log('update:', dat); },

    log: function(val) { console.log('log:', val); },
    warning: function(val) { console.warning('warning:', val); },
    error: function(val) { console.error('error:', val); }
};

The metrics object gives the size of the imaginary screen, which the VM needs to figure out the status line width. (Unfortunately you have to spell out all those fields, most of which are irrelevant. There’s no code in this layer to figure out sensible defaults.)

The important functions are init() and update(). The init() function does whatever setup you want, and then calls interface.accept().

The game then runs up to the first input point and calls update(). You see that in this stub, all update() does is log the output. Testing on good old Advent.ulx, we get this (JSON format):

{
  "type": "update",
  "gen": 1,
  "windows": [
    {
      "id": 454,
      "rock": 202,
      "type": "grid",
      "gridwidth": 100,
      "gridheight": 1,
      "left": 0,
      "top": 0,
      "width": 800,
      "height": 20
    },
    {
      "id": 452,
      "rock": 201,
      "type": "buffer",
      "left": 0,
      "top": 20,
      "width": 800,
      "height": 780
    }
  ],
  "content": [
    {
      "id": 454,
      "lines": [
        {
          "line": 0,
          "content": [
            "normal",
            " At End Of Road                                                          Score: 36    Moves: 1      "
          ]
        }
      ]
    },
    {
      "id": 452,
      "text": [
        {},
        {},
        {},
        {},
        {},
        {
          "content": [
            "normal",
            "Welcome to Adventure!"
          ]
        },
        {},
        {
          "content": [
            "header",
            "ADVENTURE"
          ]
        },
        {
          "content": [
            "normal",
            "The Interactive Original"
          ]
        },
        {
          "content": [
            "normal",
            "By Will Crowther (1973) and Don Woods (1977)"
          ]
        },
        {
          "content": [
            "normal",
            "Reconstructed in three steps by:"
          ]
        },
        {
          "content": [
            "normal",
            "Donald Ekman, David M. Baggett (1993) and Graham Nelson (1994)"
          ]
        },
        {
          "content": [
            "normal",
            "[In memoriam Stephen Bishop (1820?-1857): GN]"
          ]
        },
        {
          "content": [
            "normal",
            "Release 5 / Serial number 961209 / Inform v6.21(G0.33) Library 6/10 "
          ]
        },
        {},
        {
          "content": [
            "subheader",
            "At End Of Road"
          ]
        },
        {
          "content": [
            "normal",
            "You are standing at the end of a road before a small brick building. Around you is a forest. A small stream flows out of the building and down a gully."
          ]
        },
        {},
        {
          "content": [
            "normal",
            ">"
          ]
        }
      ]
    }
  ],
  "input": [
    {
      "id": 452,
      "type": "line",
      "gen": 1,
      "maxlen": 256,
      "initial": ""
    }
  ]
}

It’s verbose all over the place, but that’s the info you need to display the initial game screen for the player. Two windows; a 100x1 status window and a story window. Content for each of them, formatted as lines (for the status window) or paragraphs (for the story window). Also, window 452 (the story window) is requesting line input.

Next thing that happens is the player types GO EAST. You send this to interface.accept():

interface.accept({ "type":"line", "gen":1, "window":452, "value":"go east" })

The result is another update:

{
  "type": "update",
  "gen": 2,
  "windows": null,
  "content": [
    {
      "id": 454,
      "lines": [
        {
          "line": 0,
          "content": [
            "normal",
            " Inside Building Score: 36 Moves: 2 "
          ]
        }
      ]
    },
    {
      "id": 452,
      "text": [
        {
          "content": [
            "input",
            "go east"
          ],
          "append": true
        },
        {},
        {
          "content": [
            "subheader",
            "Inside Building"
          ]
        },
        {
          "content": [
            "normal",
            "You are inside a building, a well house for a large spring."
          ]
        },
        {},
        {
          "content": [
            "normal",
            "There are some keys on the ground here."
          ]
        },
        {},
        {
          "content": [
            "normal",
            "There is tasty food here."
          ]
        },
        {},
        {
          "content": [
            "normal",
            "There is a shiny brass lamp nearby."
          ]
        },
        {},
        {
          "content": [
            "normal",
            "There is an empty bottle here."
          ]
        },
        {},
        {
          "content": [
            "normal",
            ">"
          ]
        }
      ]
    }
  ],
  "input": [
    {
      "id": 452,
      "type": "line",
      "gen": 2,
      "maxlen": 256,
      "initial": ""
    }
  ]
}

I won’t say that the JSON format doesn’t have its quirks. But it’s usable. Documented here: GlkOte: a Javascript library for IF interfaces

4 Likes
8

Thanks everyone for your help! Conserving my comment quota by answering all at once.

:pray: I was able to use Nuixe as a git submodule in my TypeScript/Node.js project. Thank you for sharing this interface to the code.

One thing that I haven’t figured out yet: the QuixeStory constructor works when I give it the project.inform/Build/output.ulx contents, but not when I give it the project.materials/Release/project.gblorb contents. Here’s the relevant part of the stack trace for the latter:

(node:3280) TypeError: display_error is not a function
    at Object.fatal_error (.../nuixe/app/interpreters/glkote/glkapi.js:1142:5)
    at start_game (.../nuixe/app/interpreters/quixe/gi_load.js:661:21)
    at Object.load_run (.../nuixe/app/interpreters/quixe/gi_load.js:172:5)
    at new QuixeStory (.../nuixe/app/interpreters/quixe/index.js:21:23)

Here’s the code in glkapi.js where the problem occurs (inside the fatal_error function):

// NUIXE: Replaced `Glkote.error` by `display_error`.
display_error(msg);

By replacing display_error(msg) with console.error(msg) I was able to extract this underlying fatal error message:

Blorb file could not be parsed: ReferenceError: $ is not defined

I don’t know what this means. When trying either the .ulx or .gblorb file, I’m using the same code with the exception of the path. The code reads the file in as a Node.js Buffer object, then calls toString(‘base64’) to create the QuixeStory input parameter.

Aside: What trade-offs are there to consider when choosing between the .ulx or .gblorb file? Is there a reason not to just stick with the .ulx?

Thank you for this walkthrough, it was helpful. I used your metrics settings verbatim.

A few quick question on the returned content objects. Each array of content appears to contain pairs of strings (I understand entries can also be objects). Even-numbered strings are styles. I’ve observed values of normal, header, subheader or input. What is the list of possible style keywords? I did not see a list in the Line Data Array documentation.

https://eblong.com/zarf/glk/glkote/docs.html#linedata

Thank you for posting this. I took a look at the underlying interpreter library glulx-typescript. I haven’t tried it yet because the README warns that “Inform6-compiled Glulx games seem to work okay, Inform7 not so much.” I may circle back and give this a shot.

9

What is the list of possible style keywords?

I’m not Zarf and I haven’t really looked into this, but an educated guess would be that those are the eleven standard Glk styles: Normal, Emphasized, Preformatted, Header, Subheader, Alert, Note, BlockQuote, Input, User1, and User2.

EDIT: Weirdly, clicking the link above jumps to the wrong place in the Glk spec document for me, but just selecting the browser URL window and pressing Enter makes it jump to the right place (5.5 Styles).

2 Likes
10

I recommend looking at the example I posted, which does the same thing without modifying the source and with full support for error logging (in a very simple way).

The gblorb file is larger because it contains the image data, including the cover image. This means that the image data sits there in Javascript memory, which is inefficient, particularly if you’re never going to display any images.

It also means that displaying images has to pull the image data from the file and convert it to a data: link in the DOM. This is, in theory, the slow and nasty way to get images onto the screen. (I don’t know how much slower it actually is.)

On the other hand, if you use .ulx, then you have to export all those images as files, and then generate a StaticImageInfo object so that the browser can find the files. This is a lot of dancing around to save some Javascript memory, which is really probably not critical.

11

Right. These aren’t documented in the GlkOte docs because GlkOte doesn’t really care what they are, but that list of 11 is what Quixe generates.

12

But my modified version can launch multiple Quixe instances and play multiple stories at the same time, which I don’t think your example can do. Besides that, I agree your example is better.

However, even without GlkOte, there’s still DOM manipulation in gi_load, in the absolutize and unpack_blorb functions. So you still have to modifiy gi_load to make it work in Node.js.

It’s a reference to jQuery. I tried to remove them all, but I guess one slipped by. You’d have to remove it. (Or not use a blorb. As zarf pointed, you don’t necesseraly need one and can use a plain ulx.)

13

Ah, thank you. That was the reference I was missing.

I’ll make a note about cleaning that up, although it might not be soon.

14

do you really need all the baggage of an IF engine to do this? there are other languages suitable for basic input/output for chatbots, for example:

rivescript -
plain JS engine, runs as a node app, it also has support for multiple sessions, advanced parsing, local variables per user, state management etc.



(disclaimer i’m a very minor contributor)

superscript
loosely based on rivescript but with some NLP extensions. not as stable

a bit earlier along:

these are all basic nodeJS projects that do text IO for bots.

If you want a system with the concept of multiple NPCs, room items, inventory management etc, then yeah you’ll need something more. Like many people here, I’m working on something like that, in JS :smiley:

It’s a shame the interpreters/game engines are so closely bound to things like the DOM, I would like to have adopted something existing but couldn’t find a simple engine with a script format that was usable.