Idea for a unified framework for handling hyperlinks in Inform 7

So I’ve been thinking about the state of the Inform hyperlinks ecosystem. We have several hyperlinks extensions, and you can do a lot with them, but it can also be tricky to know which one to use, and if you want to use multiple extensions you’re probably looking at a lot of messy hacking.

I have an idea for a new extension/framework that would both allow for multiple hyperlink purposes to be used in one game, as well as giving space for future additions to the framework. The basic idea is that hyperlink values are 32bit numbers, so we can use some of them as a tag to indicate the meaning of the hyperlink, while still leaving lots of bits for the value.

So firstly, what are the purposes people want to use hyperlinks for? I can see four main ones:

  1. line input replacement commands
  2. hyperlinks acting as keypresses
  3. run some code
  4. an arbitrary number to pass to a rulebook for the author to interpret

Can you think of any more? The beauty of a framework like this is that it will be easy to add more without stepping on anyone’s toes.

So the four types of tagged data we would need are:

  1. memory address of some text
  2. a keycode number (signed)
  3. memory address of a rule/function
  4. an arbitrary number (signed)

I propose using 6 bits for a magic number (arbitrary number used to identify that the hyperlink was produced by the framework; if it isn’t present the framework skips handling the hyperlink), 2 bits for the tag number, and 24 bits for the value. The extension can handle extending the signed numbers from 24 to 32 bits. 24 bits will be enough to encode memory addresses for up to 16MB, which is far more than any I7 game I know of. (Note that is code only - there are bigger games, but they are bigger because of the multimedia they include in the blorb.) If we want more than four types of tagged data we can change to a 5 bit magic number and a 3 bit tag, or just use a second different magic number.

That’s the technical proposal, sorry if it’s confusing. As an author what would this mean? You could seamlessly use hyperlinks to act as keypresses in a menu, use hyperlinks to enter commands, use hyperlinks to expand a side panel window (like in Kerkerkruip for example), all in one game, without having to write code to say “stop interpreting hyperlink numbers as keypresses, now they refer to command replacements.”

Any thoughts? Is there anything you have ever wished you could do with hyperlinks in Inform that seemed to messy or difficult to implement?

3 Likes

I don’t 100% understand the tech but appreciate that basically you’re talking about implementing this at a more fundamental level than previous extensions have.

I don’t think I have suggestions. I’ll just remind you of the outlook of the extension I didn’t finish in case the ideas are useful at all. I called it a CYOA extension, though it wasn’t intended to make any particular type of game. It was to make in-game menus that would look after themselves. One side-effect of this was it certainly made making a straight CYOA in Inform more easy than with any other system I saw.

So the base unit was the choice. The game could dynamically assemble menus of choices without hardcoding their numbering, then keypress-number or letter – and/or hyperlink them - depending on what the game wanted to dispense, or on player preference if the author allowed. A choice made could convert to line input, or just run any bit of code you wanted. These functions were extensions atop Unified Glulx Input.

-Wade

1 Like

It would be tidier (though less efficient) to store the text and rule addresses in tables, and have the tagged data contain a row number. Then you wouldn’t have any lurking memory-size problem. You’d get type-safety too.

1 Like

For text input I did imagine that probably a system would be built on top of this low-level framework, perhaps using tables. (TBH I haven’t yet looked into how the current extensions implement text replacement.)

For rules, I do like the simplicity that simply using a direct rule address would give. It would let you do something like this for a CYOA:

This is the page one rule:
    say "Page one description";
    say "[link page two rule]Go to page two[end link]";
    say "[link page three rule]Go to page three[end link]";

This is the page two rule:
    say "Page two description";
    say "[link page three rule]Go to page three[end link]";
    say "[link page four rule]Go to page four[end link]";

There could be some runtime type checking, at least for rule functions. I suspect a lot of the text replacement extensions just output a word array, with no Glulx type byte.

I did also think about making the 24bit a setting, so you could push it to 25bits if needed, but to be honest it’d probably be better just to fork the extension. Or alternatively, rather than having type bits, the “framework” could instead just be a list of magic numbers, so that a 6bit number would be the tag for memory addresses. Extensions that want to join the framework could make their magic numbers a variable or use option, and if we organise things poorly so that two extensions accidentally use the same magic number, authors could set their own values for each. How close are we to having stories hitting 16MB of Glulx code?

FYI, the hyperlink extensions that I wrote can already do all of those things.

Ah thanks. I hadn’t looked at them in much detail. I added them to my Glulx/Glk extensions ecosystems wiki page.

I wanted to post this before I wrote any code because I expected you all would have some better ideas. Zarf is right - if we can utilise the compiler’s type checker we should.

Adjusted idea:

We still have a small marker to indicate that the hyperlink code came from the framework, perhaps the top two bits are set. But rather than having a few low-level tags like I initially proposed, they can be higher level extension codes. We can maintain a registry of them, but it should also be possible to let an author overwrite them as a last resort. Perhaps we can also reserve codes in case people need more than 24bits of address space in the future. (Maybe a four bit marker and four bit codes, and if people need more then we could drop to a two bit marker, shift the codes by two bits, and then have 26bits for addresses, up to 64MB.)

With higher level tags, you could do something like the following for a CYOA extension. Because the hyperlink handling rule is higher level it can handle adding a paragraph break and re-requesting hyperlink events. Type checking can be done in both the initial say phrase, and in the hyperlink handling rule.

Morning is a choice node. "You wake up in the morning.[paragraph break]

[choice get coffee]Drink a cup of coffee.[end choice][line break]
[choice take a shower]Take a shower.[end choice][line break]"

Get coffee is a choice node. "You look for coffee, but you're all out.[paragraph break]

[choice have some tea]Have some tea instead.[end choice][line break]
[choice make toast]Just make toast. You'll pick up a coffee on your way.[end choice][line break]"
1 Like

The Inline Hyperlinks extension relies on maintaining a list of text for all unique hyperlinks printed, populated at runtime, using the index within this list to figure out what a particular link id means. This requires dynamic memory but hyperlinks are Glulx anyway, and it’s more author-friendly than listing every possible link separately in a table. (I didn’t come up with this idea, it’s inherited from Daniel Stelzer’s extension.)

It does have a mechanism to “reserve” some ids but otherwise currently assumes that it can use the whole space (and that you stop playing before you exhaust the space, or the story explicitly clears the list at certain points, e.g. starting a new chapter or some other condition that clears the scrollback). But it wouldn’t be hard to adapt it to some more constrained number space if we want to.

I was pondering whether Inform’s units/multi-part numbers might be interesting for type-safety purposes when considering a composite id, but I think these are based around real numbers and that’s probably not a good mix when we need a regular int in the end. (And I7 would have to be taught how to use bitwise operations anyway.)


Regarding a specific encoding, I think most likely just setting aside some bits for payload and some for an extension id would be sufficient; the extension is free to sub-divide its payload to indicate additional metadata if it wants to (or register itself multiple times), but each extension might have different needs.

As for how to get an extension id, the simplest method would be to have a table with an appropriate number of blank rows (e.g. 255 rows if the id is 8 bits – but we probably don’t need that many), and on startup the extension tries to add its internal processing rulebook to a blank row of the table – if it succeeds, the row number it got is its extension id for purposes of tagging links. (And since there is no “row 0”, extension id 0 is reserved for manually set links.) If it fails, there are too many extensions included and it can print an error message to the author. If it’s done cleverly, it would theoretically be possible to allow the author to reserve space for more extensions if they need it by sacrificing more bits off the payload.

Or that can use a list rather than a table too; that might use a little less memory in the end, and it’s only going to get changed during startup anyway.

I’d be happy to code that up as well if you like.

Sadly it looks like a list-based solution might be a no-go. I had a play around with some code and the id encoding/decoding was all working well, but the core of the registry relied on this sort of concept, and Inform appears to not like it for some reason:

The hyperlink extension registry is a list of number based rulebooks producing text that varies.

Any attempt to treat that actually as a list is met with a Problem. Suspiciously, according to the Values index this declares a variable of kind lists of numbers based rulebook producing texts, which may be related to the problem.

I suspect it’s trying to parse this as a list of numbers based rulebook rather than a list of numbers based rulebook, but I don’t know how to convince Inform to read it properly, since AFAIK it doesn’t have a syntax for declaring a kind in two steps.

A tables-based solution is probably still possible, but more clunky.

Sad, too. The list-based approach would have been really elegant.

I had been thinking that the tags would be global variables; extensions would include lines like these:

Keycodes hyperlink tag is initially 1.
Command replacement hyperlink tag is initially 2.
Choice nodes hyperlink tag is initially 10.

If the community doesn’t coordinate well, or two people publish extensions at the same time that use the same code, authors would be able to change those values manually.

So, your idea of registering an extension… I actually like that more. However I still like the idea of global variables, because an extension like Choice Nodes would need to know what its tag is in order to create its hyperlinks. It could look up the table for its associated rule, but that’s messy. Instead, what if each extension requested its tag like this:

After starting the virtual machine (this is the register choice nodes rule):
	now choice nodes hyperlink tag is the next available hyperlink tag;

A To decide what number is the next available hyperlink tag: phrase would keep a counter, returning and incrementing the counter. If someone has too many registered tags then it would display a runtime error.

So now that Choice Nodes has registered itself, it can use its tag in the hyperlink handling rules and phrases to create hyperlinks:

A glulx input handling rules for a hyperlink-event (this is the handle choice nodes events rule):
	if the current hyperlink matches choice nodes hyperlink tag:
		let new node be the hyperlink payload as an object;
		...
		
To say choice (C - a choice node):
	set hyperlink with tag choice nodes hyperlink tag and payload C;

The framework extension would provide a few phrases like current hyperlink matches (T - a number), hyperlink payload as an object, hyperlink payload as a signed number, set hyperlink with tag (T - a number) and payload (P - a value). These phrases would handle all the bitshifts, bitmasks, etc, and could even make it so that if anyone did need more than 24bits of payload we could make 25 or 26bits be a compile-time use option without needing any other extensions to be modified. The number of bits for tags could also be a use option.

(BTW, tables use much less memory than lists. Lists use the Flex system so there’s a lot of overhead.)

Yeah, the next available hyperlink tag thing is probably a good way to go, since the list-based approach seems to be out.

I’m a little squeamish about putting objects directly in a hyperlink id – that’d only be safe if you could guarantee that object addresses aren’t using the bits stomped by the extension tag/id.

Yes, I know. “Elegant” was referring to the author’s code, not necessarily the memory footprint :grin:

The theory was that registration would be as simple as:

Choice link processing rules are a number based rulebook producing text.
	
Choice link processing rule for number (called N):
	say "You clicked hyperlink [N].[paragraph break]".

After starting the virtual machine:
	add choice link processing rules to the hyperlink extension registry.

(This would be in the choice extension of course, with the actual author’s code even more elegant.)

And there was a use option to define how big the extension ids are as a number of bits. I assume it’s probably possible to size tables via use options, but I’m less sure you can do that based on bitshift math.

Or rather, I think you can size I6 tables/arrays that way. I don’t think you can for I7 tables, at least not without the deep magic.

Ooh, that is elegant. Your idea is that the hyperlink handling rule would extract the tag, access the registry list/table, and then invoke the rule? I like the simplicity of that, but don’t see how the choice extension could make a hyperlink without knowing its own tag.

Immediately after you added the rulebook to the registry, then the number of entries in the hyperlink extension registry would be equal to your tag/id, which you could save in a global and use to generate links later.

That part wasn’t quite as elegant yet though.

Probably you’d want to abuse a to decide phrase a bit and have the registration code actually be something closer to:

now choice link tag is a new hyperlink tag for choice link processing rules;

If I could only figure out how to make a list of rules :cry:

Or the register phrase could simply return a number.

While I do like the idea of having the framework invoke your rule so you don’t have to have check if the hyperlink number matches your tag, considering there will probably only be half a dozen hyperlink extensions max, is it worthwhile, to save one line of code (if the current hyperlink matches choice nodes hyperlink tag)?

Also, rather than just returning an object, it should be possible to make a hyperlink payload as a (K - a kind of object) phrase which checks that the encoded address is for an object of the appropriate kind.

To decide is how you return a number.

Yeah, that’s an advantage of the method you’re proposing, you can clobber the kinds a bit more freely. Of course, some people might consider that a disadvantage. :wink:

One of the reasons I was trying to keep the number based rulebook producing text thing is that this is how my Hyperlinks extension works, which already incorporates all the logic of re-injecting command text or keypresses, so the actual processing rules are very simple.

Just rule succeeds with result "go north" or rule succeeds with result "f" etc. Or you can execute arbitrary logic without providing a command at the end, and it all just works.

Sadly I don’t think there’s a way to declare a rulebook that is based on a pair of numbers rather than a single number. (There are the structured numbers but I wouldn’t trust passing an object address through them.)

This could still be made to work but each processing rule would have to decode and check the extension id/tag rather than doing it centrally as I was hoping.

Although… that gives me an idea…

So, good news and bad news.

The good news is that this works in Git. (I’m open to better names for some of the phrases.)

The bad news is that it doesn’t work in the IDE’s interpreter (Windows IDE as of 2018, at least), which appears to only support 16-bit hyperlinks. (I can’t tell whether they’re getting shaved at printing or at clicking, but only the low 16 bits of the id makes it through.)

The extension could be adapted fairly easily for a 16-bit link range but you definitely can’t pass object ids (or anything but numbers) in that sort of range (and not many numbers; shaving bits off for a tag at all is kinda problematic).

Huh. That’s weird. I had a look through your code, as well as Windows-Glk and the Inform 7 IDE code, and couldn’t see anything that looked like it would be cutting hyperlink numbers down to 16 bits.

@DavidK any ideas?

(And don’t take this the wrong way, but I also want to write an extension my own way, though I won’t get to it as quickly as you :wink:. Then we can compare and see how to make the ultimate one.)

I suspect this line is the problem:

sStyle is a 16-bit field.

1 Like

Indeed, you are correct: I’ve just spotted that too. Whoops! I will get that fixed, but I can’t say when. It is at least now recorded as an issue: https://github.com/DavidKinder/Windows-Inform7/issues/5

4 Likes

It seems to work if instead of trying to get Inform to parse your complicated syntax, to just set the list to an initial value which fits the bill:

Lab is room.

Do nothing link processing rules are a number based rulebook producing text.

Do nothing link processing rule:
	do nothing.
	
The hyperlink extension registry is initially { the do nothing link processing rules }.

Choice link processing rules are a number based rulebook producing text.

Choice link processing rule for number (called N):
	say "You clicked hyperlink [N].[paragraph break]".

When play begins:
	add the Choice link processing rules to the hyperlink extension registry.
	
Instead of waiting:
	repeat with R running through the hyperlink extension registry:
		follow R for 5.

test me with " z / rules / z".
2 Likes

That’s a neat trick. I’ve already solved this particular issue a different way (that means I don’t need to actually keep a list around), but I’ll remember that for next time, thanks!

1 Like