The Problem of Inference in Inform

[otistdog]

Branching from the discussion at Pouring it into, Does the player mean - #21 by otistdog…

OK, calling all Mad Scientists: Let’s fix the inference problem.

Based on my experiments, this is both easier and harder than it sounds. Certain relevant parsing features are not exposed at the I7 level and not easy to represent, but that can be worked around. The bigger problem is trying to impose an intuitive model of the inference process over what’s actually going on, so as to make something easy for authors to use.

@Draconis: You have pretty much written off inference as a nuisance best removed. What would it take to make the subsystem useful to you?

@Zed: You’ve had a long-standing interest in “sanity check rules.” What form would those take, and can they be made applicable here?

Some approaches I’ve tried:

• Fine-grained inference scoring control

  • author can write arbitrary rules to assign inference score bonuses and penalties
  • pros: complete control over the score used to infer objects
  • cons: hard to anticipate how combined scores will stack up for a given object

• Fine-grained control of existing model

  • author can write rules to decide whether or not scoring components are applied, but not dictate scores
  • pros: fairly easy to eliminate “interference” of automatic rules in particular situations
  • cons: have to understand underlying model in detail to know what rules to write

• Double-duty for DTPM rules

  • the inference process consults DTPM rules when deciding inference scoring
  • pros: seems closer to the intuitive model most people have of what DTPM rules should do
  • cons: makes understanding the full impact of DTPM rules even harder

[Draconis (Daniel Stelzer)]

@otistdog

@Draconis: You have pretty much written off inference as a nuisance best removed. What would it take to make the subsystem useful to you?

Well…

Overall, I think the current inference system is a solution looking for a problem, and creating new problems in the process. The only times I’ve seen someone type TAKE without a noun, it’s because they were exploiting inference to figure out if there’s a single object in a dark room, or something along those lines. (This was a frequent “feature” of the Phoenix games.) If someone types UNSCREW PANEL without a second noun, it’s either because they didn’t expect it to require a second noun at all, or the choice of second noun is so obvious they thought it could be implied.

In either case, “(with the apple)” is not an appropriate response. If there’s an obvious choice, that has to be decided based on the specific action and the specific circumstances, not on the number of things in scope. If there isn’t an obvious choice (even if there’s only one thing in scope), “you must supply a noun” or “with what?” is far more appropriate.

The normal way to handle this in I7 doesn’t require an inference system at all. If an action has an obvious second noun, like UNSCREW needing a screwdriver or FILL needing a liquid, then the author is the best one to determine this; they can make a separate one-noun action (like Locksmith does for UNLOCK), which either fills in an appropriate second noun (if possible), or prints a suitable error message (if not). You can even do this without another action using “supplying a missing [second] noun”, if you want.

For any other case, an error message is better than grabbing the nearest item. Imo, this would have all the benefits of the current inference system, and also avoid GET grabbing an undescribed privately-named scenery thing that you’ve tried very hard to keep out of scope, or UNLOCK DOOR supplying “(with the door)”.

The main thing I7 can’t currently do, which I think would be helpful here, is let an author print “with what?” and have the next command fill in the missing [second] noun. Currently, that can only be done by the parser internals in a way that’s generally inaccessible to authors. If Inform could do that, then authors could handle these things entirely at the I7 level, and inference would be needed no more. Dialog handles this with a global variable holding the incomplete action (with nil in place of the missing noun or second noun). When the parser sees a noun-only command, it checks that variable and returns the completed action if possible. Either way, the variable is cleared after the next input.

(Does Inform have a name for this “with what?” mechanism? Dialog calls them “implicit actions”, but that means something entirely different in I7. ZIL calls them “orphan commands”, but that technically applies to any ambiguous command, like PUSH BUTTON with two buttons present. For lack of a better term, I might call them incomplete actions.)

[Zed (Zed Lopez)]

@Draconis

For any other case, an error message is better than grabbing the nearest item.

Agreed.

@Draconis

Does Inform have a name for this “with what?” mechanism? […] For lack of a better term, I might call them incomplete actions.

The place in NounDomain where they’re asked is after the .Incomplete label. But given that we’re talking about something during parsing, I think I’d be happier with incomplete command to avoid implying an action exists yet. (To my knowledge, there’s no hint of an existing name.)

@otistdog

Double-duty for DTPM rules

It’s probably wholly infeasible to try to crowbar this into the current code, but conceptually it seems like it’d be something close to ideal if this were when supplying a missing noun and supplying a missing second noun occurred instead of requiring us to make separate grammar lines.

I need to be off to bed, but I’ll think more about it later…

[otistdog]

OK – the opening bid is “Please just make it stop.”

Here’s a typical example of out-of-the-box interaction:

Lab
A typical Mad Scientist's laboratory.

You can see a beaker washer and a scarred lab table (on which are a beaker of red liquid, a beaker of green liquid and a beaker of blue liquid) here.

>I
You are carrying:
  a clipboard

>PUT
(on the clipboard)
You can't wear that!

>PUT CLIPBOARD
(in the clipboard)
You can't put something inside itself.

It’s not that hard to prevent inference (i.e. trying to determine a noun without corresponding command words), yielding:

Lab
A typical Mad Scientist's laboratory.

You can see a beaker washer and a scarred lab table (on which are a beaker of red liquid, a beaker of green liquid and a beaker of blue liquid) here.

>I
You are carrying:
  a clipboard

>PUT
What do you want to put on?

>PUT CLIPBOARD
What do you want to put the clipboard in?

>TABLE
That can't contain things.

>PUT CLIPBOARD ON
You can't wear that!

>PUT CLIPBOARD ONTO
What do you want to put the clipboard onto?

I’m not sure it’s a huge improvement, though it does have the benefit of not actually going ahead with the wrong actions (mostly).

Arguably part of the problem here is the set of default grammar lines around >PUT. There’s no ability to directly control precedence of those in I7.

The case of >PUT CLIPBOARD ON is interesting. Assuming that it’s not a wearable object, a person would probably judge this far more likely to be an incomplete command lacking a second noun than a complete but nonsensical command.

[otistdog]

@Draconis

The main thing I7 can’t currently do, which I think would be helpful here, is let an author print “with what?” and have the next command fill in the missing [second] noun.

Actually, it can. It just prefers to infer.

@Zed

It’s probably wholly infeasible to try to crowbar this into the current code, but conceptually it seems like it’d be something close to ideal

No, it’s definitely possible. The drawback is that it requires a mini-library of DTPM rules that cover the built-in grammar lines.

[pontusi]

I have no solutions to offer, but as a player, this certainly isn’t desirable:

"Untitled"

Laboratory is a room.

>give
(yourself yourself)
(first taking yourself)
You are always self-possessed.

Now, this is an edge case, but it still highlights that there’s something wrong.

[otistdog]

Reviewing the guts, the inference process does consult DTPM rules out-of-the-box, as part of ScoreMatchL(), it’s just that the details of constructing correct ones for two-parameter actions often are not obvious. So maybe part of the problem is the paucity of DTPM rules in the Standard Rules. However, there’s no out-of-the-box to support a “never infer this” decision (something offered by Disambiguation Control, I think).

Zed’s right that the case of a “short grammar” for an action bypasses inference and drops the problem of determining nouns into the for supplying a missing... activities, and that some default rules for those rulebooks that consult DTPM could help. The main drawback of working with those activities is that they are post-parsing and therefore can’t take advantage of the asking which do you mean activity or the prompt to provide more input.

[zarf (Andrew Plotkin)]

@otistdog

The main drawback of working with those activities is that they are post-parsing and therefore can’t take advantage of the asking which do you mean activity or the prompt to provide more input.

Just what draconis meant by

let an author print “with what?” and have the next command fill in the missing [second] noun.

There’s a clearly a use case for being able to say “try getting (something the player will tell us what it is)” at action-processing time.

[Draconis (Daniel Stelzer)]

Having any version of that would be a great improvement imo, but for bonus points: Dialog also lets the author specify what sort of missing noun the parser should look for. That way, if you PUSH something that’s pushable between rooms, Dialog will specifically look for a direction in the next input, and turn it into PUSH TROLLEY EAST or whatever. If you UNSCREW PANEL, it’s looking for a thing, so typing EAST will produce “going east” rather than “unscrewing the panel with east”.

(Dialog specifically only allows “thing” and “direction”, which is enough for 90% of cases, but something like TURN DIAL asking “to which number?” requires modifying the library.)

[otistdog]

@zarf

Just what draconis meant

Oh, OK – Thanks, zarf. I misunderstood since I was equating “next command” with “next input.”

As Zed pointed out, the closest equivalent is what’s seen at .Incomplete in NounDomain(). That part of the code asks for more information mid-parsing of the current command, but it is generally never executed because it is treated as a last resort for when inference fails.

Unless you specifically want there to be separate action-processing cycles (something that impacts turn sequence processing), then the interaction pattern of the player being prompted for more to complete a command is quite similar to Dialog’s player-facing behavior. That’s what’s going on in the example transcript above – the modification just prevents inference entirely, so it falls back to the .Incomplete block.

@Zed, I think I was reading your post too quickly, too. Are you saying that inference should consult SMN and SMSN, so that a Standard Rule like:

Rule for supplying a missing noun while an actor smelling (this is the ambient odour rule):
	now the noun is the touchability ceiling of the player.

would be invoked to set the noun while inferring for the grammar line

Understand "smell [something]" as smelling.

instead of relying on the existence of the “short grammar” from

Understand "smell" as smelling.

and making the noun assignment post-parsing?

If you disable the short grammar for the smelling action, you can get the same basic result from:

After deciding the scope of the player while smelling:
	place the location in scope.

Does the player mean smelling the location:
	it is likely.

Rule for clarifying the parser's choice of a room while smelling:
	do nothing.

I think it should be possible to hotwire those activities into the main parsing sequence, though.

[Celtic_Minstrel]

Not sure how closely this is related to the inference problem, but… blocking phrases that ask for input and then return (like player consents, but dealing with non-boolean types) would be helpful in a wide variety of cases, possibly including this one (perhaps you could request a pushable thing from the player in for supplying a missing noun, for example).

[zarf (Andrew Plotkin)]

This is not a good use for a blocking input phrase. Any disambiguation question should allow for the possibility that the player will ignore it and type a whole new command. Also you might need multiple stages of questions. (“With what?” KNIFE “Which do you mean, the rusty knife or the nasty knife?” NASTY)

The feature really has to be tightly integrated with the parser loop.

[otistdog]

A prototype of using SMN/SMSN to override inference is pretty straightforward. There might be some edge case bugs, but as a proof-of-concept, it works.

With some test rules:

For supplying a missing noun while smelling (this is the smell the environment if no noun supplied rule):
	now the noun is the touchability ceiling of the current actor.

For clarifying the parser's choice of a room while smelling:
	do nothing.

For supplying a missing second noun while inserting into (this is the choose a random container if no second noun supplied rule):
	let vessel be a random ((touchable container that is not the noun) that is not enclosed by the noun) that does not enclose the noun;
	unless vessel is nothing:
		now second noun is vessel.

For supplying a missing second noun while putting on (this is the choose a random supporter if no second noun supplied rule):
	let platform be a random ((touchable supporter that is not the noun) that is not enclosed by the noun) that does not enclose the noun;
	unless platform is nothing:
		now second noun is platform.

the scenario yields interaction such as:

Lab
A typical Mad Scientist's laboratory.

You can see a beaker washer and a scarred lab table (on which are a 100mL beaker, a 250mL beaker and a 500mL beaker) here.

>SHOWVERB SMELL
Verb 'smell' 'sniff' 
	 * noun -> Smell

>ACTIONS
Actions listing on.

>SMELL
[smelling Lab]
You smell nothing unexpected.
[smelling Lab - succeeded]

>ACTIONS OFF
Actions listing off.

>SHOWVERB PUT
Verb 'put' 
	 * 'on' noun=Routine(344354) -> Wear
	 * noun=Routine(344404) 'on' -> Wear
	 * 'down' multiheld -> Drop
	 * multiheld 'down' -> Drop
	 * multiexcept 'in' / 'inside' / 'into' noun -> Insert
	 * multiexcept 'on' / 'onto' noun -> PutOn

>PUT CLIPBOARD
(in the 250mL beaker)
You put the clipboard into the 250mL beaker.

>PUT CLIPBOARD ON
(the scarred lab table)
(first taking the clipboard)
You put the clipboard on the scarred lab table.

>PUT 100ML
(in the 250mL beaker)
(first taking the 100mL beaker)
You put the 100mL beaker into the 250mL beaker.

>PUT 250ML
(in the 500mL beaker)
(first taking the 250mL beaker)
You put the 250mL beaker into the 500mL beaker.

>PUT 500ML
What do you want to put the 500mL beaker in?

>TAKE
What do you want to take?

>500ML
Taken.

>I
You are carrying:
  a 500mL beaker
	a 250mL beaker
	  a 100mL beaker

Note that I redefined the grammar for “put” to require a [wearable thing] so that the parser would stop wanting to infer the wearing action all the time.

Also note that it’s happenstance that the smaller beakers ended up inside larger ones, since that’s not being enforced… which brings up a general point: If you want some intelligent inference to happen, you have to provide the logic for it somewhere, which means becoming familiar with the rules already in place and how activities are processed, so that you can craft special case rules.

EDIT: Also also note, the beaker washer is just a thing, not a container or supporter, which is why it’s ignored when inserting it into or putting it on.

[Zed (Zed Lopez)]

Came back to this thread and found a never-finished draft I’d left unposted.

@otistdog

No, it’s definitely possible. The drawback is that it requires a mini-library of DTPM rules that cover the built-in grammar lines.

My suspicion of difficulty was in regard to the notion of recontextualizing the supplying a missing noun activities, per se, 'cause those could potentially set the a noun to an object that hadn’t been in scope. (And if we were handling missing nouns at this stage, there’d be no remaining point to those activities occurring during generate action.)

@otistdog

The case of >PUT CLIPBOARD ON is interesting. Assuming that it’s not a wearable object, a person would probably judge this far more likely to be an incomplete command lacking a second noun than a complete but nonsensical command.

Like I said above, it’d be ideal if we didn’t need to manually specify missing-noun variants of grammar lines, but for the parser to automatically prompt for a noun. Then we could change that grammar line to:

Understand "put [wearable thing] on" as wearing.

and a PUT <non-wearable-thing> ON command would get something like

What do you want to put it on?

---

New stuff, now.

@otistdog

Are you saying that inference should consult SMN and SMSN so that a Standard Rule like the ambient odour rule would be invoked to set the noun while inferring for the grammar line Understand "smell [something]" as smelling instead of relying on the existence of the “short grammar” from Understand "smell" as smelling and making the noun assignment post-parsing?

Sort of. My thinking is functionality like that provided by those activities is what should happen instead of inference as it exists today. But I didn’t mean that the parser should literally invoke the existing missing noun activities while the tests to potentially invoke those same activities during ActionPrimitive live on. (I’m not sure whether there’d still be a point to having such a facility during action processing if the parser had this, but maybe there’s a compelling reason I’m not thinking of.)

@otistdog

A prototype of using SMN/SMSN to override inference is pretty straightforward. There might be some edge case bugs, but as a proof-of-concept, it works.

Care to share the prototype that enables this? I took a whack at it and got it to work almost like yours, but it refuses to consider the scarred lab table as second noun with >PUT CLIPBOARD ON because the lab table does enclose the clipboard at that point so it prompts

What do you want to put the clipboard on?

instead.