ISHML: New JavaScript Library for Parser IF

I’ve just released version 1 of a JavaScript Library for Interactive fiction and I thought I’d share some of the details with you.

ISHML stands for Interactive Story grapH Management Library, but call it “Ishmael.” Its intent is to facilitate the creation of interactive fiction in JavaScript and is intended for client-side applications running in modern browsers.

The ISHML library is a fluent API with straightforwardly named properties and methods, many of which are chainable.

Eventually, ISHML will address all aspects of creating interactive fiction. For now, though, ISHML is just a really flexible and powerful recursive descent parser with backtracking, which is capable of returning multiple interpretations of a given input text.

In ISHML, you create a parser by defining a grammar. A grammar is a set of nested rules that describes the syntax tree to be generated during parsing. The structure of the grammar mirrors the structure of the syntax tree. Rules are, in spirit, a JavaScript adaptation of BNF notation.

Here's a small example of a grammar.
//Create a set of nested rules which mirror the wanted syntax tree.
var command = ISHML.Rule()
command.snip("verb").snip("nounPhrase")
command.nounPhrase.snip("article").snip("adjectives").snip("noun")

//Configure behavior of some of the rules with .configure().

command.verb.configure({ filter: (definition) => definition.part === "verb" })

command.nounPhrase.article
  .configure({ minimum: 0, filter: (definition) => definition.part === "article" })

command.nounPhrase.adjectives
  .configure(
  {
    minimum: 0, maximum: Infinity,
    filter: (definition) => definition.part === "adjective"
  })

//alternatively the rule's options may be set directly.
command.nounPhrase.noun.filter=(definition) => definition.part === "noun"

There are many, many ways to configure the rules.

If you are looking to write Parser IF in JavaScript but don’t want to hand code a parser from scratch, this may be the library you’ve been looking for. I won’t say writing a good grammar is easy. It isn’t. However, it’s definitely easier than writing a parser. There is a lengthy tutorial and API reference here:

https://whitewhalestories.com

I welcome all feedback.

PS Of course it’s open source.

3 Likes

This looks amazing!

One website thing–the link to the API at the end of Tutorial 2 (I think) goes to “API” rather than “API.html”, which makes it 404.

1 Like

Fixed now, thanks.

Hi,

Nice work!

There appears to be a problem with the preposition semantics in part2. The line

var prepositions=new Set(gist.verb.prepositions)

Doesn’t get anything and causes the function to fail. For example; "take ruby from slipper"

Also,

I had a go at upgrading nounPhrase to be something like:

ADJNOUN := [ARTICLE] [ADJS] NOUN
NP := ADJNOUN [PREP NP]

But i can’t find a way to make a rule reference itself. Having the weaker notion of;

NP := ADJNOUN [PREP ADJNOUN]

Works, but then some interpretations are missed.

Here are my attempted hacks;

var adjNoun=ISHML.Rule()

adjNoun
    .snip("article").snip("adjectives").snip("noun")

adjNoun.article
    .configure({minimum:0, filter:(definition)=>definition.part==="article" })
adjNoun.adjectives
    .configure(
    { minimum:0, maximum:Infinity,
            filter:(definition)=>definition.part==="adjective"
    })

    adjNoun.noun.configure({filter:(definition)=>definition.part==="noun" })

    var nounPhrase=ISHML.Rule()
    
nounPhrase.snip("NP", adjNoun).snip("relation")
    nounPhrase.relation.snip("preposition").snip("RNP",adjNoun)

// Would like above to be:
// nounPhrase.relation.snip("preposition").snip("RNP",nounPhrase)

nounPhrase.relation.configure({minimum:0})
nounPhrase.relation.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})


var command=ISHML.Rule()

command.snip("subject",nounPhrase).snip("verb").snip("object")
command.subject.configure({minimum:0})
command.verb.configure({filter:(definition)=>definition.part==="verb"})
command.object.configure({minimum:0,mode:ISHML.enum.mode.any})
    .snip(1)
    .snip(2)

command.object[1].snip("directObject",nounPhrase).snip("indirectObject")
command.object[1].indirectObject.snip("preposition").snip("nounPhrase",nounPhrase)
command.object[1].indirectObject.configure({minimum:0})
command.object[1].indirectObject.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})

    command.object[2].snip("indirectObject",nounPhrase).snip("directObject",nounPhrase)

    var parser=ISHML.Parser({lexicon:lexicon,grammar:command})

I’ll fix the semantics example. Thanks for letting me know.

For the other issue. When you create a rule from another rule, it makes a clone of the source rule rather than referencing it directly. Generally, this is much safer and easier for beginners, but then you can’t do this sort of recursive deconstructing of the nounPhrase that you want.

Fortunately, this is an easy change to make to the library. I should have a new version out in a couple of hours.

Really do appreciate the feedback!

1 Like

Based on feedback I’ve made some updates to the library and tutorial. If you’ve been playing around with ISHML, you’ll want to change your script tag to:
<script src="https://cdn.jsdelivr.net/gh/bikibird/ishml@1.1.2/src/ishml.js"></script>

.snip(key, rule) no longer automatically clones a rule. Unless you are creating a self-referencing rule you should snip this way: .snip(key,rule.clone()). The tutorial has been updated to reflect this.

Many thanks to @jkj_yuio for their feedback.

If you update to version 1.1.2 and use the script below, you should be able to get the tree you were going for. It returns 3 interpretations for “take slipper from ruby to jim”. Personally, I wouldn’t have used a self-referencing rule. Instead, I would have tried making relation a repeating item, but that’s just because recursion makes my head hurt.

Let me know if you disagree with my results. I really like this example and will probably use it when I write part 3 of the tutorial.

var adjNoun=ISHML.Rule()

adjNoun
    .snip("article").snip("adjectives").snip("noun")

adjNoun.article
    .configure({minimum:0, filter:(definition)=>definition.part==="article" })
adjNoun.adjectives
    .configure(
    { minimum:0, maximum:Infinity,
            filter:(definition)=>definition.part==="adjective"
    })

adjNoun.noun.configure({filter:(definition)=>definition.part==="noun" })

    var nounPhrase=ISHML.Rule()
    
nounPhrase.snip("NP", adjNoun.clone()).snip("relation")

nounPhrase.relation.snip("preposition").snip("RNP",nounPhrase)

nounPhrase.relation.configure({minimum:0})
nounPhrase.relation.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})


var command=ISHML.Rule()

command.snip("subject",nounPhrase.clone()).snip("verb").snip("object")
command.subject.configure({minimum:0})
command.verb.configure({filter:(definition)=>definition.part==="verb"})
command.object.configure({minimum:0,mode:ISHML.enum.mode.any})
    .snip(1)
    .snip(2)

command.object[1].snip("directObject",nounPhrase.clone()).snip("indirectObject")
command.object[1].indirectObject.snip("preposition").snip("nounPhrase",nounPhrase.clone())
command.object[1].indirectObject.configure({minimum:0})
command.object[1].indirectObject.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})

    command.object[2].snip("indirectObject",nounPhrase.clone()).snip("directObject",nounPhrase.clone())

    var parser=ISHML.Parser({lexicon:lexicon,grammar:command})

Great work!

I’m going to give this idea a go later. I might also try your “repeater” suggestion, although i know that I’m also going to need recursive rules too, so i think your clone() idea is the way to go.

Usually either left or right recursion gets into trouble, depending on the implementation. Or, at least one form is a lot more efficient than the other.

For example;

FOO := FOO BAR

vs

FOO := MOO FOO

In the first example, sometimes FOO left recursion will fail as no tokens are consumed, and sometimes (depending on the implementation) it doesn’t

In the second example, when it works, there can sometimes be a stack of MOOs, and sometimes not (depending on the implementation).

I’ll fiddle around and report.

1 Like

So,

For the while, I’ve taken out the second indirect form, as this is rare. So to concentrate on adding relativizers (eg “cat in the hat”) and the enigmatic and

You’re right about repeaters. I used this for “and”, and it can also be used for a chain of relativizers, but recursive is slightly better here as later, each of those relative clauses need to be semantically resolved. In a chain, it’s a bit more confusing.

“and” is ok because it’s associative.

   var lexicon=ISHML.Lexicon()
lexicon
lexicon
    .register("the", "a", "an").as({ part: "article" })
    .register("take", "steal", "grab")
        .as({ key: "take", part: "verb", prepositions: ["to", "from"] })
    .register("drop", "leave").as({ key: "drop", part: "verb", prepositions: [] })
    .register("ring").as({ key: "ring", part: "noun", role: "thing" })
    .register("slipper").as({ key: "slipper", part: "noun", role: "thing" })
    .register("diamond").as({ key: "ring", part: "adjective", role: "thing" })
    .register("diamond jim").as({ key: "jim", part: "noun", role: "npc" })
    .register("jim").as({ key: "james", part: "noun", role: "npc" })
    .register("ruby").as({ key: "ring", part: "adjective", role: "thing" })
    .register("ruby").as({ key: "ruby", part: "noun", role: "thing" })
    .register("ruby").as({ key: "slipper", part: "adjective", role: "thing" })
    .register("glass").as({ key: "slipper", part: "adjective", role: "thing" })
    .register("glass").as({ key: "tumbler", part: "noun", role: "thing" })
    .register("looking glass").as({ key: "mirror", part: "noun", role: "thing" })
    .register("good looking").as({ key: "tumbler", part: "adjective", role: "thing" })
    .register("good").as({ key: "mirror", part: "adjective", role: "thing" })
    .register("tumbler").as({ key: "tumbler", part: "noun", role: "thing" })
    .register("ruby").as({ key: "miss_ruby", part: "noun", role: "npc" })
    .register("sam").as({ key: "sam", part: "noun", role: "npc" })
    .register("from").as({ key: "from", part: "preposition" })
    .register("to").as({ key: "to", part: "preposition" })
    .register("in").as({ key: "in", part: "preposition" })
    .register("and").as({ key: "and", part: "conjunction" })

        // a single noun with adjectives
var aNoun=ISHML.Rule()
aNoun.snip("article").snip("adjectives").snip("noun")
aNoun.article
    .configure({minimum:0, filter:(definition)=>definition.part==="article" })
aNoun.adjectives
    .configure(
    { minimum:0, maximum:Infinity,
            filter:(definition)=>definition.part==="adjective"
    })
    aNoun.noun.configure({filter:(definition)=>definition.part==="noun" })

    // a noun with optional relativizer
var arNoun=ISHML.Rule()
arNoun.snip("AN", aNoun.clone()).snip("relation")
arNoun.relation.snip("preposition").snip("RN",arNoun)
arNoun.relation.configure({minimum:0})
arNoun.relation.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})


    // a list of nouns
    var nouns=ISHML.Rule()
    nouns.snip("ARN", arNoun.clone()).snip("andARN")
    nouns.andARN.snip("and").snip("ARN", arNoun.clone())
    nouns.andARN.configure({minimum:0,maximum:Infinity})
    nouns.andARN.and
    .configure({filter:(definition)=>definition.part==="conjunction"})

var command=ISHML.Rule()

command.snip("subject",arNoun.clone()).snip("verb").snip("object")
command.subject.configure({minimum:0})
command.verb.configure({filter:(definition)=>definition.part==="verb"})

command.object.snip("directObject",nouns.clone()).snip("indirectObject")
command.object.indirectObject.snip("preposition").snip("nouns",nouns.clone())
command.object.indirectObject.configure({minimum:0})
command.object.indirectObject.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})

    var parser=ISHML.Parser({lexicon:lexicon,grammar:command})

    // console.log(parser.analyze("take ruby from slipper"))
    // console.log(parser.analyze("take ruby ring in glass slipper and diamond ring"))
    //console.log(parser.analyze("take ruby from ring in slipper"))
    // console.log(parser.analyze("take ruby and ring in slipper from jim"))
    // missing ((ruby and ring) in slipper) from (jim)

Results:

  1. “take ruby from slipper”
    3 interpretations.

  2. “take ruby ring in glass slipper and diamond ring”
    3 interpretations

  3. “take ruby from ring in slipper”
    3 interpretations

  4. “take ruby and ring in slipper from Jim”
    3 interpretations

Unfortunately we’re missing a few. For example (4) we have already

take (ruby) and (ring in slipper) from (Jim)
take (ruby) and (ring in slipper from Jim)
take (ruby and ring) in (slipper from Jim)

but not;

take ((ruby and ring) in slipper) from (Jim)

This last case is a bit like the phrase “put the cup and saucer in the cupboard on the table.” meaning;

put the ((cup and saucer) that are in the cupboard) onto the table,

However, this is not a bug because the grammar defined above only relativizes single nouns. I tried to define half of nouns before arNoun, but it didn’t like it.

The problem with recursively defined grammars like this is that they become too recursive to handle.

failed attempt:

    var nouns=ISHML.Rule()

    // a noun with optional relativizer
var arNoun=ISHML.Rule()
arNoun.snip("AN", nouns).snip("relation")
arNoun.relation.snip("preposition").snip("RN",nouns)
arNoun.relation.configure({minimum:0})
arNoun.relation.preposition
    .configure({filter:(definition)=>definition.part==="preposition"})


    // a list of nouns
//    var nouns=ISHML.Rule()
    nouns.snip("ARN", arNoun.clone()).snip("andARN")
    nouns.andARN.snip("and").snip("ARN", arNoun.clone())
    nouns.andARN.configure({minimum:0,maximum:Infinity})
    nouns.andARN.and
    .configure({filter:(definition)=>definition.part==="conjunction"})

Wow, this is great. This is making my head spin a bit, though.

I think the issue is that you need to wrap nouns in something like this, but come up with better names than I did.

var superNouns=ISHML.Rule()
superNouns.snip("nouns", nouns.clone()).snip("relation")
superNouns.relations.snip("preposition").snip("RN",arNoun)

//And configure, etc.

Or something like that. So an individual noun can have a chain of relativizers, but so can the whole array, nouns.

Question: is there a reason to use a recursive descent parser instead of Earley’s algorithm? I switched to Earley when I was running into these sorts of troubles, and while the underlying code is a bit more convoluted, I haven’t yet found a grammar it couldn’t handle. (It’s also O(n^3) in the general case, which is nice.)

Yes, it’s a naive implementation of a top-down parser. Readability/maintainability of code and ease of use were my top concerns. (Also, naive happens to fit my skill set.)

I chose simplicity over speed. It’s not intended for writing a compiler, just for parsing the relatively small amounts of text associated with Parser IF.

After I finish the rest of the ISHML library, I may go back and look at speeding up the parser, especially if people report that it’s bogging down on a realistically sized piece of IF. I will keep Earley in mind. Thanks for the tip.

Didn’t quite understand this. So far we’ve been discussing how best to write a grammar to get the results that are wanted. I don’t quite follow what that has to do with the implementation of the parser. Unless I’ve misunderstood the feedback so far, the parser seems to be giving back the results implied by the grammar. It’s just that writing a good grammar is challenging. @jkj_yuio, did I misinterpret some of your results?

Fair enough!

Oh, I was just referring to this:

Earley’s algorithm is best-known for being able to deal with both, though left-recursion is just a bit more efficient. In other words, it would be removing restrictions on the grammar, not making it more effective on any particular grammar.

Well that’s an awesome reason to adopt it. I was looking at left vs. right just now. I agree, the ISHML parser cannot handle left recursive rules. The following simple grammar example illustrates that. The goat rule is right recursive. The sheep rule is left recursive.

var lexicon = ISHML.Lexicon()
lexicon
    .register("baa").as({animal: "sheep", part:"bleat" })
    .register("baa").as({animal: "goat", part:"bleat" })
  
var goat=ISHML.Rule()
goat.configure({minimum:0,maximum:5}).snip("bleat").snip("goat",goat)
goat.bleat.filter=(definition)=>definition.animal==="goat"

var sheep=ISHML.Rule()
sheep.configure({minimum:0,maximum:5}).snip("sheep",sheep).snip("bleat")
sheep.bleat.filter=(definition)=>definition.animal==="sheep"

//Returns 5 interpretations
console.log(ISHML.Parser({lexicon:lexicon,grammar:goat}).analyze("baa baa baa"))

//Returns 2,355,510 interpretations in about 15 seconds
console.time()
console.log(ISHML.Parser({lexicon:lexicon,grammar:goat}).analyze("baa baa baa baa baa baa baa baa baa baa baa baa baa baa"))
console.timeEnd()

//Exceeds call stack
console.log(ISHML.Parser({lexicon:lexicon,grammar:sheep}).analyze("baa baa baa"))

Conclusion: Goats stack, sheep don’t.

2 Likes

Yeah. I always recommend Robert Heckendorn’s Practical Tutorial on Context Free Grammars which explains common idioms and gives a general approach to designing a grammar. It’s also more readable and less unnecessarily technical than any other introduction to the subject that I’ve found.


Some self-indulgent comments on the Earley algorithm since I’m kind of a parsing-algorithm-geek…

It generally has more overhead than a recursive-descent parser. So it’s usually not faster, though it does avoid the worst-case exponential-time cases.

The big thing is that it removes the restrictions on the grammar. It can parse any CFG (that is, anything you can write in the Symbol -> Other Symbols | Yet More Symbols form) which is nice. But that also means that it will cheerfully let you write ambiguous grammars and will return all the possible parses (which you then have to deal with). So it’s not all roses.

Nearley.js is a decent implementation, if you want an existing one.

It’s possible to build a very small Earley parser: my little toy one is a bit over 200 lines of JavaScript.

And Loup Vaillant-David’s Earley Parsing Explained is a fair explanation of the algorithm.

But yeah, there’s nothing too wrong with the basic top-down recursive-descent algorithm. ISHML looks pretty cool as far as I’ve gotten time to play with it.

2 Likes

I had a go with some of these examines in an Earley grammar, using this implementation.

My example recursive grammar;

#define TERMINALS "the a and get take in from ruby ring slipper jim"
    
static const char *description =
"\n"
"TERM bogus " TERMINALS ";\n"
"S  : V RNS     # s(0 1)\n"
"   | V RNS IND  # s(0 1 2)\n"
"   ;\n"
"IND : PREP RNS   # ind(0 1)\n"
"   ;\n"
"AA : ADJ      # adj(0)\n"
"   | AA ADJ   # adj(0 1)\n"
"   ;\n"
"DN : N        # noun(0)\n"
"   | DET N    # noun(1)\n"
"   | DET AA N # noun(2 1)\n"
"   ;\n"
"RN : DN        #0\n"
"   | RNS PREP RNS  # rel(1 0 2)\n"
"   ;\n"
"RNS : RN       #0\n"
"    | RNS CONJ RN  # and(0 2)\n"
"    ;\n"    
"DET : the     #0\n"
"    | a       #0\n"
"    ;\n"
"CONJ : and    #0\n"
"    ;\n"    
"V  : get      # verb(0)\n"
"   | take     # verb(0)\n"
"   ;\n"
"PREP : in     #0\n"
"     | from     #0\n"
"   ;\n"
"ADJ : ruby    #0\n"
"   ;\n"
"N  : ring     #0\n"
"   | ruby     #0\n"
"   | slipper    #0\n"
"   | jim   #0\n"        
"   ;\n"    
    ;

Results:

 "take ruby and ring in slipper from jim"

    (s (verb take) (and (noun ruby) (noun ring)) (ind in (rel from (noun slipper) (noun jim))))

    (s (verb take)
        (rel in (and (noun ruby) (noun ring)) (rel from (noun slipper) (noun jim)))

        (rel from
            (rel in (and (noun ruby) (noun ring)) (noun slipper))

            (and (noun ruby) (rel in (noun ring) (noun slipper)))
             (noun jim))

        (and (noun ruby)
            (rel in (noun ring) (rel from (noun slipper) (noun jim)))

            (rel from (rel in (noun ring) (noun slipper)) (noun jim))
            )
        )

    (s (verb take)
        (rel in (and (noun ruby) (noun ring)) (noun slipper))

        (and (noun ruby) (rel in (noun ring) (noun slipper)))
         (ind from (noun jim)))

8 parses, what larks!

as @ JoshGrams points out, you don’t really need it this general as, most of the time, you can just express each term in terms of subordinates without a “recursive loop”.

For example;

RN -> RNS PREP RNS for relativization is overkill, and RN -> RN PREP RN would do just fine for games. This would leave the ambiguities to be relativization vs indirect object and most of that can be resolved by eager semantic validation.

Nothing really wrong with a standard recursive desent method. You can even build quite a good game parser in yacc/bison using a few tricks.

Oh, I somehow never connected the “JoshuaGrams” of that implementation to your username! Props to you; I’m currently doing some computational linguistics work using an Earley parser (which is why I’m such a fan of the algorithm) and your page was one of the main resources I used while building it.

4 Likes

I’ve created a part three for the parsing tutorial https://whitewhalestories.com/parsing3.html.

It focuses on recursive rules and includes a simple calculator grammar. I also cleaned up some minor errors in parts one and two and added a few syntax diagrams for fun.

I’ve really appreciated all the thoughtful (and thought provoking!) comments on this thread. There is nothing else like this forum and its community.

Hi!

Nice update.

I had a go at adding unary minus to calculator, eg:

console.log(parser.analyze("2+3*-(2+3)",{lax:true, greedy:true}))

and

console.log(parser.analyze("1--2",{lax:true, greedy:true}))

var lexicon=ISHML.Lexicon()
lexicon
	.register("0","1","2","3","4","5","6","7","8","9").as({part:"digit"})
	.register("(").as({part:"begin"})
	.register(")").as({part:"end"})
	.register("+").as({part:"termOp", operation:(a,b)=>a+b})
	.register("-").as({part:"termOp",operation:(a,b)=>a-b})
	.register("*").as({part:"factorOp",operation:(a,b)=>a*b})
	.register("/").as({part:"factorOp",operation:(a,b)=>a/b})
	.register("^","**").as({part:"powerOp",operation:(a,b)=>a**b})
    .register("-").as({part:"uniMinus"})


var expression=ISHML.Rule()
var term=ISHML.Rule()	
var power=ISHML.Rule()
var group=ISHML.Rule()
var number=ISHML.Rule()
var ugroup=ISHML.Rule()

expression.snip("term",term).snip("operations")
expression.operations.snip("operator").snip("operand",term)
	.configure({minimum:0, maximum:Infinity,greedy:true})
expression.operations.operator.configure({filter:(definition)=>definition.part==="termOp"})

term.snip("power",power).snip("operations")
term.operations.snip("operator").snip("operand",power)
	.configure({minimum:0, maximum:Infinity, greedy:true})
term.operations.operator.configure({filter:(definition)=>definition.part==="factorOp"})

power.snip("group",group).snip("operations")
power.operations.snip("operator").snip("operand",power)
	.configure({minimum:0, greedy:true})
    power.operations.operator.configure({filter:(definition)=>definition.part==="powerOp"})

    group.snip("minus").snip("ugroup",ugroup)
    .semantics=(interpretation)=>
	{
    if (interpretation.gist.minus)
    {
    	interpretation.gist=-interpretation.gist.ugroup
    }
    else
    {
        interpretation.gist=interpretation.gist.ugroup
    }
	return interpretation
	}

    group.minus.configure(
    	{minimum:0, maximum:1, filter:(definition)=>definition.part==="uniMinus" }
    )


ugroup.configure({mode:ISHML.enum.mode.apt})
	.snip(1)
	.snip(2,number)
ugroup[1].snip("begin").snip("expression",expression).snip("end")
ugroup[1].begin.configure({keep:false,filter:(definition)=>definition.part==="begin"})         
ugroup[1].end.configure({keep:false,filter:(definition)=>definition.part==="end"}) 

number.configure({maximum:Infinity,greedy:true,filter:(definition)=>definition.part==="digit"})
	.semantics=(interpretation)=>
	{
		interpretation.gist=Number(interpretation.gist.map(({lexeme})=>lexeme).join(""))
		return interpretation
	}

getOperation=(interpretation)=>
{
	interpretation.gist=interpretation.gist.definitions[0].operation
	return interpretation
}

calculate=(interpretation)=>
{
	var {gist} = interpretation
	var {expression, term, power, group, operand,operations}=gist
	var result=expression || term || power || group ||operand|| gist
	if (operations)
	{
		if (operations instanceof Array)
		{
			operations.forEach(function(operation)
			{
			result=operation.operator(result,operation.operand)
			})
		}
		else
		{
			result=operations.operator(result,operations.operand)
		}	
		
	}
	interpretation.gist=result
	return interpretation
}

expression.operations.operator.semantics=getOperation
term.operations.operator.semantics=getOperation	
power.operations.operator.semantics=getOperation

expression.semantics=calculate
term.semantics=calculate    
ugroup.semantics=calculate
power.semantics=calculate

var parser=ISHML.Parser({lexicon:lexicon,grammar:expression}) 

There might be a better way than my hacky “minus” semantic.

Your solution looks pretty good actually. I found this article helpful when thinking about order of operations: http://mathforum.org/library/drmath/view/53194.html. I think you’ve got it right?

I had a longer version of the calculator that allowed real numbers and treated the minus sign as part of the definition of number. That seemed to work, but I didn’t include it in the tutorial because it added a lot of length to the example without providing more enlightenment. However, that is not quite the unary operator.


Ideally, I want to combine the calculator grammar with the command grammar so that you can issue commands like “calculate 1+2”. However, when I started thinking about it, I realized there was a deficiency in how the tokenizer works. ISHML has the notion of separators (characters, like white space, between terms, which are thrown away), but not word boundaries. Using the lax setting to make white space optional sort of works, but is problematic because a command like “takerubyslipper” would tokenize successfully.

Really, I’m starting to think the word boundary needs to be part of the definition in the lexicon. I would probably default it to white space, but you would have the option of specifying a regex.

If anyone thinks there is a better way, please, speak up!