Need advice to expand my DSL ScribeScript to be more writer-friendly

Technical Development Development Systems
1

Hey everyone,

Over the past weeks I have been hard at work developing ChronicleHub. There was a lot of bug fixing, extremely valuable feedback and some poking at the system left and right.

The platform has a DSL called “ScribeScript”, which evolved out of a shorthand parser language I made up when I was only using Google Sheets to write storylets for a now abandoned project, and which informed my design of the current data model.

Now ScribeScript has evolved to be increasingly powerful, allowing it both to handle basic text substitution like this:

A manila folder sat on my desk. It had been dropped there by the lieutenant while I was acquainting myself with my morning coffee. 
_"Another murder, {$last_name},..."_ he told me when I asked what surprise he had for me _"...just your thing."_

It looked at me. Pictures and reports were already poking out on all sides.

And more complex operations that define game mechanics like this:

{@drunk = {%pick[involved; 1, $.secret == drunk_backing]} },
{ @drunk != nothing :
    {@villain = {%pick[involved; 1, {$.secret.implicated} == 1]} },
    { @villain != nothing : {${@drunk}.role} = {${@villain}.role} | {${@drunk}.role} = {%pick[role_definitions; 1, $. < 1]} }
},

However, I have a programmer’s mindset in addition to a creative writer’s one. Code like this makes complete sense to me, but I might be blind to how well the language works for other people.

The other day, someone asked about a bug and why an effect did not apply to a variable. Turns out they had used $variable == 1, in the effects string, rather than $variable = 1,

In most programming languages I know, == is used for comparison and = for assignment. But that might not be obvious to everyone. In fact, even I make that mistake when I’m not paying attention.

Because of this I am considering whether to include something like natural language syntax, so you get operations like $variable becomes 1, or { $variable equals 1 : Conditional Text }

I would really appreciate people’s ideas and what would help them or make the language more convenient to use. I would love to hear your thoughts!

2 Likes
2

Will try to write a more thoughtful reply later, but at first sight: I’m a programmer, and your DSL still looks to me like the result of a head-on collision between Perl and JSON. There’s less noise in Zig, and that’s a pretty noisy language. You’re just making more work for yourself, too, with all that syntax. Your parser has to handle it all.

More specifically, plenty of languages use = to compare for equality. Like Basic, that also uses it for assignment (recursive descent parsing for the win!) or Pascal, that has a separate assignment operator mainly for clarity. In fact I strongly suspect that the need for separate operators in curly-brace languages was due to C being parsed with yacc.

But no, I don’t think laypeople need to be outright coddled when it comes to syntax. It’s okay for a programming language to look like one. Think Rexx, Lua… or Alan 3.

1 Like
3

Admittedly, I originally wrote the parser so I could do simple conditional blocks in Quality Based Narrative style games.

Just very simple syntax that was essentially an anonymous else / if ( ... ) { ... } block, using curly braces to tell the parser to evaluate everything inside as logic.

image
image990×672 80.6 KB

But, as I started building the platform, I realised I could conditionally render my own game rules and set or get qualities based on variable conditions. Then I introduced alias assignment to help write more compact conditional blocks, then some macros, and not long after, I found myself using the “effects” field to conditionally program the gamestate.

{C6233DBC-59C5-4E33-A013-279DC836538B}
{C6233DBC-59C5-4E33-A013-279DC836538B}1110×861 192 KB

This also led me to me revisit the way the effects field is parsed, as I found it was sequentially resolving variables based on the depth of the curly braces { ... }, where the deepest nested effects were resolved first, and worked outward until it was a clean effects string. This was intended behaviour originally, as that is precisely how I wrote it and wanted it to work in a regular text field, but the effects field should handle logic sequentially, so each statement between commas can affect the next.

I’d like to note I never intended to go this deep, and the language really is mostly intended for simple variable text insertion. It also very explicitely was written to avoid reserving double " and single ' quotation marks a special role in syntax, such as marking out a string, as I found this much more convenient when it came to writing dialogue, so they wouldn’t need to be escaped.

But, it is an interactive fiction platform, so ideally I want to make sure it’s convenient to use for writers. I don’t intend for them to write elaborate effects strings like the one above (and that really is me abusing my own system), but I want to make the base experience as frictionless as possible.

Also, in the original parser, = for both assignment and evaluation was treated equally, but this was something I couldn’t quite get the parser to work with correctly. Or maybe it was my learned behaviour getting in the way.

4

Ooh, okay. Templating languages can get cryptic fast. Look no further than mushcode, or MPI. (Interestingly, both are Lisp-like.) Never tried writing one of those (the one in Ramus just embeds JS for expressions), but I did play with a programming language concept without explicit keywords, using the C ternary operator for conditions and so on. It didn’t go far.

Not sure what to advise. Maybe walk back and focus on the templating functionality? The markup language of Ink might offer some inspiration, not that it’s much less convoluted.

5

So, just to clarify, what about this looks like a templating language?

6

I think time has shown that people can learn any scripting language if they have the right motivation to. ChoiceScript has a ton of keywords, Ink uses punctuation-mark operators with various arcane meanings, and StoryNexus did it all with drop-down menus, but none of them seems to have been a serious turn-off.

So my only specific suggestion would be to have stronger error-checking to capture mistakes like = vs ==. The types of the two expressions should be different, which should be enough to capture most slips there.

More broadly, I think there are three things people want in a scripting language:

  • Conciseness. This is the advantage = and == have over becomes and equals—it saves typing, and it keeps expressions from getting too long to handle.
  • Readability. The average line of code is written once but read many times. This is the advantage pick NUMBER from TABLE where CONDITION has over %pick[TABLE; NUMBER, CONDITION]: it’s immediately obvious which parameter does what.
  • Stability. Once people start using their system, they won’t want their old code to break. This is why a lot of Inform 6 syntax fails both of the first two conditions: the overall syntax can’t be cleaned up without messing up old projects.

How exactly you manage these tradeoffs is up to you. The people who have actually written games in ChronicleHub are probably best-placed to tell you what they’re having trouble with; good error-checking and some kind of autocomplete or syntax suggestions can help with all of them.

6 Likes
7

..The part where code blocks are used to insert dynamic text into otherwise static prose?! Or for that matter show alternative text depending on a condition?

8

Ah, yes, fair enough. I’d never seen that name used for it, but I can see the connection now.

I think it’s that exact quote used in a thread on Inform 7 that got me thinking about this.

This is good advice, thank you. I had already been trying to improve my error checking tools, but I may revisit that at a much more basic level. I feel my current logger might actually be a little overkill.

9

Is there a way to make the code more vertical instead of horizontal?
re: readability: being able to read it in an English-like way is helpful. For instance: if x >= 3 && y != 'foo' can be read “if x is greater than or equal to three and y is not equal to ‘foo’”. I’m not sure how to interpret that second screenshot.

10

So, the way I wrote the current parser, is that verticality should not be an issue. So, I am able to format the code as I like.

What I am showing there is as close to an abomination as I have been able to create, as I was trying to stretch the limits of my system to see where it broke (and it actually ended up breaking several times, and I realised I had overlooked several edge cases)

But, just taking a random snippet from the example shown:

image
image1019×132 36.5 KB

This can also be written as:

image
image544×503 43.4 KB

Which I admit is far more legible and made me realise I might have bracketed the wrong way. Oops.

Anyway, what this operation does is, it first assigns a random value to the quality $total_outcasts, which it stores as its level.

Then, since there will always be at least 1 level of $total_outcasts it doesn’t use a conditional block.

It assigns a temporary variable called @target1 which stores the quality ID which the %pick macro returns. %pick in this string will look for a quality assigned to the category involved, it will be asked to return only 1, where the quality picked (here referred to as the $. or the ‘self’ sigil) has a .secret property ($. can be extended with a property accessor without needing to use double .) which must be equal to an empty string. Ergo, our outcast target must be someone who doesn’t already have a secret role.

Then, it stores a temporary variable of a role, by picking 1 from outcast_definitions, with a level less than 1 and where the bitwise operation is less than 3 for which the logic is performed on the custom property called .ledger which exists on whatever quality @target1 could be. The “role ledger” is basically a very big number, which is actually a representation of all 4 narrative states a suspect can be in with a particular role. But looking at it now, I since changed some logic in the actual game I’m building, and need to alter this bit of ScribeScript.

Then we resolve @target1 to the actual ID by wrapping it in brackets and forcing its resolution into the stored value.

At this point I realised this piece of ScribeScript should not actually work, and I never got so far as to test it, because I had been really busy writing other parts of this game.

So yes. I actually should have formatted this sooner, and I might have spotted the bugs. Duped by my own parser language :joy:

4 Likes
11

I’m the one who had the (to me) baffling = vs == error. I get it and Randozart has good things going - some of the fields actually error check and alert for missing braces or if it can’t resolve the expression.

I had suggested eq or is for == and becomes or is now / isnow for transformations.

The dual booleans like >= are understandable, but for a prose-writer like me, gte is a little easier to type fast and reach on the keyboard than > = + - characters.

Storynexus did do it with stacks of conditions and drop down fields as I recall, similar to no code rule constructions that IIRC are used in both Quest and Adrift.

Seriously, you guys, ChronicleHub is going to be so good once the bugs are stomped! It’s partially responsible for poking me out of my writing slump.

2 Likes
12

There’s no reason why you cant have both gte and >=. I do this with conditionals. for example, you can say ?(FOO and BAR) or ?(FOO && BAR). Programmers like the latter, but the former is more readable.

On “=”, is there no way to make this work for both assignment and conditionals?

2 Likes
13

I’ll be honest, it probably can, but I’d have to make the parser be stricter about checking which context it’s in.

Just as a side note. What are nice alternative operators people would enjoy most?

Is shorthand like gte preferable, or full words like greater or equal

Either way, I’m warming up to adding dropdowns which just add ScribeScript under the hood.

14

I might use gte and similar because Twine/SugarCube does, so those will be familiar to a bunch of people? (scroll down a bit from Twinescript Operators to see the comparison operator list)

Edit: also, SugarCube allows both punctuation >= and letters gte – you said “alternative operators” so I assume that’s what you’re thinking but it’s nice because then people can use whichever they prefer…

2 Likes
15

I think Python hit a nice balance, using C’s comparison and assignment operators, but words and or not for logic.

1 Like
16

Oh, that’s a good call! People in the intfiction community are bound to be comfortable with those.

Even reminds me a bit of SQL. It’s worth giving consideration at least!

17

Another option is <- for assignment if you want to distinguish it from comparison. But really, imo there’s no ambiguity if your syntax distinguishes between expressions and statements, and most imperative languages do. (C is an outlier there and it’s caused no end of bugs, so now most compilers do make the distinction and warn about using = in expression contexts and vice versa.)

You can also just have several synonyms for each one (eq, is, ==, etc). If you do that, though, just be careful not to have two operators with very similar but distinct semantics, like JavaScript’s == and ===, or Python’s == and is. If you do, people will struggle to remember which ones are synonyms and which ones aren’t.

And remember that @, %, ^, $, #, and ? are used differently by many different languages, so those are the best single characters to assign your own eclectic meanings to, if you need something outside the standard set. Python defines @ as an operator with no default implementation so that people can assign their own meanings in various libraries, for example; a common one is cross product or matrix multiplication. Ink uses ? for the “contains” operator.

3 Likes
18

I think this might actually be very doable. I’ll look into this. I think it might add to the author experience. I’ll need to make a list of possible alternatives for each operator.

Oh definitely! And I have made gracious use of those. I think I have used ^ only for Ligature right now, not yet for ScribeScript