Semantic Highlighting in Inform 7

Some thoughts:

Re: look and feel: The example I made was definitely garish, to make it very obvious what was being proposed. I feel there’s a danger in making the effect too subtle, though, which might make differences between correctly and incorrectly declared things too easy to miss. However, a middle ground can certainly be found.

I put up a version that lets you swap out a number of different stylesheets (including colored text, styles, underlines, a more subtle version of the original highlighting, etc) here:

aaronareed.net/if/semantic-test2.html

Colored text is the way most IDEs with this feature go, but it looks somehow awkward to me in this context. Maybe it’s that, in combination with the existing color changes, it’s just much more colorful than we’re used to looking at. Interested to hear other people’s thoughts.

Re: spell/grammar check: I like the metaphor of copying the red/green underlines for spelling and grammar errors, although it’s not as obvious how you’d extend this to cover more cases (object highlighting etc.) As was mentioned, the Mac IDE already does spelling correction (I think it’s built-in to the OS); getting this standardized across platforms would be cool (although probably a different feature request).

Re: turning on and off: I agree this should be easy to do at will, perhaps with a hot key? Although I think this should be seperate from turning on/off background compilation. The highlighting would be most useful if you could very quickly switch it on to verify something, without having to wait for a compile.

The spelling/grammar checking also represents a problem: the new syntax highlighting will have to coexist peacefully with both the current minimalist highlighting and the spelling and grammar check underlining. (I’m not convinced of the utility of the latter—but then I don’t ever use it, in any application…)

I’m afraid that this syntax highlighting thing is going to have to be pretty darn ugly if it’s going to be useful…

Good point. Background compilation could have a checkbox in the preferences, while highlighting has a button and/or hotkey.

–Erik

Rmph… I just re-read my post, and I don’t think I was sounding irritated. I wasn’t irritated, in fact. I was just making a suggestion about how I thought the limited time of a group of volunteer developers might best be prioritized. Sorry if it came off as irritation.

With respect to the double-quote bug, this was caused, I’m sure, by a fix for the apostrophe-at-the-end-of-a-word-in-the-title bug, which has indeed been resolved. I was very happy to see it fixed! And being lazy, I decided that the new bug was about 10% as likely to trip anybody up as the old one, so I might as well leave well enough alone.

Not to split hairs, but I can see another interpretation of what you wrote. What you wrote was this: “I’m not really identifying much of anything that would qualify as an unresolved ‘library bug’ on Mantis at the moment.” That could mean, and probably does mean, “All of the library bugs that were reported following the release of 6E72 have been resolved.”

But if that’s what it means, then I will confess to being, if not irritated, at least perplexed. If there are no more outstanding library bugs, why not release a new version? Because there are still compiler bugs, I would guess. Okay, fair enough.

I’ve returned to I6, so it’s not a big deal for me either way. I was just chipping in on the subject of highlighting, and offering an unsolicited opinion about priorities. The reason for the latter opinion, however, was left implicit, so I should make it explicit.

In general, I feel that improvements in functionality are more important than improvements in the IDE. The IDE is already pretty darn good! What will aid authors most directly, I feel, are improvements in functionality. The rest is just icing on the cake. IDE improvements may save authors some time, but I’m not sure they’ll actually lead to better games.

–JA

I think there’s a basic but profound reason for this confusion: you have different definitions for “library bug”. A library bug is a mistake in the Standard Library, something that affects the model world. For example if the player would be able to pick up scenery, that would be a library bug. Your definition for a library bug is apparently wider than this, so you’re talking about two different things.

I searched the bug database for world model bugs and there were none - it’s not that the library bugs have all been fixed, it’s that none have been found. The first library bug I’ve seen after the previous release is https://intfiction.org/t/bugs-in-the-behavior-of-the-players-holdall/1178/1 and it hasn’t been reported yet.

Thanks - it has been reported now (and I deleted the bug report on Uservoice). Let me know if I did this one right:

inform7.com/mantis/view.php?id=278

(Upon checking, the current Mac IDE puts a red dotted underline beneath misspelled words, not green. So green is for grammar.)

Any suggestions for which Problem messages that will appear as some kind of markup in the source text, and how? Here’s a few.

would stick a green dotted underline beneath ‘when the player is in spot and the player is in outer-space’ of course. There’s a similar problem message that, for conditions with multiple pieces connected by AND but the compiler still understands a few of the smaller pieces, tacks on something like “…the part … was OK, the part … I couldn’t understand, the part … was OK…”, would of course underline only the incomprehensible piece of the complex conditional.

which would underline from the first word to the colon that the problem message says it found.

is very specific, and could underline only the ‘loking’ if it wanted.

could underline the action list, from and including the “except”, to, but not including, the “when”.

For “Every turn: Daphne ponders the computer.”

Underlines the whole invocation, as it doesn’t match any of the To phrases in the game. I see a lot of underlines starting with a semicolon and ending at the next semicolon, within rule bodies.

So that’s just a sampling of the ones I see a lot, with some concrete examples of how it could work. Aaron’s idea of hovering the mouse over the underlined portion for the Problem message in a tooltip is nice.

(Of course, the feature this thread is on calls for “color syntax highlighting” which is a little different from the “Problem message highlighting” I’m talking about here. But just as, for example, objects are written in one color and scene names in another color, so can invalid code be in another “color”. )

(And sometimes, one of the IDEs does highlight a problem paragraph – the whole paragraph – in reverse red (very hard to read!) rather than a more targeted piece-of-phrase. I think it does this when I click the orange hyperlink arrow the Problem message provides. I do not like that red-reverse very much, it is hard to read, though it does narrow things down a little bit when jumping to a whole new section of code.)

(Cap. Mikee: your bug has been fixed for the next release according to the site.)

I think it’s worth splitting the “semantic highlighting” feature request into “semantic highlighting of all code” and “semantic highlighting of problems”. The latter is much easier, obviously, and would probably be implemented first. But I don’t want to lose the pony for the trees.

Try this: eblong.com/zarf/tmp/highlight.html

It doesn’t try to color every bit of code on the screen; only the construct you’ve moused over. But then it shows all the construct you’ve moused over, up to the sentence level.

This is totally a mockup UI, obviously. In real life you’d want a tooltip-style delay before the popup appeared. Also the popup should stick around if you mouse down into it, so that you can click the doc and source links.

EDIT-ADD: Use this URL instead: eblong.com/zarf/i7index/highlight-example.html

I like this—it provides the functionality without the visual clutter.

–Erik

I like this too, as long as there’s also a way to indicate compile-time errors in some way without having to mouseover the text.

This already kinda exists in uservoice: inform7.uservoice.com/forums/573 … -code-with

(I love the pony for the trees line.)

Yeah – like I said, it’s worth treating “highlight problem code” and “highlight semantic features of all code” as separate features. (I will add a uservoice entry.) EDIT-ADD: Ah, there already is one, right. Thanks.

My sample incidentally demonstrates that this is a whole friggin’ lot of semantic information. Rendered in HTML (which is how I7’s index pages work), I’ve inflated 60 characters of source code to over 6 kilobytes of markup. If it’s integrated into the editor app, it would be more efficient, but there would still be a large lump of info for the editor to digest on every compile.

Ooh, yes, this is very slick. I like.

The only thing I think is lost from my version is the ability to see at a glance that you’ve done something wrong. (Take the “Old Man and the Sea” example, where you think you’ve created one object but have actually made two). It feels like this would only help out if you already suspected something was wrong and were trying to work out what it was.

Perhaps after each background compile, it could highlight-and-fade the names of newly created objects?

On the other hand, maybe I’m overstating the importance of this.

I thought about having a highlight color scale for phrase nesting. So each sentence would be pale yellow, a phrase within the sentence would be slightly darker yellow, subphrases would be darker, and so on – the same block structure I have now, but always visible.

That would distinguish a single object from two objects, at a glance. But it would also turn the source into a mosaic of colored bands. I guess I’ll toss up a modified demo, just to see.

I saw your comment about “newly-created” objects but I don’t think I understood it. Do you mean the earliest declaration of each object in the source code? Or objects added since the last compile?

I’m doubtful about the latter, because I compile a lot – and I’m looking at the right-hand pane, either to notice compiler errors or test what I just typed. Throwing in transient information on the left feels like a mistake. I’m losing information if I recompile, which means I’d either hesitate to recompile or ignore the information. Neither is great.

That is a lot of info. I got confused, at least until I figured out it’s putting, say, the whole “let…be…” line in every pop-up, at the bottom. I guess cause that line moves around so much – sometimes its at the top, alone, sometimes it’s at the bottom, etc., made it look like there was even more info than there was. Can you make a similar where the “largest” line is always at top, so they don’t shuffle their order as I slowly drag the mouse cursor from left to right?

Anyway, I think that amount of info should probably require a right-click. (That’s the “intellisense” that Pacian mentioned, right?) It’d be annoying to have all these auto-popups happening all the time.

I liked your shaded gradations idea, though I think maybe two would suffice for me: the words of the invoked To-phrase be black, the parameters to it be in almost-black. It would strengthen the resemblance of to-phrases to those fill-in-the-blank ad-libs.

I originally tried the lines in the other order. The problem is that the highlighted construct – the most specific one the user is pointing at – winds up on the bottom, and in an inconsistent location to boot. I wanted that to be on top; it’s what you’re actually looking at.

The shifting is annoying, I agree. On the other hand, if you’re right-clicking, it might not matter. Also remember that this is one line of code, which is an artificial example. If you had a screenful of the stuff, you’d be moving the mouse more vertically, which would produce random changes no matter what.

I tried the gradation idea, but it’s not practical. Four shades of yellow are almost indistinguishable when next to each other. I had to go from white to dark orange to get clear distinctions, and even that wouldn’t be enough with six or eight nesting levels.

I don’t think your notion of two shades will work either. You say “the invoked to-phrase”, but which one? The top-level one? That’s only going to be interesting to the player in a minority of cases. (It would do nothing for our canonical “old man and the sea” case.) In this one, only “let” and “be” would be highlighted. For top-level assertions, nearly every highlighted word would be an “is”.

Hmm, I’ve got to go the other way on this. I’ve used syntax highlighters in other programs, mostly text editors of course and some programming IDEs and the highlighter just seems to slow down the application. The larger the document, the longer the parsing time if highlighter updates in real time.

I also feel the Inform IDE does a splendid job of displaying necessary information as is. Adding highlighting I think would just “jumble up” the interface more.

Cheers.

I have to say that as the project I’m working on gets larger, the existing text formatting has gotten unbearably slow. Every time I type an open bracket or a quotation mark, I have to wait several seconds for the IDE to start accepting input again!

Whatever formatting or highlighting is done, it should be done at a very low priority so it doesn’t get in the way of editing. Perhaps highlighting could even be suspended whenever there are unbalanced quotes or brackets (or even indentation).

The Supercollider IDE takes a different approach. Source files are saved as RTF by default, and you can format them however you like. But if you hit cmd-’ or choose menu item Format -> Syntax Colorize, it will apply its own formatting.

I suspect, however, that the problem has more to do with display than with formatting codes. That’s probably why syntax highlighting is ignorably fast in MacVim/GVim - everything is fixed-width text in the same size and font. Is there some way we could speed up rendering of just what’s being edited currently, or does that go too deep into HTML display libraries?

The text formatting has become much faster in the latest release of the OS X IDE—Andrew must have changed something. I used to have a good few seconds pause every time I opened a comment without M-/; now I hardly notice anything.

It doesn’t much feel like it. Granted, the reformatting runs at a lower priority so I can… kinda… sorta… move… the cursor while it’s reformatting, but I think the same amount of work is going on.