Updating extensions in the GitHub repo for Inform7 v10

Technical Development Inform Extensions
1

The “new Inform 7” has a few incompatible syntaxes vs. 6M62 and before. Perhaps most notably, Include (- ... -) instead of "section" in "file.i6t" needs to become Include (- ... -) replacing "function". Is there an agreed-upon procedure for making updates of that nature to the i7/extensions repo?

Like, should we wait with updates until the new version is formally released later this month? Would it perhaps be a good idea to keep a “backup”/archive of the current, mostly-6M62-compatible versions of all extensions around somewhere?

(Edit: just noticed that this is currently in the process of being discussed in a GitHub issue: Repository reorganization for multiple Inform versions · Issue #80 · i7/extensions (github.com))

2

it’s too late here for me to start now, but I was going to start yammering about this tomorrow myself…

3

Right, I’ll make a pull request based on the result of that discussion next week. If someone wants to update the extensions before that, they can make a pull request to wait for merging after the repository is reorganized.

4

I don’t think we should just label the existing extensions as being for 6M62. About 30% of them don’t even compile in 6M62. Some were for 6G60 and never updated. Some never worked. Article Bug Fix by Daniel Stelzer compiles in 6M62 but is irrelevant 'cause it addresses bugs that are fixed by 6M62. Remembering by Aaron Reed compiles but doesn’t work.

Now, of course I know that by design “some are ready for public use, others are barely working experiments.” But I think it’s a disservice to the community for us to go on like that without even an attempt to categorize them. I spent a lot of time and effort in the last year learning what did and didn’t work and why. I don’t want to see other people wasting their time trying to get something to work and then, if they’re lucky, doing a forum search here and finding a comment from 2017 saying “oh, that hasn’t been touched since 2012 and hasn’t worked since 2014.”

This is especially true 'cause we’re probably going to get a lot of newcomers giving I7 a spin when the new IDEs are released. And it’s easy for a newcomer to walk away forever if one early experience is bad and I would hate for our community to lose those voices.

What I would beg and plead for: however we end up organizing I7-version-specific extensions, let’s create the expectation that a given extension’s presence in a version-specific location stands as an assertion that that extension is expected to actually work in that version. There will doubtless be mistakes along the way, but I’d really like to see that be what we aspire to. (Doing this will be a lot more work. I’m willing to do a lot of this work, but for personal reasons, the next two months will be very busy for me; unfortunately I might not be able to do as much as I’d like in as timely a fashion as I’d like.)

Something I have yet to investigate is the use of the 6M62 compiler one can build with v10 with make retrospective. For newcomers, “compile in 6M62” will mean selecting an option in the IDE, not using a whole different application. Without testing, I’m not sure we can expect that 6M62 as built with v10 handles the exact same things as 6M62-classic, so that’s another potential twist here.

Dr. Nelson’s 2020 talk said “where I want to go with this is that extension authors will post their extensions as GitHub repositories with semantic versioning tags, and inbuild will be able to fetch from them. At that point it will be a full-on package manager, the need for which I talked about at Narrascope I. We’re not there yet, but the infrastructure is coming into place.” That was two years ago; we don’t know how fleshed out that thinking may be now. So one frustrating thing is that whatever we do now will probably prove to be not what we will eventually wish we had done if we’d had foreknowledge.

Another twist is that I just yesterday reported I7-2093: if you build the Project Index while compiling a project (i.e., you don’t specify -no-index), it’s currently the case that if inform7 can’t parse any extension (or stray file) under External/Extensions, compilation will fail. I figure this is likely to be fixed, but at the moment, extensions that can’t be parsed are a big problem.

Anyway, right now I’d suggest we end up with 3 main subdirectories under i7/extensions:
6M62, v10, Unspecified (or whatever name). Unspecified gets everything; the others get things believed to work with those versions.

3 Likes
5

We didn’t bother marking out versions originally because it wasn’t intended to be the primary place you’d obtain extensions. But now, for a lot of extensions, it has become the de facto prime repository. So more structure of some kind is needed.

I liked the idea of branches, but maybe making version folders in one branch would be better. Probably we don’t need an Unspecified folder, instead we can just have folders for the older branches once we can determine what version the extensions do actually work in. The date of the files will also help with that.

The disadvantage is that we may then have to copy files when they work in multiple versions. A true package repository that could store the versions would be better. If anyone wants to revive i7el, please go ahead :wink:

6

There are things that never worked in any version; we’ll want somewhere for them to go.

7

One issue with moving everything into new folders is that all the links will break.

8

If we want to keep the existing links working, we can define the top-level to be the unspecified directory, and add a comment to the extensions there – I figure at the top, first thing after the rubric(s) – explaining that this may or not work, look to the version-specific directories for extensions believed to work with that version.

9

Maybe we should come up with some system for extensions to mark within themselves which releases they support, and then we could automatically propagate version folders/branches (could even do it as a Github action). In order to be backwards compatible it would need to be a comment. Could be something like this:

[Supported releases: 6L38, 6M62]

We don’t normally touch old extensions but I’d be fine with adding in comments like these to all the extensions, without updating their /date version numbers.

This would let us keep the master branch as the main development branch (and without breaking any links). But we should also be able to make it so that you could commit updates to the 6M62 etc branch too: first update the master branch comment to exclude 6M62, which will stop it from copying from the master branch into the 6M62 branch and overwriting any 6M62 specific changes. Then you can commit directly to the 6M62 branch.

1 Like
10

That sounds good, but it seems to me like there’s a complication with the new Inform.

With semantic version numbering, the major version number changing flags the introduction of backwards-incompatible public API changes and the absence of a change to the major number should indicate the absence of such changes. But 1) that’s aspirational and it’s likely at some point something will change some aspect of the public API without having been accompanied by a major version number change and 2) historically, lots and lots of extensions have extensively made use of things outside the public API like, say, much of your ouevre, @Dannii .

For extensions made of pure Inform 7, it’ll probably be adequate to say categorically “compatible with v10” and for that to remain true. But for all of the many weirder extensions, it becomes likely that some given extension will be compatible with, say >= 10.1.3. Or, eventually, >= 10.1.3 and < 10.3. Or even compatible with a discontinuous range, because 10.2.7 introduced a bug that broke this extension, but it was fixed in 10.2.8.

So I’m not sure how to flag releases supported with new Inform going forward. Maybe just define our level of granularity as counting major version and minor version. There’s a v10.1 branch for all of 10.1.*, then eventually a v10.2 branch. If there are more details regarding compatibility or incompatibility with particular patch levels, they should be indicated in comments at the top, but these comments are for humans and are ignored by the notional automation tools that we’re mooting.

An alternative could be a scheme for indicating version compatibility in the semantic version number’s build metadata.

And, of course, in practice how much this matters will be tied to how often new patch levels or minor versions will be declared, which we plain don’t know now.

11

Hmm. Well I think there’s plans (I don’t know if it’s supported yet) for the semver number of extensions to be part of the file name. We could easily do that when we copy the extension files over into the release branches. So then Xorshift-v1.0.i7x would exist alongside Xorshift-v2.0.i7x. For pre-10.1 preleases, just keep one version, without it in the file name?

12

There’ll be no problem with Xorshift-v1.0.i7x existing alongside Xorshift-v2.0.i7x. I’m talking about the complications surrounding the possibility that Xorshift-v1.0.i7x is compatible with Inform 7 versions >= 10.1.3 but < 10.3 and Xorshift-v2.0.i7x is compatible with 10.3+. How are we indicate anything like that alongside [Supported releases: 6L38, 6M62] ?

13

Ah. Well the way I was thinking of it would be that the automation would just look at the master branch. So when you release Xorshift v2.0, you’d update the supported releases tag to only say 10.3. If you also wanted to release an update to v1.0, then you could manually update it in the 10.1 branch.

14

Okay, I’ve committed a simple script which does as I proposed above. What do you think? We can add more complexity for v10.1 semver stuff later on. If we like this then I can set up a Github action to run the processing on each commit to master.

15

Script looks good at first glimpse.

And perhaps we should define something that would declare what an extension that’s known to be unfinished or broken is intended to support just to provide info to humans, not the script, e.g.,

[ Targeted release: 6M62 ]

1 Like
16

One suggestion I have for the script would be to allow both “types” of versioning for older releases (previous releases were retroactively numbered in a mostly semver manner). So “9.3” should be read the same as “6M62”, and so on.

17

This is exciting. I am currently working on other things but I may come back to Inform now that it’s open-source.

Without looking, I’m frankly expecting Gender Options to be a PITA to update, since to do its work it essentially rips the guts out of a hunk of Inform 6 core-library code, including not merely functions, but static data structures. I’m not sure when I’ll get a chance to take a look at it. It’ll probably need a new version for v10. All of my extensions were 6M62-designed and 6M62-tested, FWIW, and all of them hack into Inform 6 core-library code.

Personally for my workflow, since all my extensions are hacking into the core library and changing it, I’d prefer a scheme where I can indicate that a particular published version of the extension is tested with one or more particular versions of Inform (and may work with later versions), while simultaneously having another published version of the extension intended for a different version of Inform. I’m not sure what the best way to do that is. It is unlikely that it will be possible to write a version of any of my extensions which works with multiple versions of Inform; they’re too finicky.

On the other hand, given what my extensions are doing, arguably I should just fork the main Inform release. Hmm.

18

I have an extension in the repository that was sort of experimental, and unfinished, and which I do not plan to finish. It might compile, but I do not recommend it for general use. I’d rather it not be there at all. Is there any way for me to remove it, or at least, if you re-organize things, will there be a place to put it where it will be out of the way?

19

So, it’s possible we’re both overthinking this and underusing the abilities of Git.

Perhaps the thing to do is to keep the repository folder structure more-or-less as-is but create a new series of BRANCHES. It’s possible to have a branch for each version. It’s possible to selectively pull changes from one branch over to another, selectively, though it’s a little messy. I’ve done it though, on other projects. “git cherry-pick” is the key to doing it cleanly.

The current “master” branch should probably be abandoned and renamed for political reasons anyway. So it could become the base of the “old-experimental” branch (containing everything, including stuff which never worked). Meanwhile live branches for “v10”, “6M62”, and “6G60” could be maintained – with the recommendation to newbies, in the README.md file, to be “check out the v10 branch”.

This would also allow for more specific v10.1 branches if that became necessary.

This corresponds with Graham Nelson’s idea of using git tags, but tags aren’t gonna cut it since updates to the v10.1 version may be necessary, so branches are the way to go.

In git, branches are cheap and porting stuff from one branch to another is relatively cheap. So I think this is likely the best way to go for maintaining different versions of the Extensions repo for different versions of Inform.

After thinking about it, this is what I recommend. First, update the README.md to explain the new branch structure. Then make “old-experimental”, “v10”, “6M62”, “6G10”, and “v10-upgrade-work” branches forked from the current master branch. Initially all the branches will be populated with everything. If Zed is willing, since he’s done the work to figure out which works where, then he can “git rm” the stuff which doesn’t work in 6M62 from the “6M62” branch, the stuff which doesn’t work in 6G10 from the “6G10” branch, the stuff which doesn’t work in v10 from the “v10” branch.

Set the “v10” branch to be the default one shown on the Github front page.

Then we can work on upgrading old stuff to v10 on the “v10-upgrade-work” branch, and when they’re ready (one at a time) add the revised versions of the extensions back to the “v10” branch.

Or we could just do the upgrade work on a single “experimental” branch, now that I think about it, and not have separate “old-experimental” and “v10-upgrade-work” branches. I kind of prefer separating “old-experimental” from “v10-upgrade-work” though

1 Like
20

We definitely could just use branches like that. But a lot of people use the repo who aren’t git experts. My thought was that it would be easier for someone who’s never used git to just update their extensions in one place, and then we can use some automated scripts to update the other branches. And it also means that if you go to the master branch to get your extensions they won’t be out of date.