Open Source!

Right. What I’m trying to do is figure out something that we might be able to accomplish without trying to put pressure on Graham to speed up or change the development of core Inform, because I think the team has made it pretty clear that Graham works in the way he works, and is aware of the bugs in the database, and that repeating ourselves louder about those bugs isn’t going to accomplish anything. This isn’t meant as a criticism of Graham or of those of us who would like more attention to be paid to the issues, but it’s one of those things where he has whatever demands on his time that he has, Emily certain has demands on her time and as she has said is spending time on a lot of IF community support that isn’t I7, and while we would all like to pitch in and help, it would be very demanding on Graham’s time to put things in a situation where we can. As zarf said, he wants to get the code rewritten, cleaned up, and documented–and that’s a ton of work.

(Aside: Some bugs seem to have workarounds or fixes that we can figure out ourselves, particularly in the open parts of Inform like the Standard Rules or I6 templates. Here’s one. We could make a repository for those, or an extension or series of extensions that fixes them, and if it were posted in a prominent place on the site it could be helpful. And then there are some bugs in the open part of Inform, like this, where we haven’t figured out a workaround, because the parser is really hard to figure out and/or the people with the wherewithal to fix it haven’t put a high priority on it. Which I think points to some of the difficulties in going open source–it’d take a lot of work to get the code into a state where lots of people could just up and solve a bug like this.)

Yes, and the IDE is on github. Part of the issue is that the documentation has gotten out of sync with the IDE here (it still talks about the Skein and Transcript panels), plus it’s not clear exactly how the functionality for releasing along with a solution should work with the new IDE. This is one of the more irritating problems for people–but it seems to me that much of it could be solved simply by adequately documenting the new Testing panel, as long as there were someone with the authorization and time to do it. So there’s at least something that we could help with here if we volunteered to take it on, and the team let us.

I’m not a pro here, but it seems to me that most of the currently bugs in core Inform aren’t really show-stoppers. (Except for the ParseTree bug which as I understand it is causing a lot of trouble for foreign-language Inform–that’s important, of course–and the Flexible Survival people sometimes run into trouble with their massive project busting the compiler, but last I checked they had a special patched version that let them keep working.) The opacity of how they’re being handled is a problem, but again I think some of this frustration could be resolved without burdening the core team by having people post some bug-fix extensions somewhere on the main site. Which, again, would require us getting the keys to the main site. But not that much more than that.

And a lot a lot of the frustration I see here on the forum and elsewhere has to do with the difficulty of finding proper versions of extensions, and figuring out which ones are up to date. (I don’t think I know whether Disambiguation Control is up to date, and that’s one I helped update myself!) That really seems like something that could be solved on the website without further troubling the core Inform people. And if there could be an extension team (again, seems natural for the IFTF to supervise) that could do things like help keep track of what’s maintained instead of relying on individual extension authors to come back out of the woodwork every time there’s a new version, that would be even better. There was a thread on here for checking extension compatibility with 6L I think, but if some group had a clear institutionalized role to do this I think it’d help keep that going.

I feel a bit brass-faced saying this, because it’s essentially “Why don’t you all take on this work?”–I can’t really volunteer for this myself (I know nothing about websites, and while I probably would pitch in on some extension maintenance I don’t feel like I can take on any more of an institutional role over and above what I’m doing with this forum). But, well, it’d be nice to do.

bg, thanks! To be clear, I don’t know what Graham thinks at all, and I’m not really clear as to who’s doing what on the team. And part of the issue is that if we just put something on uservoice it’s not clear how fast it would get handled. One thing I think is that before anyone asks for the keys to the website there ought to be a team in place that’s prepared to take the responsibility, and maybe has some plans for what to do with it. Which would require those people to put in some work on spec, I’m afraid. But I guess someone would have to take the first step.

I’d like to add that my plea was not to pressure Graham or demand anything or raise pitchforks.

I’m pleading the case that open source development offers tremendous benefits and if done properly, Graham could still maintain his vision and yet have the benefit of more eyes and hands on the nuts and bots of the code.

Good points overall. One thing I want to point out.

But showstoppers is only part of the problem. Long-lasting bugs lead to technical debt. People end up writing extensions to deal with bugs rather than just dealing with the bug. Or people write extensions that are perhaps unknowingly relying on a bug. People end up promoting habits of “workarounds” that now become so enshrined that when/if the bug is fixed, lots of things break and existing habits become harder to change. Documentation starts getting written around bugs and then the documentation drifts when/if the bug is fixed. Sometimes new design choices are made but now those are undergirded by bugs and then you have to disentangle new feature from existing bugs, and determining which is outpacing the other.

Again, take this as me being biased due to QA and Testing career. So I know I’m micro-focusing on something that might not seem as important.

That being said, even this wouldn’t necessarily have to argue for a full open source release. For example, there could be a private repo where only core committers can make bug fixes. If Inform 7 has a good test suite, regressions can be tested for with each build. One challenge would be making sure that bugs that introduce design changes go through a review process, where Graham would have to be involved. But at least then his involvement would be more focused.

So a controlled and limited open source could be a happy medium here.

Regarding design choices versus bugs, I now finally understand what’s being discussed with the Skein/Transcript/Testing issue. That is a perfect example of intransparency by feature accretion. It looks like the Mac version and the Windows version of the IDE completely diverge in terms of the implementation. Which one is considered the “right” design? We shouldn’t document the Testing approach if that’s not the way it’s going to go. I presume someone did that work for a reason. But someone equally clearly didn’t do that work on the other platforms. So … which is it?

This is also where a pull request system can be helpful. That doesn’t have to be open source, of course, but the more eyes on it, the more likely someone is to question why a completely different implementation is being put in place.

I think the good thing is we’re all looking at alternatives here, from full open source, to partial open source, to business-as-usual-but-with-additions, etc.

That’s easy. The 6M62 changelog says:

As noted above, the IDEs are open source, so github.com/erkyrath/Inform. (I’m not the maintainer but I’ll do builds.)

However:

This is obvious, but maybe not obvious to all readers: you’re pleading the case to literally nobody. Everyone participating in this thread wants to see I7 go open-source.

I’m only pleading to Graham, hoping he respects my opinion, and considers open source a viable choice in his works. The user voice thing is to see if anyone else wishes to make the same plea.

But I will concur the discussion here is pointless.

But reading that – “some of these developments” and “are likely to appear” does not imply this is the "right’ way, though. Or at least there are multiple “right” ways. Even further, perhaps to Matt’s point, if this was the case of the “right” approach even if only on one platform, the documentation for the Mac version wasn’t updated to reflect this change at all. So, at the very least, mixed messaging which maybe further leads to some of the what might have been reflected in Emily’s post.

That said, I agree this is orthogonal to the discussion of open sourcing Inform 7 since the IDEs are open sourced. Although it’s not clear to me what work on the IDE would possibly necessitate changes to Inform 7 and vice versa.

Well, it has surfaced some interesting discussion and I think this kind of thing is healthy. But I agree that we should all set our expectations accordingly in terms of any of us being able to influence decisions one way or the other.

I don’t think it’s pointless at all. What I think should be done next is contact him via email/phone/etc… so that this can be pinned down definitively. A lot of people really want to know what’s going on and we can’t do that if Graham decides to not be part of the conversation.

Fully expect that only a very very tiny minority can actually contact him directly. At any rate I think it’s imperative at this rate.

I used to have direct access to Graham. It’s been awhile since that was true. This is one of the reasons Emily’s blog post unnerves me. She put herself in between the community and Graham, which had huge benefits to the development of I7 and its entire platform. By removing herself, the community is left without any liaison to Graham outside of the rather formal issue and bug tracking systems. This has probably been in effect for awhile and I guess I’m just now recognizing how much of an impact this is having on I7 as a platform (I7 as a stand-alone application is fine).

The platform as a whole could use more persistent guidance. That’s the part that open source can help with the most.

I started working on another extensions site, that would have search, and track multiple versions of extensions. The plan is for it to be integrated into the IDEs, but I haven’t heard from Graham or the IDE developers or the extension librarian in quite a while. (And so I haven’t worked on it myself in quite a while.) But maybe the community should just use and publicise it itself.

i7el.herokuapp.com/

If anyone else is a node developer I’d appreciate other people working on it!

Moving forward on that (the new extensions site) would be directly valuable to the I7 community, even without IDE integration.

I know it sounds like “we have three extensions sites now, let’s solve this problem by creating a fourth.” But if it is complete (for users of both old and new I7 releases), then it would be a great improvement over the current situation.

(on preview, zarf has said a bunch of this more concisely, as usual)

Hey Dannii,
This was actually something that I had in mind as a model for what we could do–or maybe that someone on the team was working on. (I’m not quite sure who’s on the team.) I think this would be a great thing for us to keep going on with, but maybe it would be better if it didn’t have to be coordinated with Graham and the IDE developers.

In particular, I think that rather than try to integrate it with the IDE, it would be good to have it work as a landing page, so someone who visited it would be able to do something like: Find an extension hosted there; quickly tell which extensions were updated for which releases; and get links to some of the other places to find extensions, possibly including the official/old site and the repositories. It’d definitely be great to have something that’s easier for the end-user than Github.

It’d also be nice if this could somehow have a role in coordinating work on updating extensions–in particular ones like Flexible Windows and Disambiguation Control which are pretty widely popular and which break with every update–but that might be too much to ask.

In fact, now that I think of it, maybe one approach to what I was talking about earlier would be, instead of asking for the keys to the official site, to make an unofficial fan site sort of thing with a guide to the official site. This could do things like give links to the best parts of the official site (in particular: warn people against the legacy extension site), host the updated-for-Sierra code, host extensions, provide some of the bugfixes that we can have, etc. Insofar as I see some of the real problems for newbies as coming from the website not staying up to date, we could provide a friendlier place to point them to. (And by “we” I, er, mean you. I don’t even know what a node developer is, really.)

I hear what you’re saying here, but I guess this isn’t my experience of what happens with the bugs when I7 doesn’t get updated for a while. It seems like the newest versions of I7 usually address all the bugs–at least Graham aspires to clear them all–so the debt gets wiped clean every cycle. But also, some critical extensions (thinking of Flexible Windows and Disambiguation Control again) get broken with the updates, because they depend on low-level code that gets changed with the updates. Which at least for Disambiguation Control I guess comes down to technical debt from the parser code being so finicky–but that seems like something that’ll never be discharged.

I’m not sure if there’s a moral I think I can draw from that.

This is excellent, although the typeface seems rather large to me (nitpick), but overall it looks excellent. I’d also consider using a Lucene engine and the search bar enabling auto-type searches (show result as the user types).

Is it ready for extension authors to add their extensions? If so, how would we do that?

I can’t remember exactly how far I got towards allowing everyone to add their own extensions - I think that isn’t enabled yet.

But what I can do is add people to the list of “editors”, who are able to add updates to everyone’s extensions. I don’t want to add everyone to this list of course, but bg, matt w, David are all people I’d be very happy to add. If you PM me a preferred Google account (does not need to be a gmail address) then I’ll add you. (In the future adding other OAuth 2.0 options would be possible.)

It needs the built in extensions from 6M62 added too. And ideally then all the extensions from the other sites.

David, the site is using Bootstrap CSS because I’m not a great designer, so that’s where the default font size comes from. I think it’s okay, but would be very happy for a skilled designer to give the site a facelift. Lucene could be good too. It’s using PostgreSQL currently, but it could be migrated to another system. It also has very limited AJAX, just for autotype results for tags - I stuck with a simple web Express site because I was learning as I was going. If anyone else wanted to commit to working on the site and wanted to transform it into a React or Angular or Vue site I’d be very happy. Or indeed to switch to something other than Node! But as with most things in the IF world, people with the time and expertise to commit are the limiting factor. I probably can’t commit much time in the near future towards it.

Perhaps as part of the Public Library in the IDE (or the website?) there could be a link to an interface for “Experimental User Extensions” with a warning that these extensions are not formally tested and are updated frequently, and to use at the author’s own risk?

(With no shade to extension authors as these extensions usually work as intended, but might sometimes be case-specific, temporary updates, or someone’s personal kludges which are not completely documented… :slight_smile: )

I think this is a great idea–or we could have a link to the place that they are hosted (Github or whatever). And I definitely think that this is something that we should plan as part of the website Dannii’s working on–this definitely seems to be a case where it’s better to get out ahead of the official IDE/website team rather than try to coordinate with them. Maybe if we make something really nice they’ll want to make it official somehow.

Also, I think we should probably start a new thread on this, as it’s got pretty far afield from the original post. (This probably is mostly my fault.)

I’ve started a new thread here: New Inform 7 extensions site

Thank you!

I put my votes in today. As I’d like to have the Inform 7 compiler on Android.