Using Glk through an FFI?

Technical Development
1

I’m thinking about using Glk for rendering, in hopes of adding some familiarity and street cred to something I’m working on. My reference implementation is written in Lua, and I’m trying to figure out how I’d use Glk through LuaJIT’s FFI.

GLK provides a main function, and it calls a user-provided glk_main which doesn’t terminate until the player quits. LuaJIT also provides a main function, so I’m not sure how to handle this. With something like SDL, which doesn’t provide a main or call any long-running functions, it’s straightforward… compile SDL as a DLL or shared object, write FFI bindings for the functions I want to call, and call them from my game (in fact it’s even simpler than that; can grab a pre-compiled DLL from the web, and the SO should be available in repo, so I don’t have to build it at all).

As far as I can tell, what needs to happen with Glk is something like this:

  • Rename LuaJIT’s main to something else (let’s say lj_entrypoint), and compile LuaJIT as a static library.

  • Write a small glk_main that just calls lj_entrypoint with argc/argv hardwired to load, say, main.lua.

  • Build a Glk implementation along with my glk_main, linked against my altered-source LuaJIT, as an executable.

  • The real game loop is in main.lua; call the glk_* functions from there through the FFI.

Somehow this doesn’t feel right. Ideally I’d like to build the Glk implementation as a library, link it dynamically at run time, and leave LuaJIT alone (creating an altered-source version of the Glk implementation instead), but I can’t quite wrap my head around it.

Anyone have advice on using Glk through an FFI?

2

Defining glk_main is more of a convention than a strict requirement. It works well when you’re statically compiling a C program with a Glk library. It provides a consistent API (that’s not part of Glk proper) for parsing CLI arguments and loading files; but it’s not essential. If you can set up FFI bindings to an existing shared library then you are probably good to go.

3

Hmm, thanks. Thinking about it more I suppose I could rename Glk’s main to something like glk_init, strip out the calls to glk_main and glk_exit, and then in my Lua script, call glk_init, do the game loop, and (maybe) call glk_exit.

I’ve only looked at cheapglk so far, not sure if this will work across the board (or at all; if it’s really that easy I’m not sure why Glk provides a main in the first place). I’ll try that first, though.

4

@zarf, maybe a bit late to the game here, but what do you think about adding a glk_init and glk_quit to the spec?

I’m thinking that main would simply call glk_init, glk_main, and glk_exit (which calls glk_quit), and that’s it. That way, when compiled as a library, we’re not missing any setup routines in main, and we can have control over the order of teardown routines when quitting.

  • main
    • glk_init
      • Handle command line args
      • Call SDL_Init / initscr / etc.
    • glk_main
    • glk_exit
      • glk_quit
        • Call SDL_Quit / endwin / etc.
      • exit

Why glk_init? SDL-based nanoglk calls SDL_Init from main, curses-based glkterm calls initscr from main, etc. If these kinds of things were done from glk_init, they could be reused in the library. It would have the same function signature as main.

Why glk_quit? It would work exactly like glk_exit, except it would not actually call exit. Teardown routines like endwin or SDL_Quit would live here. This way, library cleanup can be done when something else needs to be done afterwards before the program terminates. It would be called from glk_exit.

None of this should impact the existing API, since the new functions would be called from the old functions. If implementations conform to this, users should (ideally) be able to swap out pre-compiled libraries and have everything “just work,” as long as the libraries support all game features.

5

So you want to formalize the use of Glk as a library in some other program? Instead of, as it really currently is, an application which can be compiled with a portable VM/game core. (The VM is the part which is a “library” in the standard sense.)

You can do that but I don’t think it needs to be in the spec.

In your proposal, glk_init and glk_exit would be implemented by the Glk library, whereas glk_main is implemented by the VM. This isn’t consistent and it doesn’t really add anything to the spec. That is: if you’re implementing a Glk library, you can write your main() this way, but it doesn’t add any interoperability because (e.g.) the glulxe VM code doesn’t call glk_init. On the flip side, it doesn’t go against the spec either. Except for some confusion in naming the functions glk_init and glk_quit.

6

So you want to formalize the use of Glk as a library in some other program? Instead of, as it really currently is, an application which can be compiled with a portable VM/game core.

Yes, exactly. Well, sort of; I don’t care if you say “this is a library now,” but it would be nice to facilitate its use as a library (without hampering its use as an application). I assume this would just give the VM more choice about how to use Glk; the library approach and the application approach should both be viable. Either implement glk_main and compile everything together as an application, or compile Glk implementation as a library (or grab DLL from web) and call glk_init at the top of your own main and glk_quit at the bottom.

You can do that but I don’t think it needs to be in the spec.

It doesn’t need to be, but putting it in the spec would allow for interchangeability among libraries. For example, Windows-Glk distributes a DLL and exposes an InitGlk, and if I want to commit to that Glk implementation, I can call that. But then I need a different code path for a different implementation, can’t support unknown future implementations, and the user can’t simply swap out the DLL for something else, because InitGlk only exists in that library. If all libraries had the same interface for initialization, we could write implementation-agnostic Glk code.

I’m a little confused about last paragraph.

In your proposal, glk_init and glk_exit would be implemented by the Glk library,

I’m thinking glk_init and glk_quit would be stuff that’s already part of Glk implementations, it’s just that it would be moved out of main and glk_exit, so it can be accessible (without terminating program) when Glk implementations are compiled as libraries.

whereas glk_main is implemented by the VM.

Right, isn’t it that way already?

This isn’t consistent and it doesn’t really add anything to the spec. That is: if you’re implementing a Glk library, you can write your main() this way, but it doesn’t add any interoperability because (e.g.) the glulxe VM code doesn’t call glk_init.

Something like glulxe wouldn’t need to call glk_init, because main calls glk_init, and it calls glk_main which would be implemented by glulxe, right?

So, it doesn’t add anything to that scenario, it stays the same… the interoperability benefits are all for the case where it’s used as a library.

On the flip side, it doesn’t go against the spec either. Except for some confusion in naming the functions glk_init and glk_quit.

Right, if it’s not in spec I’d treat glk_* as reserved. But the point of getting it in spec would be so that glk_init works in any Glk implementation conforming to that version of spec, so that clients of Glk libraries can be implementation-agnostic.

I feel like either the suggestion wasn’t as clear as it should have been, or I’m missing something about how it works, or misreading your reply somehow, but thanks for taking a look!

Alright, reading over this again, I think I see what you’re getting at – supporting the current approach and the library approach at the same time is inconsistent. What I’m suggesting is that client code could use one approach or the other, but not both. I’m not sure consistency is an issue here if client code is picking one approach or the other instead of mixing them somehow. In the Glk implementations, it would just be a minor change, making things that are already there a little bit more granular, mostly so that nothing exists only in main.

The library approach is useful for cases where Glk is being combined with something else that also defines main, or in cases when dynamic linking is preferred. The current approach is already in use, and it’s convenient for static builds, so I don’t want to suggest replacing it, but there’s no reason it can’t coexist with the library approach (Windows-Glk is already doing this, as far as I can tell).

7

I don’t mean to drag this out, but could you summarize what your library approach is for your specific use case? I haven’t used Lua or FFI. (I assume FFI is “foreign file interface”? The Lua docs don’t say.)

The narrow-and-ugly change that seems to do what you want is “tweak the Glk implementation’s code to rename main() to notreallymain(); then Lua calls that.” Is that on target? If so, we can consider tidier ways of achieving that. If not, I’m still confused.

8

I don’t mean to drag this out, but could you summarize what your library approach is for your specific use case? I haven’t used Lua or FFI. (I assume FFI is “foreign file interface”? The Lua docs don’t say.)

FFI is a foreign function interface… it allows calling C functions from some other language. A lot of interpreted or JIT-ed languages (LuaJIT, Python, Racket, etc.) have an FFI.

Here’s some info on LuaJIT’s FFI. You probably won’t need to look further than “motivating example” to see what’s going on. Basically we give it something similar to a C header, then we can call whatever functions we want from either a statically or dynamically linked library.

Usually we’d drop in something like sdl.dll or curses.dll, adapt a copy of the C headers to work for us, and start calling functions. I’ve done some little games in LuaJIT with SDL, and it usually looks like calling SDL_Init, calling some more SDL functions to create a window and rendering context, then a game loop that’s mostly non-SDL stuff except drawing things and input handling, then after the game loop, an SDL_Quit.

Basically I’d like to do something similar with Glk. I’d like to build it as a library, not as part of something else, then link it dynamically and call the glk_* functions in a similar way to how I’d use SDL, curses, or other C libraries (typically there’s a setup and teardown function, and a bunch of other functions you’d call in between for IO or other stuff).

The narrow-and-ugly change that seems to do what you want is “tweak the Glk implementation’s code to rename main() to notreallymain(); then Lua calls that.” Is that on target?

Not quite, unfortunately. One thing is that notreallymain wants to call glk_main which doesn’t exist unless we create it. If we create a glk_main that does nothing, notreallymain will terminate the program with glk_exit after calling it. And if we create a a long-running glk_main, it’ll block everything else.

What would need to happen, as far as I can tell, is that we never call main from our code. Instead, we call some setup function, then we have our game loop, then we call a teardown function – exactly like what main is doing, but without ever calling it. The problem here is that the setup code doesn’t exist in a predictable place – it could be in the body of main, or in another function with different names and signatures across different implementations. A teardown function (glk_exit) exists, but it also causes the program to terminate, which could be a (smaller) problem.

None of this is necessarily specific to FFIs, it would be useful for anyone who prefers dynamic linking – it’s just that it’s common to dynamically link things like this when using an FFI. Glk would be a great candidate for allowing user to swap out the DLL/SO to get different behaviors, so dynamic linking could be really useful – imagine a GTK-based Linux distro symlinking gtk-glk.so as glk.so, a QT-based distro symlinking qt-glk.so, and so on, which the user can override with something like update-alternatives. Then they all ship the same glulxe, which doesn’t care which Glk it uses.

9

Glk is designed in the spirit of a virtual machine, in which platform-independent game engines can live. Since application startup (including asking for a story file) is platform-dependent, it is abstracted in Glk.

Meanwhile, your program is running inside another virtual machine (Lua), which is also designed to host platform-independent code, and where application startup is abstracted in a different way.

You’d like to make use of some of the abstraction mechanisms of Glk (presumably the I/O functionality for text presentation), but not all. But you also need some of the abstraction mechanisms of Lua (the script engine). There’s no easy way to extricate either feature from their context. You argue that it could be done for Glk, but doing it that way is against the spirit of the Glk abstraction, and might be incompatible with non-unix-like systems, e.g. those that employ cooperative multitasking. Likewise, one could argue that it should be possible to extricate the script engine from Lua, and embed that into a regular Glk application. I’m actually surprised that Lua isn’t designed for that scenorio in the first place.

I don’t know if this is a viable option, but could you make a client-server setup? One Lua application calling out via a socket API to a generic Glk application that provides a text presentation layer?

10

Glk is designed in the spirit of a virtual machine, in which platform-independent game engines can live. Since application startup (including asking for a story file) is platform-dependent, it is abstracted in Glk.

Sure, but is there a drawback to doing the application startup stuff in a dedicated function instead of an arbitrary one, or just in the body of main? Exactly like GTK’s gtk_init(&argc,&argv), or QT’s QApplication myApp(argc, argv), or kinda like SDL_Init, or curses initscr, etc. This is pretty much the usual way of doing things, isn’t it?

You argue that it could be done for Glk, but doing it that way is against the spirit of the Glk abstraction, and might be incompatible with non-unix-like systems, e.g. those that employ cooperative multitasking.

All I’m proposing is making Glk slightly more granular, by specifying where the initialization code goes, and by making a version of glk_exit that doesn’t terminate the program. Glk could still be used in exactly the same way it’s being used now, unless I’m missing something. This would just add another way to use it – instead of client code relying on Glk’s main and implementing glk_main, we could call glk_init and glk_quit – or we can do it the current way and totally ignore those new functions.

The only thing that would change is that Glk’s main would call the new glk_init instead of putting init code in the body of main, or in another arbitrary function, and the teardown code would move from glk_exit into glk_quit, which glk_exit would call before terminating the program. As far as I can tell it would be a very minor organizational change in every implementation I’ve looked at; I’d be willing to submit patches myself.

The proposal should work just as well with static linking, if someone happens to prefer that combination, so cooperative multitasking should be no problem either way.

I’m actually surprised that Lua isn’t designed for that scenorio in the first place.

It could work the way I described in the OP… actually LuaJIT’s main is very minimal, so we could just write a glk_main with a copy of the body of LuaJIT’s main. This doesn’t change the fact that building a separate Glk+LuaJIT for every flavor of Glk that users might want is a pain.

Ultimately this isn’t about Lua though, it’s about wanting the option to dynamically link Glk. Which does stem from the fact that dynamic linking is preferred in this scenario, but it’s also preferred in other scenarios like software repos (not just by me, but by Debian packaging guidelines, for example), or when we just want to use a binary distribution and not have to build unfamiliar code.

I don’t know if this is a viable option, but could you make a client-server setup? One Lua application calling out via a socket API to a generic Glk application that provides a text presentation layer?

It would be possible, but sounds more convoluted than just having a dedicated setup and teardown function, allowing Glk to be used like many C libraries. But maybe less convoluted than building a separate Glk+LuaJIT for every Glk implementation out there.

11

But you already can dynamically link Glk. That’s how Windows-Glk and Garglk work.

12

Exactly, which goes to show it can work well! The only problem is that that init function isn’t in the spec, so we have to know that it’s InitGlk for Windows-Glk, whatever Garglk uses, and so on, which breaks implementation-agnostic approach and forward-compat.

And of course implementations like nanoglk that put the init stuff in the body of main, there’s no hope of dynamic linking without modifying source.

1 Like
13

Okay, the way you were writing I wasn’t sure you were aware.

Yes having a more standard way of initialising Glk would be good so that you don’t need to have a bunch of preprocessor code for each Glk lib. All that really takes is to ask the Glk implementation authors to agree on a name. I’m not sure of the merits of your proposed glk_quit though.

On the other hand, you don’t need to do that before working on your Lua project.

14

glk_quit is definitely less important… probably. After all there’s atexit in a pinch. But this would be more in line with most other C libraries that don’t terminate the program in the teardown function.

And you’re right, I’ll probably try some Glk flavors as-is for now (although it would be much quicker to evaluate the different implementations if I could just drop in a DLL :wink: )

1 Like