MOD tracker formats in Glulx

Technical Development Interpreters
1

A recent suggestion on the uservoice forums was (paraphrased) “support more modern MOD formats”. Fair enough. If you go looking for what audio formats Glulx supports, you get referred to the Glk spec, which refers to the Blorb spec, which I wrote in 1998. The MOD section says:

“For generality, MOD files in Blorb are limited to the ProTracker 2.0 format: 31 note samples, up to 128 note patterns. The magic number at byte 1080 should be ‘M.K.’ or ‘M!K!’.”

…which was conservative in 1998. It’s a little more than conservative now.

David Kinder pointed out that the Windows interpreters use libModPlug, which is open-source (public domain, more or less) and supports this list of formats (as of libmodplug-0.8.8.1):

“MOD, S3M, XM, IT, 669, AMF (both of them), AMS, DBM, DMF, DSM, FAR, MDL, MED, MTM, OKT, PTM, STM, ULT, UMX, MT2, PSM”

I have no idea what that means. But it seems reasonable to change the spec and suggest that interpreters use libModPlug, so that game authors can rely on whatever it supports. David, Ben Cressey, and Andrew Hunter are on board with this (for WinGlulxe, Gargoyle, and Zoom).

This does add some of uncertainty for authors, because I’m not going to mandate a particular version of the library. So if you come back in a year and try to use a format that was added in libmodplug-0.9 (or whatever), you may find that not all interpreters support it. But hopefully this will give us a good base of reliable formats, and it’ll certainly be better than what’s documented now.

Further problems: David says the stock libModPlug is buggy. He has a patched version in WinGlulxe. We recommend that everybody use that, which is no problem on Mac/Windows interpreters. However, it might be a hassle for Linux packagers. Ben is looking at how it will work.

Overall, I’d like to do this. Does anyone else have any comments? Other interpreter (and Glk library) supporters?

2

The only formats that matter today are IT and MO3. And maybe XM. The rest are historical remains from the Commodore Amiga days :stuck_out_tongue:

There’s also libmikmod. It’s LGPL.

3

Well, if there are freely available MODs in the older format, it makes sense to support them if we can; authors might find a song from 25 years ago that fits their story perfectly.

libmikmod can only play one file at once and must load that file from disk. It’s unmaintained and hasn’t been updated in over six years. It’s non-reentrant and crashes if you invoke it more than once, or if you link to more than one library that links to and initializes it.

I say bring on libmodplug. :slight_smile:

The stock libmodplug seems to work OK on Mac. It looks like several of the fixes that have come in over the last few years pertain to 64 bit or endian issues, which aren’t a factor on Windows but would be needed for Linux / Snow Leopard x64 and PPC Macs. These would take some effort to analyze and backport, assuming they aren’t simply the result of poor coding in the post-fork patches.

I haven’t yet tried building garglk with libmodplug support under OS X 10.4, then running that binary on 10.6. libmikmod fails spectacularly under those conditions, and I am optimistic that libmodplug will manage it better.

4

To specify a library like this is probably a good idea if you want to support those formats. That way the end result can be expected to sound the same in different interpreters. Playback of these formats is inconsistent and varies between implementations.

(My experience with tracker formats comes from maintaining an engine for graphical adventure games, and using first IT then MO3 in games of mine. I replaced most music with Ogg Vorbis in the latest version of my latest game, though. Partly to avoid some technical issues, but mostly to get better sound quality.)

However with the possible exception of MO3 (which unsurprisingly isn’t on the list - I think only the BASS library supports that format), for most things not retro rendering the music to Ogg Vorbis will actually make a smaller file size. So maybe you’re mostly adding bloat? But since MOD already is specified as supported, why not. Any terp already has to have tracker playback.

5

Thanks for the experiential comments.

The original querent said that Ogg files would be larger (for the music he wanted to write). I haven’t measured, but I’m sure there’s a tradeoff for length – Ogg size is roughly proportional to length, where MOD size is proportional to the number of different instruments you use. Allowing very long pieces without bloat seems good.

6

I’m still developing QGlk, but slowly, and sound is a long way down the list of things to do.

However, modplug looks OK to me. I’d like something that works on MinGW, which it looks like it does. (And Windows Gargoyle is MinGW anyway, isn’t it?)

I’d say it’s desirable for authors to convert the songs to newer formats, if possible. I don’t know a lot about trackers but I think you would usually be able to do that.

But you can leave it up to the authors to decide - if they use obscure formats they have less chance of their games working in the year 2050.

7

FastTracker 2 XM all the way!

Before there was popular pattern-based music software like Fruity Loops Studio and professional soft-/hardware like Native Instruments Maschine I usually composed all my electronic music with FastTracker 2 in the XM (Extended Module) format, as an alternative to MIDI format. I have over 1000 XM songs in my personal archive which I composed between 1992 and 2002. And hundreds of thousands of modules made by other people. I still use MOD trackers on occasion nowadays. Just like the Interactive Fiction community there exists an entire MOD tracker community on the net, the so-called tracker scene, which exists even to this day. The advantage of tracker formats over MIDI is that all music data is exported and stored in one file, not only notes and patterns, but also the entire instrument and WAV sample bank used by a song. Therefore some tracker files can be huge in size, e.g. 5-10 megabytes.

The most popular MOD tracker formats in the scene, sorted by ranking:

  1. ImpulseTracker (*.IT)
  2. FastTracker 2 (*.XM)
  3. ScreamTracker (*.S3M)
  4. Standard Module (*.MOD)

Other tracker formats are usually rare.

P.S. Also keep in mind that classic first-person shooters like good old Unreal, Unreal Tournament '99 and Deus Ex all used music composed in one or all of those four tracker formats listed above, thanks to the Unreal Engine. They were usually stored in Unreal’s own music container format (*.UMX). Using MOD tracker formats is very retro and old-fashioned, but it’s still cool. :sunglasses:

8

For playback of modules I strongly recommend ModPlug Player (MPP), as an alternative to WinAmp. It can play most tracker formats in high quality and even save/export them as WAV, so you can convert it to MP3 later on.

A screenshot from my desktop showing MPP in action, playing DOOM music converted from MIDI to XM by me:

9

Hi, since most Glulx/Glk interpreter combinations are written in C, this won’t matter to most people - and probably only to ZMPP (Zag/Zing ?), which is simply using MuXM through JavaSound to playback MOD files. It seems that MuXM supports ProTracker (standard MOD), FastTracker and ScreamTracker.

I still own a couple of working Amigas (and would unlikely ever part with them), but I never got into SoundTracker (I always used TFMX on the Miggy) to know much about the format.

Wei-ju

10

Gargoyle uses MinGW, yes. I had a fair bit of trouble with the stock libmodplug code today, though. There are some unofficial patches floating around to “unbreak” mingw support, but that doesn’t exactly inspire confidence.

I plan to try David Kinder’s fork from the Windows Glk source tomorrow, and I’ll report back on how that goes.

11

Are you planning for any kind of capability testing other than just trying to play the file in glk_schannel_play() and having it fail? It would probably be nice to be able to play a regular MOD if some funky format is unsupported.

And don’t write off Javascript… with the new audio data APIs someone will probably write a tracker implementation soon enough.

12

I am not currently thinking of changing any APIs. Having the play command fail seems good enough.

13

An ancilliary problem that I should have mentioned: chunks in Blorb files have no filenames. Can libModPlug work out the right thing to do with its data when there’s no filename suffix attached?

14

While I have not used libModPlug, I can’t imagine that would be a problem. (It’s not a problem for any of the libraries I have used: FMOD, BASS and DUMB.)

15

It doesn’t even have an interface to look at the filename: all the interface takes is a buffer to the input data, from which it figures out the format. The Windows Glk implementation of sound doesn’t ever write out the sound data to a temporary file.

16

Okay, that’s good.

17

Here’s what I’ve written:

MOD is an Amiga-originated format for music synthesized from note samples. Over the years, other formats of this type – generally called “tracker” or “module music” formats – have arisen. Blorb supports four: original “.MOD” files, ImpulseTracker (".IT"), FastTracker 2 Extended (".XM"), and ScreamTracker 3 (".S3M").

Because tracker-playing libraries typically handle many formats, it is most practical for Blorb to lump them all together. Regardless of which tracker format is used, the chunk type will be 'MOD '.

The formats are described here:

digitalpreservation.gov/form … 0126.shtml

This spec does not attempt to distinguish variations within the four supported formats. (".MOD" is particularly ill-defined, although I have saved comments on the original MOD format at eblong.com/zarf/blorb/mod-spec.txt.) Instead, we recommend that C implementations embed libmodplug, a public-domain tracker-playing library. Its home page is:

modplug-xmms.sourceforge.net/

However, some bug fixes are included in the version packaged with Windows Glk.

(Note that it may be safer to compile libmodplug with the MODPLUG_BASIC_SUPPORT option, which eliminates many obscure tracker formats that Blorb does not support.)

Where libmodplug is not practical, implementations should use whatever tracker-playing library claims to support the four formats in question. We trust, perhaps beyond reason, that implementation differences will not lead game creators to their doom.

1 Like
18

Everyone seems to be happy with that, so I’ve posted it to eblong.com/zarf/blorb/ .