A rising tide is lifting all boats at the moment. I visit this site regularly and I think I detect an effort in every traditional IF framework to support multimedia.
I congratulate everyone who is motivated to work in this area and wish you every success!
My own IF framework, Balladeer is explicitly a web-native framework, and so benefits from all the standards-based implementation which you’d find in a modern browser.
Nevertheless, I find I’m treating each form of media differently in terms of how they are handled in the framework.
My central paradigm is around the concept of Speech Acts which are expressed as Prologue, Dialogue and Epilogue, all captured in a markup system called SpeechMark. This markup allows:
- Declaring metadata
- Raising procedural events
- Assigning CSS classes
- Applying themes and styles
- Naming media assets such as audio files
So I find myself anticipating the manner in which these multimedia would be used. In my case I’m gravitating towards the following:
- All layout determined by CSS grid, hence class assignment can affect position on the screen.
- Applying contextual images (icons and backgrounds) is a matter of styling and hence also implemented in CSS.
- In contrast, I find that Audio files align with the Speech aspect and so are a structural component of the page (although optional because of how browsers control playback).
- Video assets treated likewise, but more likely to appear in interstitial titles than alongside story text.
I know that the philosophy of each IF framework is different. Here’s the question: When it becomes possible to offer support for rich text or multimedia, how would you say it’s best used in the context of the tradition of your platform?