Glk: Image scaling in buffer windows

(Following on from this thread: Glk extension proposal: images in buffer windows )

The draft text for the Glk spec update:


extern glui32 glk_image_draw_scaled_ext(winid_t win, glui32 image,
    glsi32 val1, glsi32 val2, glui32 width, glui32 height,
    glui32 imagerule, glui32 maxwidth);

Another version of glk_image_draw(), with more options for image scaling.

The imagerule option encodes a width rule and a height rule. (You must supply one of each when calling this function.)

Width rules:

  • imagerule_WidthOrig: Use the image’s standard width. The width argument is ignored.
  • imagerule_WidthFixed: Use the width given in the width argument (as an integer).
  • imagerule_WidthRatio: The image width will be proportional to the window width. The proportion is given by the width argument, which in this case is a fixed-point fraction. $10000 (1.0) means that the image width will be 100% of the window width. $8000 (0.5) means 50% of the window width, and so on.

Height rules:

  • imagerule_HeightOrig: Use the image’s standard height. The height argument is ignored.
  • imagerule_HeightFixed: Use the height given in the height argument (as an integer).
  • imagerule_AspectRatio: The image height will be a fixed aspect ratio compared to the width. (The width is as defined above.) The height argument, in this case, is a fixed-point fraction which is multiplied by the image’s standard aspect ratio. $10000 (1.0) means that the image will always retain its original aspect ratio. $20000 (2.0) means that it will be stretched vertically by a factor of 2.

[Remember that imagerule_WidthRatio defines the image width relative to the window width. imagerule_AspectRatio defines the image height relative to the image width.]

The maxwidth argument, if nonzero, applies an additional upper bound based on the window width. It is treated as fixed-point fraction of the window width. If the image is wider than this, it is reduced proportionally.

[Thus if you use imagerule_WidthFixed, width=600, maxwidth=$10000, then the image will appear with a width of 600 or the window width, whichever is smaller. If you use imagerule_WidthOrig, maxwidth=$8000, then the image will appear with its original width or half the window width, whichever is smaller.]

[You can combine imagerule_AspectRatio with the above options. The interpreter always figures out the width first, then the height (again, imagerule_AspectRatio only controls height). Then it applies maxwidth, which may cause a proportional reduction (regardless of how height was determined).]

[Note that applying both imagerule_WidthRatio and maxwidth is pointless. The wider constraint will be ignored.]

[If the window is resized, what happens? This depends on the window type. See below.]

[glk_image_draw_scaled_ext() can do everything the previous two functions can do. glk_image_draw() is equivalent to imagerule_WidthOrig|imagerule_HeightOrig, maxwidth=$10000. glk_image_draw_scaled() is equivalent to imagerule_WidthFixed|imagerule_HeightFixed with the given size, maxwidth=$10000.]

…In a text buffer window, imagerule_WidthRatio is dynamically computed; the image width will always be relative to the current window width. If the text buffer window is resized (by the user or a window arrangement call), the image will resize too.

…The imagerule_WidthRatio option does not dynamically resize in a graphics window. The image size is computed when glk_image_draw_scaled_ext() is called, and then the image is painted to the canvas.

Also, the docs for glk_image_draw() and glk_image_draw_scaled() now have this note:

If the image is wider than the window it is drawn in, it will be proportionally reduced to fit with window width. This is a behavior change in Glk 0.7.6.

1 Like

I have a test implementation in Quixe here:

https://eblong.com/zarf/tmp/quixe-draw-ext/play.html

You can try various image options at the parser prompt. Don’t worry too much about the 1/1x1/1 syntax; that’s just an I6 grammar token I whipped up to test my code. What you care about is the equivalent I6 function call, listed above the test command.

(The 1, 0 arguments in every call give the imagealign_InlineUp alignment. You can ignore that.)


Draw image original size, limit to window width:
glk_image_draw(win, image, 1, 0);

> image 0 worig horig maxwidth

Scaled to 400x300, limit to window width:
glk_image_draw_scaled(win, image, 1, 0, 400, 300);

> image 0 wfix hfix 400x300 maxwidth

Scaled to 800x300, limit to window width:
glk_image_draw_scaled(win, image, 1, 0, 800, 300);

> image 0 wfix hfix 800x300 maxwidth

Scaled to 800x300, do not limit to window width:
glk_image_draw_scaled_ext(win, image, 1, 0, 800, 300, imagerule_WidthFixed|imagerule_HeightFixed, 0);

> image 0 wfix hfix 800x300

Full window width, normal aspect ratio:
glk_image_draw_scaled_ext(win, image, 1, 0, $10000, $10000, imagerule_WidthRatio|imagerule_AspectRatio, 0);

> image 0 wratio haspect 1/1x1/1

50% window width, normal aspect ratio:
glk_image_draw_scaled_ext(win, image, 1, 0, $08000, $10000, imagerule_WidthRatio|imagerule_AspectRatio, 0);

> image 0 wratio haspect 1/2x1/1

Full window width, fixed height of 100 pixels:
glk_image_draw_scaled_ext(win, image, 1, 0, $10000, 100, imagerule_WidthRatio|imagerule_HeightFixed, 0);

> image 0 wratio hfix 1/1x100

2 Likes

I see that one of the other threads has moved on to HTML imagesets. (Providing multiple images to support different display resolutions.) That feature is not part of this proposal; there’s no way to do it in pure Glk or Glulx. For the time being, use Vorple or Bisquixe for that sort of thing.

1 Like

That said, the discussion of imagesets is mostly because interpreters didn’t previously scale images to fit the window, which this fixes.

This all looks great to me!

1 Like

Oh, the imagetest I6 source code is here:

This all looks great! Just a few small things.

What happens if you set maxwidth=0 or even bigger than 100%? Can we encourage interpreters to clip it rather than adding horizontal scrollbars?

I think a note is needed that if you use imagerule_HeightFixed then the height is not adjusted by maxwidth, which is for the use case of fleurons/borders. Only imagerule_HeightOrig and imagerule_AspectRatio will be adjusted for the maxwidth.

Because there’s a default maxwidth, using imagerule_HeightFixed would actually result in the aspect ratio not being preserved. glk_image_draw_scaled will need to use the provided dimensions to calculate an aspect ratio. This appears to be what your test implementation does.

I don’t see this new function ever being used in graphics windows. If you wanted to size it relative to the window width that’s easy enough to do already, and unless you’re just drawing the image flush to the left you’d need to calculate the width yourself in order to center it. (Most of the time this would be a library doing it for the author, so it can handle all the calculations automatically.) IMO I think the new function should just be for buffer windows.

Again, I don’t think this makes sense for graphics windows as you might be deliberately drawing an image bigger than the window.

1 Like

If the width goes to zero, the image does not appear. (And it should not take any vertical space; my current implementation is missing that bit.)

If it’s wider than the window, it should clip (horizontally_. GlkOte has always done this but it’s worth adding to the behavior specification.

I think a note is needed that if you use imagerule_HeightFixed then the height is not adjusted by maxwidth , which is for the use case of fleurons/borders.

In my current implementation the height is adjusted by maxwidth. This is a little bit fussy, but it seems to fall naturally out of the implementation.

I expect fleurons to use imagerule_WidthRatio rather than maxwidth, for a horizontal-bar sort of divider. The sort of divider that looks like three stars (or flowers) should scale down proportionally; stars don’t look good squished. So I think my rule is justified.

(By the way, I never saw the term “fleuron” before this discussion.)

I don’t see this new function ever being used in graphics windows.

I agree that there’s no benefit, but for consistency’s sake I’d prefer to support it.

Again, I don’t think this makes sense for graphics windows as you might be deliberately drawing an image bigger than the window.

Now that is a good point. The maxwidth concept is a bad idea for graphics windows. I’ll drop it.

Hm. How about I drop the maxwidth argument entirely, and use val2 for the maxwidth for buffer windows? (val2 is currently unused for buffer windows.) That would segment the functionality nicely.

The only problem there is all the existing code which passes val2=0! I’d have to allow 0 to mean 100%, and -1 to mean “no limit”…

Pulling in comments from the other thread:

I think that while for the Glk API it’s probably simplest for the aspect ratio to be a multiplier of the natural aspect ratio, for the GlkOte protocol it should be the desired end ratio, which can be plugged straight into CSS’s aspect-ratio , which I think can just be a double

I came to the same conclusion, except that I decided to pass aspectwidth:W, aspectheight:H in the GlkOte protocol. That can be plugged straight into CSS as "W/H" (with a literal slash).

If you do add a max-width option, then from my tests it’s actually crucial to not set a CSS height, you have to use aspect-ratio instead.

I agree, and that’s what I did.

1 Like

I meant if you set the maxwidth parameter to 0, and the image’s original width is wider than the screen, in which case it should be clipped.

Quite a few posts in the previous discussion refered to them, including one you replied to :stuck_out_tongue:.

It depends on what type of image you’re using. If you’re using one like this you’d want the height to scale:

But one like this you might be okay to lose its aspect ratio:

There could also be an image which has no horizontal details, like a rainbow or a barcode, which you would want stretched.

I had been thinking of a situation in which you might want to have a border image with a original width of 600px but then apply a 80% max-width onto it, keeping the height the same in both cases. But this might be overkill. Not having special rules for imagerule_HeightFixed would certainly be simpler.

@Draconis You were one of the people originally pushing for fleuron-style support. Is a percentage of the screen width with fixed height good enough?

Hmm, that could be a good addition. Would it then be useable in the original functions too?

Going back to these:

The first one will need to be converted to an aspect ratio for the final CSS (with a default maxwidth=100%).

The second one is the fixed height image, maxwidth is not used. This is the only one that would result in a height actually being set.

The third one is still not needed (it could just be sent in the first format), but would now be width:W, aspectwidth:AW, aspectheight:AH.

The fourth would now be widthratio:R, aspectwidth:AW, aspectheight:AH.

Will aspectwidth and aspectheight be integers or floats? imagerule_AspectRatio might result in a non-integer ratio, so floats are probably needed.

We’ll need a new protocol support flag.

Full screen width with fixed height works for me!

1 Like

Yes. Although by far the most common case is that AspectRatio is 1 ($10000), so aspectwidth and aspectheight can be the image’s original width and height.

I will update the glkote doc with all of this soon.

We’ll need a new protocol support flag.

Let’s call it 'graphicsext'.

1 Like