[I6] Arrays as properties?

I came across this old thread [I6: Property arrays] while trying to figure out a way to create a property that is an array.

It’s possible to define a global array and then use that global array as the value of a single property, and it doesn’t seem to matter what type of array (normal, string, table, etc.) is used. However, it doesn’t seem possible to declare an array “in line” while defining object properties. In other words:

Array extern_tab table "abc" "def" "ghi";

Object thing1 "thing 1"
    with    name 'thing' 'one' '1//',
            attached_array extern_tab; ! this is OK

!Object thing2 "thing 2"
!    with    name 'thing' 'two' '2//',
!            attached_array intern_tab table "zyx" "wvu" "tsr"; ! this is NOT OK

It seems like the object property table is certainly capable of storing an array, so two questions:

  1. Is there some way to get this effect that I haven’t come across? (I did examine all 100 mentions of “array” in the current release notes, but I may have missed something.)
  2. If there is no way to get the effect, is the inability to do this just a limitation of the compiler not understanding the syntax, or is there some deeper technical reason to prohibit declaring properties that are arrays?
  1. Not that I know of.

  2. More than anything else, I think it would be a challenge to explain this in a simple enough way to programmers. They can already have property arrays in properties. It is also possible to check if a property contains an array of values or just a single value, and if it’s an array you can check how many elements the array has. If you point programmers at a way to declare a regular array inline for properties, you also have to explain that they get a different kind of array, since:

  • The program can’t know the size of the array unless it’s a table.
  • If it is a table, the method of figuring out the size will be quite different from how it’s done with a property array
  • It will always look like a single value, and you can’t tell if the value is a packed address (the address of a string or routine), or a normal address (typically an array address or a dictionary word address). So, unlike property arrays, the program must expect this special kind of array to be there, or all kinds of crazy stuff can happen. Even if you just want to add a single value for a certain object, you still have to put it in an array.

The way it is now, a knowledgable programmer can put the address of an array in a property, and he/she should understand it will have certain oddities and limitations but it also has its advantages, and it’s on them to avoid the pitfalls and make these arrays work.

I have put regular arrays in properties. I’ve found it useful. I would never suggest having the compiler support creating those arrays inline, or explaining in the Designer’s Manual how to put regular arrays in properties. Beginners would quickly make a mess out of it, and complain that the language/compiler support for this kind of array is broken.

@fredrik, I appreciate the detailed response, so thank you.

I’m curious – would you elaborate on the “oddities and limitations” that are present when an array address is placed in a property? What kinds of pitfalls await the unwary in doing this?

A regular array will apper odd and limited in the context where users have gotten the hang of property arrays and now learn that you can put regular arrays in properties, thus getting rid of that irritating size limit. But you can’t tell if a property holds a routine address, a string address or the address of a regular array. If you know it holds the address of an array, you still can’t check the length of the array, unless it’s a table. If it is a table, you can check the length, but not using the methods you’ve learnt about for property arrays.

Oh, and another great reason not to make them readily available:

  • Users would expect these arrays to work for properties used by the library too. I.e. at least beginners would expect properties like found_in to take a regular array, which would be great for items which appear in 33+ locations. But the library could never do that, since it can’t tell a pointer to a regular array from the packed address of a routine.

@fredrik, OK, thank you for the clarification. (I thought that maybe you were referring to additional things beyond your original list.)

In response, I would argue that:

  1. The program needing to know how to use the contents of a property is not specific to this case. Since Inform 6 is weakly typed (which I hope is the correct technical description), that’s pretty much how things always are. The metaclass() routine can distinguish classes, strings and routines, but it doesn’t distinguish between word literals, character literals, numbers and arrays. If the inability to distinguish an array would be a real problem, perhaps it would be possible to expand metaclass() to cover arrays? (I don’t really know how metaclass() works now, but it seems like it might be at least theoretically possible to alter it to identify an array.)

  2. The point about not being able to know how many entries are in an inline array is a good one (and would be very relevant to class method definitions, for example), but wouldn’t restricting legal declarations to the array subtypes of table, string or buffer take care of this?

  3. It would indeed be different to figure out the size of an attached array, but I don’t think it would be different in a bad way. A normal array placed in a property can just be inspected with obj.array_prop->0 or obj.array_prop-->0 as appropriate to the array subtype (assuming subtypes are restricted as mentioned above) to determine the length. I, personally, think that this might be easier for a beginner to understand than dealing with the special .# and .& operators and any necessary adaptive byte/word conversion arithmetic. (I know it took me a while to make sense of all that, which is why I was excited to discover that an externally-defined array could be attached to a property.)

As for typical array-like properties (like found_in), I see your point, and I’ll think about it some more.

The less obvious case may be additive properties which hold routines. If you create a class which has a before routine and then an object belonging to this class, which also has a before routine, you get a property array with one routine address per entry.

Not sure if you’re suggesting that we stop using property arrays all together and just use regular tables for all properties?

I think the designers of the Z-machine understood perfectly well that they could store a regular array address in a property, but they also saw that this would create problems because an array address can’t be distinguished from a routine address or string address, and so they created property arrays.

@fredrik, no, absolutely not. I’ve just been working through the DM4 exercises, and I’ve found it frustrating that normal arrays can’t be used in an object declaration. I had thought that there was perhaps some deeper technical reason for prohibiting arrays as object properties, and I wanted to know more about it.

I (think I) understand that when an array is actually present in the object table (call them “embedded arrays”), that’s when it becomes necessary to use the .# and .& operators to access its various entries. And I get the impression that there is a hard limit (32?) on the number of possible entries, presumably because that’s a limit to their size in the object table. And I’m aware that there is special processing of embedded arrays when a given obj.embedded_array() is called (which is the basis for additive properties).

But in the example above, the array isn’t present in the object table, only a single pointer to an array elsewhere in memory (call them “attached arrays”). I guess what I’m imagining is that the compiler would accept in-line definitions of attached arrays (of only those subtypes which include an automatic entry at -->0 or ->0 to indicate the total number of entries), and then automatically create the declared array elsewhere in memory, leaving only the pointer to it in the object table. That would mean that the “typical” operators --> and -> could be used on the attached array, that larger arrays could be used as object properties, that classes could include properties that are attached arrays (and instances could provide non-default versions), and that classes which did include attached array properties would be able to know how many entries to expect.

I don’t see any reason why allowing that would affect the function of embedded arrays as they exist now. (But please tell me if I’m wrong.)

You can call it a technical reason or a reason of being hard to teach and would make it easy for programmers to make mistakes. You can’t make these regular arrays work as smoothly with the property system as property arrays do. Example:

If I create an object with a name property which has a single name in it, it’s stored as a single value. If I add a name in the code, it’s stored as a property array instead. And code which examines the name property can find out whether it’s a single value or an array of values.

Let’s say you’re in charge of the Inform 6 library, and now you want to allow more than 32 names in a name property. You can do this with a regular array. Now you have to require all game authors to always declare a regular array for the name property, even if they just want to put a single word in it.

Yes. As far as the Z-machine is concerned, they can be 64 bytes long, and each entry can take up one or two bytes. However, Inform doesn’t really allow you to use it as a byte array, so in Inform it’s a maximum of 32 word entries.

This is correct. It just doesn’t integrate very well into the property system. A property that can take a regular array for a value must always have a regular array for a value, or no value at all.

It’s possible to use regular arrays as property values already. You are free to do so. I’m sure a lot of games have used it.

Making regular arrays look like an integral part of the property system, having the compiler facilitate their usage and documenting them in the Designer’s Manual would be a huge mistake, IMHO. Advanced programmers already understand that they can use them, and they understand that they do so at their own risk. Novice programmer’s may not yet understand that they can use them, and it’s best kept that way, since they’d get themselves into a mess and blame the language and blame the library for having a broken system for properties.

@fredrik, I appreciate your efforts to explain this, but I think there must be some things that I’m not yet seeing. Suppose, for example, I wanted to set up a class of active NPCs that keep track of the last 50 things that they’ve seen – perhaps to allow them to make idle comments about things in the environment. I would want to do something like:

Class ObservantNPC
    with    objects_seen table 50, ! imaginary syntax
            choose_observed_object [ i chosen_obj ;
                i = random(self.objects_seen-->0);
                chosen_obj = self.objects_seen-->i;
                if (chosen_obj == nothing) ... ! handle looking for another choice if entry is empty
                return chosen_obj;
            ],
            make_object_comment [ obj ;
                obj = self.choose_observed_object();
                switch(obj) {
                    sword: "My grandfather had one like that over the fireplace in his small white house.";
                    ... ! assorted default statements here
                }
            ],
            daemon [;
                ... ! examine the NPC object's location and update self.objects_seen accordingly
            ];

with the idea that I can later check (npc_var ofClass ObservantNPC) and conditionally send npc_var.make_object_comment();. (Please forgive any faults in the imaginary logic above.)

This is a situation where (from what I understand now) it would require declaring an external array for every ObservantNPC instance and then manually assigning the appropriate array to each instance in Initialise() or the like. I’m in no way an advanced programmer, so if there’s some elegant shortcut to avoid this requirement, please share it – I would appreciate it.

As you point out, the existing property arrays (what I’m calling “embedded arrays” above) can’t be wholesale replaced, and shouldn’t be. But the idea would be that the objects_seen property is just a single-entry embedded array (what I’m calling an “attached array” above), more like favorite_color or current_age than name, and just like it would exist if the “naked” property is declared without a value (as must be done now if one wants to attach an external array). It would hold the pointer to an compiler-generated external array structurally identical to one declared via:

Array insert_generated_name_here table 50;

That kind of automatic array would only be generated if keywords table, string or buffer followed the property name. A simple list of values would follow current rules, and would be compiled to a property array in the object’s property table.

I do agree that there is room for confusion about the difference between a “normal array” and a “property array”… but I suggest that there would be no net new confusion added by this change, as the distinction already exists and is covered in DM4. I wouldn’t expect a hypothetical beginning programmer to be confused when the property that they explicitly ask to be set up as an attached array does, in fact, function as one!

As one data point: My own personal confusion in precisely the learning scenario that you outline was that the compiler didn’t seem to allow declaration of an attached array under any circumstances. (… which was why I was later pleased to discover a possible workaround, then even later started to wonder about why it had to be worked around in the first place.)

That doesn’t make sense. Then you must also have been shocked that this didn’t work?

Class DoorClass
with_key Object “key”,
door_to …,
door_dir …,
has door;

Anyway, we both agree that it’s sometimes useful to have a property hold a pointer to a regular array. You think it would be a good idea to have the language support inline declarations of regular arrays in properties. I think it would be a bad idea. I can’t see us getting any further than that.

@fredrik, no, there’s no good reason to think that something like that might work. What confused me was that:

  1. It’s possible to declare properties holding numbers, strings or routines in properties at will.
  2. It’s possible to declare any of these inline without the slightest complaint from the compiler.
  3. When declared inline, any of these function identically to how they would if defined as a standalone (with the exception of default return values for routines changing, a useful change).
  4. Declaring any of these inline imposes no special restraints on their normal functional range.
  5. Any of these can be accessed with the same . (dot) operator.
  6. The value of any of these can be replaced in a property via simple assignment at any point at runtime.

The treatment of arrays is quite different, and the reasons for this inconsistency are not explained in DM4. In fact, DM4 starts the discussion of property arrays with:

Properties can be arrays instead of variables. If two or more consecutive values are given for the same property, it becomes an array.

which, as covered above, is not really true – it becomes a special object table property array with its own rules and features. In early experimentation, I attempted to assign an external array to replace the values within a declared array, as in:

Array ext_tab table 4 5 6;

Object thing
    with    name 'thing',
            favorite_numbers 1 2 3;

...

thing.favorite_numbers = ext_tab; ! expect (thing.&favorite_numbers-->1 == 4) is true now, etc.

This is accepted by the compiler (6.31) but doesn’t work as might be naively expected given the way that the same pattern would function for numbers, strings, or routines. The reasons why it doesn’t work didn’t become clear until I started to understand several things never touched on by DM4. Perhaps I (and other hypothetical beginning learners) can be be forgiven for not having deduced the structure of the object table and the nature of the compiler’s handling of property arrays after having read only to page 58 of DM4.

Fair enough. We do understand each other, and I appreciate your engagement on the topic, which helped to refine this idea of how Inform 6 could be improved.