C programmers, Git interpreter on Android, Twisty - glkop

So git compiles fine on the Twisty 2.0 code base, I even updated it to latest version of git interpreter. I’m trying to figure out where the previous developers left off and see what we can do to finish off the features of this interpreter.

First big problem is saw is that loading Counterfeit Monkey gblorg completely crashes the C code in a fatal error that Java can not trap/catch :wink: So, I am starting with that. I traced the fatal crash down to git/glkop.c line 405 github.com/BroadcastGames/twist … kop.c#L405

The value of funcnum is 97 that case the crash, hex value 0x0061. At line 405 I added if (funcnum == 97) skip Phase 3 and now Counterfeit Monkey no longer crashes the Android App. So, where do we go from here, how do I figure out how to properly implement glkop 0x0061 ? It looks like gi_dispa.c has these, and 0x0061 == fileref_create_by_name? So Counterfeit Monkey is trying to write a file?

As an aside, is there any way we can make this code more robust at the C level so that it doesn’t abend the entire Android App when it encounters a problem like this? Can we implement some try/catch here so that we can have a recoverable error and not a fatal Linux signal?

This going from Java --> C —> Java gives you headaches. But I think I found the general issue. This glkop 0x0061 seems to eventually land back up in Java code TwistyGlk.java github.com/BroadcastGames/twist … k.java#L89 line 89 method namedFile.

Which returns a null, which I guess the developers hadn’t completed this work - and it is supposed to return a file of the given name. I tried adding “return new file” (on a writable part of the sdcard), but that now causes a new crash inside the C code.

If the library is properly implemented, it will not crash in glkop.c. It may call a fatal_error function somewhere, but it shouldn’t crash. So trying to catch the error is irrelevant – you need to fix the bug.

(The glkop.c file is, or should be, the same as in the master Git repository at github.com/DavidKinder/Git .)

I’m not familiar with this library (or with Android) so I can’t say what the bug is.

I’m back to looking at both apps together, Son of Hunkypunk vs. Twisty. On Twisty I was able to upgrade the git from 1.3.2 to 1.3.4 without any negative issues. But Twisty has an older git interpreter built on 1.2.8… and now I think I found exactly where the crashes are happening, and now reviewing the Version History I see why:

1.3.1 2012-11-09  Further fixes to glkop.c, following similar fixes added to
                  Glulxe 0.5.0.

1.3.0 2011-12-16  Fixed a bug in glkop.c dispatching, to do with arrays
                  of opaque objects, following a similar fix in Glulxe.
                  Fixed a problem with the memory heap not being sorted
                  correctly on restore, contributed by Brady Garvin.

1.2.9 2011-08-28  Fixed a bug in glkop.c dispatching, to do with optional
                  array arguments, following a similar fix in Glulxe.
                  Glk array and string operations are now checked for memory
                  overflows (though not for ROM writing), following a similar
                  fix in Glulxe.

so, I need to adopt Son of Hunkypunk’s glkop.c interface to account for these problems. Because I am having crashes right in the C code for arrays! C programmer help appreciated :wink:

Upgrading to Git 1.3.4 is almost certainly going to be the best move.

I’m stabbing in the dark here, because I really don’t know much about C nor about Glk to know why exactly Git 1.2.8 works and Git 1.3.4 crashes. I’ve narrowed it down to some C array operation form incoming data that is passed down from java. I do have 1.3.4 working with a different Glk Java library over on the Twisty side; I’m painfully trying to find the relevant code (it has to do with taking line input from the user GUI).

I managed to identify the crash starts with in Git Interpreter version 1.3.1, this commit: github.com/DavidKinder/Git/comm … ed65ea9fb0 So, now I have to figure out what interfaces changed that is causing Son of HunkyPunk to crash.

Ok, seems after a lot of wasted time, I find that git_mac.c is the home of void fatalError (const char * s) - not really an ideal place for a Linux/Android build :wink: The code is just doing an exit(1), which is a big no-no on Android - because the message won’t even be shown. So, the code isn’t crashing, it’s deliberately ending due to a data structure change :wink:

I’m still horribly stuck on this. Son of Hunkeypunk and Twisty have very different approaches to the GLk interfacing to Android. The c code is structured very differently (Twisty uses a lot more static methods on the C side and is more structured in general, but only works with two interpreters).

I know the problem has to do with c array / memory management problem. To recap

  1. Git 1.3.0 works fine
  2. Git 1.3.4 changes the array structures and will not work with Son of Hunkypunk when it gets INPUT from the user (“go west” in a story).
  3. Twisty works fine with Git 1.3.4 and is a possible Android Java app reference, but the code is pretty different in it’s approach to these particular C to Java interfaces.

All of the breaking code is in this one checkin, which is the upgrade from Git 1.3.0 to 1.3.4 and some extra logging added: github.com/BroadcastGames/son_o … d9eb0b9133

The run-time error is on this line 1414, here is a link to the specific line of the runtime error and I made a comment about the output values. github.com/BroadcastGames/son_o … 7746eR1414

Git 1.3.0 to 1.3.4 changed the macros used to work with the arrays. I just don’t understand c programming enough to see why the change of macros is failing. Attempts to graft in old macros from version 1.3.0 also has proven a failure for me. Overall, the probelm isnt’ really with Git (it works fine in Twisty) but clearly Git is giving us a clue as to what is wrong.

So the reason for the crash is that array->len != len (256 vs 0)? Sounds like a byte is overflowing; since len is passed in as a uint32, it would have happened somewhere before. Are you able to get a stack trace?

The display library (SoHP in this case) has to register and unregister arrays when it’s doing line input, using the functions set during the gidispatch_set_retained_registry() call. It has to do this right or everything breaks. This is not super-well-documented, I’m afraid.

I’d love to say “you can just copy and paste stuff over” but, well, that’s obviously not true.

My guess here is that SoHP is allocating an array of chars (8-bit) when it should be allocating an array of integers (32-bit), and therefore there’s a memory overflow error. Or something like that. I wish I could be more useful but I just don’t know SoHP/Twisty/Android.

I’ve started over from scratch a couple times, learning as I go :wink: Twisty allocates and calls gidispatch_set_retained_registry() before invoking Java for the RequestLineEvent - but Son of Hunkypunk calls Java first, then back to C gidispatch_set_retained_registry(), then resumes back in Java. There are C macros and pointers that make all this hard to follow, but I think I’m down to the few lines of code constructing the problem array.

ok, some real progress. Testing the change, but it looks like a Java long variable type passed to C code long was turning 0 into 1 value. There is a type jlong that is supposed to be used. Git 1.3.4 was checking this value and Git 1.3.0 was not.

ok, this highlights the disaster of calling Java to C with a long parameter on an ARMv7 processor :wink:

JNIEXPORT int Java_org_andglkmod_glk_Window_retainVmArrayTest(JNIEnv *env, jobject this, int buffer, jlong length, long lengthTwo)
    LOGE("WTFBBQ? Java_org_andglkmod_glk_Window_retainVmArrayTest method length %d lengthTwo %d / alt %lu %lu / alt2 %llu %llu", length, lengthTwo, length, lengthTwo, length, lengthTwo);
    return 0;

note that lengthTwo param is a jlong not like length parameter. Here is the runtime output when you call it with the the same long variable sent down into both length and lengthTwo parameters:

WTFBBQ? Java_org_andglkmod_glk_Window_retainVmArrayTest method length 256 lengthTwo 256 / alt 0 256 / alt2 256 12067396232331395328

So, problem comes down to a single character jlong vs long :wink: I opened Issue #36 on Github for Son of HunkyPunk to describe the problem: github.com/retrobits/son_of_hunkypunk/issues/36