Woof. I can tell you I ran at a similar problem last year, only to conclude I was not up to the challenge. Or, said differently, was more work than justifiable for my needs. Here are two threads that discuss adjacent aspects of this problem: @jbg’s legitimate deep dive, and my initial design musings that convinced me the elephant was too large.
I’m not sure how many of these threads’ concerns are applicable to your situation. In my case, I reverted to just creating and instantiating potentially large numbers of instances. So far, their specialized use has not translated to detectable performance issues, even when hundreds are in existence (not least of which because I hide them in an inventory container). I spent a LOT of time confirming that trash collection cleaned them up when disposed of.
I guess what I’m saying is good luck! With all these big brains on the problem, it’ll crack EVENTUALLY!