You can separate TADS 3 and adv3 easily enough; just don’t include the adv3 library in your project. But what you then have is a programming language without a world model, not something you can write Interactive Fiction it at all (unless you’re prepared to build the entire world model, parser and all, from scratch, but that’s not a challenge most IF authors want to take on).
The adv3 library doesn’t make it easy to achieve a clean split between “core” classes and “add-ons”, because everything is so tightly integrated and coupled together. One of the aims of the adv3Lite library is to make precisely this split easier to achieve. I don’t know if you’ve looked at the adv3Lite manual but it’s divided in precisely this way, with a section on “core” classes followed by one on the optional extras. The adv3Lite conversation system, which can get quite complex, is effectively one of those optional extras; if you don’t want it, you can simply exclude the entire file in which it resides (and, if you wish, replace it with your own implementation).
I did initially think in terms of an “adv3min” which might be closer to what you had in mind, i.e. an absolutely minimalistic library with a totally barebones world model, but I quickly came to the conclusion that it would be too minimalistic to be of any practical use in writing Interactive Fiction. I suspect then that we wouldn’t be in agreement on what needs to be included in the “core” library and what constitutes an optional extra; there is in any case no objectively correct “right” answer to that, it’s simply a judgement call on which people will inevitably disagree. My concern is that if you leave too much out of the core you risk actually making it harder for people by forcing them to reinvent commonly-used wheels. To you “Taking away the complexity without removing the sophistication” may look like a recipe for failure, but from my perspective it’s a necessary compromise.
Experience suggests that a novice pretty soon will need doors and connectors (a door, after all, is just a special kind of connector); these are not easy thing for most novices to craft unaided. As I said, if you don’t want a conversation system you can easily ditch actor.t and you don’t need to use it (it’s much easier to get rid of than it is in adv3, since adv3Lite is a lot more modular). The “core” library of adv3Lite doesn’t really contain any “exotic special cases” that I can see (though perhaps your notion of what constitutes an “exotic special case” in Interactive Fiction differs from mine).
One might perhaps question some of your assumptions, such as that novices in general will only require a rudimentary conversation system, and that they’ll always be ready to reconsider if they really need x, y or z. Maybe it would be better if they were prepared to reconsider that, but many novices seem to me to be a bit more ambitious than you imply, and they don’t always give up that easily on their more ambitious ideas. Again, a rudimentary conversation system makes it hard to produce anything but a rudimentary conversation. Maybe that’s okay if that’s all your game needs, but if a novice wants a to write game in which interaction with NPCs plays a significant role, a rudimentary conversation system may very quickly become extremely limiting, and devising a more sophisticated one from scratch may be a bit of a tall order for a novice. Of course you could argue that the absolute novice would be well advised to stay away from something as difficult as conversation in the first place, and there may be some wisdom in that, but at some point along the line the more ambition not-quite-such-a-novice will often want to tackle more than rudimentary conversation, and if the library s/he’s using doesn’t provide for it, s/he may be a bit stuck.
To put it more briefly, adv3Lite is intended to be scalable, so that the complete novice can indeed start with the kind of core basics you envisage, but having invested time and effort in learning the basics of adv3Lite can go on using to to a higher degree of sophistication without having to move to adv3 (or some other system) straight away. If adv3Lite didn’t possess that potential, it would simply be a throwaway learning tool, not a serious contender as an IF library at all.