I actually find it incredibly aggravating that videogames include music and or audio cues for combat, generally. I will always mute both the OST and any environmental noises (footsteps, weapons, weather, etc) if it’s an option, as I am typically listening to a documentary or video essay on X2 speed in the background.
I only begrudgingly turn on audio for IF in particular when the author explicitly tells us it’s a crucial part of the experience, and even then I typically mute it after deciding I don’t want to listen to squeaky doors or narration. I am also generally listening to my own, unrelated music, and it’s irritating to have to turn off my nightcore temporarily to disable a game’s tunes. It’s especially annoying in IF, where I expect a purely textual experience typically. For visual novels there’s a little more leeway, but I have to be in a particular mood to play those.