I don’t think there is an easy answer to this. I use a lot of different techniques to come up with puzzles. Once I have a basic storyline and map, I start to populate it with potential NPCs and objects and find that things fall into place. Imagine that I have a stable. What would I find in a stable? Maybe a horse, horse manure, horse shoes, bridle, hay. Think like MacGyver. How can I use these objects either here or somewhere else?
I like to use ordinary objects in unusual ways. I use a lot of chained puzzles where you need to do A in order to do B in order to do C. Mind you, you won’t know what the sequence is until you’ve discovered all the relevant objects and tried a few things and received some hints along the way. How do I make things more difficult? Hide things? Make them visible, but inaccessible?
I also use a lot of branching so that you can do various things in a different order. I rarely use a puzzle dependency chart. Perhaps because of this, I sometimes have trouble working out the optimum sequence to solve my own puzzles. But that’s a good sign, right?
When working out the puzzle mechanics, I use little charts and sketches and truth tables and lists and goodness knows what else. I also write little transcripts. All these methods lead to different ideas and “what if” scenarios.
I keep a folder of “adventure ideas”. Some of these are storylines, some are ideas for puzzles, some are mini walkthroughs. I draw on lots of resources, such as puzzles in books or newspapers. However, you have to be careful of puzzles for puzzles’ sake. If I have an idea for a puzzle that I want to use, I make sure that it is integrated into the story and fits into the environment.
I also do a lot of research. This often leads to puzzle ideas. For example, when was the safety match invented? If it didn’t exist in the pirate era, then you’re going to have to use a different means of lighting fires. This leads to new objects and new puzzle ideas.
I never thought of my puzzles as being “traditional puzzles”, but I know what you mean. I think most of my games have at least one idea that has never been used before, but puzzley parser-based games will always have a lot of things that aren’t entirely new. There’s nothing wrong with that, as it gives the player a comfortable feeling of familiarity.
If you’re short of ideas, put it aside for a while and something will spring to mind when you least expect it. Or work on a different part of the game.
I’ve never had any trouble with scope creep. On the contrary. I get to a point where I can’t think of what else to add. That’s when I know it’s done, at least from the puzzles point of view.