Hello,
Giving experience based on objectives' results is indeed a good idea.
I think we can easily pull out a few different situations:
The player is freely exploring and facing non quest-related foes ;
The player is facing quest-related foes during its quest ;
The player is facing quest-related foes after the quest or at the point from which the foes are considered as "passed through", i.e the player as gained its experience for passing them somehow.
In all three possibilities, the player can kill the mobs.
In the first case, removing the experience gain seems irrelevant. If a weak and peaceful character can't kill an ogre, surely will he be able to get some experience thank to bargaining or by some other ways which will be closed to a full warrior.
In the second case, killing the foe is taking a solution to a problem. It should gives the same experience to slaughter everyone and to sneak through them.
However, the problems come here. What if he kills half the mob of a pack, backs in safety and then sneak through same? Or if he convinces them to let him through, kill half the mobs for some reasons and then sneak through the now hostile foes? Same experience in both ways?
Should a "perfect" solution gives more experience than a flawed one?
I think yes, it should all give the same experience, just so you can focus on the results and struggle your way as you wish.
In the third case, to me, its just a mechanic abuse if you can get some experience for killing NPCs you choose to spare a few seconds ago.
Now, here is an idea I wish to submit:
What if you determine some "quests' areas", closed and independent sub-areas inside a quest. Note that those specials areas are NOT meant to cover the world, as it would enclose it. But, inside these areas, which can be fairly large, the experience is smartly capped. In this way, taking a specific solution and executing it smoothly or failing and struggling through the different possibilities are both possible without penalty, as the one struggling will eventually hit the experience cap and won't be able to abuse game mechanics.
Now, imaging that there are links between those areas. If you fail to sneak past some mobs and to kill them quickly, some alarm is given. If it happens, the next area will be harder, but will not give more experience than its easier counterpart.
Wouldn't that be immersive and still fair?