Personally I am not too much for minigames. If they are optional and fit into the world as a whole and at least are moderately fun (and does not take too many resources from the core game) fine, but mandatory minigames please no. That is not to say I dislike them as such, I just rarely play them. One of the few I liked was pazzak, it was fast, moderately fun and within lore of the game. I would however be irritated if I had to play pazzak 5-10 times in a row, which could happen in such a system even if I was not constantly reloading to get a successful roll.
A hard part of the game could result in me dying regularly and thus would also trigger it.
If there where to be a penalty like this I would rather go with the style that New Vegas (IIRC) had where you could not play in the casino for something like 1 minute after reloading. A similar system could be made for skillchecks. That is not to say I liked it, but I would prefer it over many other options.
The point is: Not liking minigames indeed IS the point. It's all about making some sort of investment by the player. Just imagine you think about to open a chest with your mage, but you don't know if it is trapped ... Today you just try. Error means reload. There is no need to sacrifice some spell-slot or magic energy on a 'find trap' spell. Now think about you have to play a very small, probably boring, minigame that maybe justs take 20seconds before reloading. How do you react? Likely you will make the investment for the spell to avoid that bloody minigame. am I right? I think that adds to the story of roleplaying experience when you don't always have the perfect character(s) which can do everything.
It would help the less disciplined people, who want to do this kind of roleplaying but can't resist the temptation, be more disciplined. Like myself .
For those who are just annoyed by this simple thing, it could be easily diabled in the options.
There are many games that increas in fun when the player has to make a personal investment ... Poker for an instance ;-) So it is not very unusual.
Just by the way: Minigames inside the game .. I am also no fan of that..