Here's some ideas for what each spell cast could cost you:
- Your field of vision contracts by 1%.
- Your vision loses 1% saturation.
- You lose 5% of your HP.
- Tooltips display one fewer piece of information on enemies. Example: you no longer know enemy HP.
- Your attack rate becomes more variable.
- Your minimap becomes less exact (as your memory worsens).
- You become exhausted more quickly.
- Your ability to carry and use heavy equipment decreases.
- Your shots are more likely to go wide. (Not a decrease in accuracy as much as an increase in shots that are seriously off, by like 15 degrees.)
- The delay between pressing the attack button and the attack happening becomes longer. (But remains consistent.)
- Enemies' attack telegraph animations become faster. (So you have less time to react.)
- You develop an increasingly loud wheeze that alerts enemies even when they're not in view.
- You become sallow and develop an unpleasant smell. Shopkeepers' prices rise. Wild animals start attacking you.
- You have to stand still for an increasingly long time before you can use ranged weapons.
- Drinking a potion causes you to stand still, unable to attack, for an increasingly long time. (No more quick uncorking and gulping. Your trembling fingers have to fumble open the cork and slowly raise the bottle to your mouth.)
- Popping open the inventory or menus no longer pauses the game.
- Your target reticle becomes smaller and fainter, and acquires a delay and wobble. You have to keep your mouse steady for a small fraction of a second if you want to aim precisely. (Assuming this is a top-down game.) The view begins following around the target reticle instead of being centered on you, making it hard to see what's behind you.
- Useful sounds like the noises of monsters around corners become fainter. Useless sounds like dripping water, creaking, buzzing flies, and your own heartbeat begin dominating the soundscape.
- Your knowledge of ancient languages becomes fuzzy, keeping you from reading inscriptions on walls and doorways and labels on potions you find.
- Your abilities are greatly reduced in the first thirty seconds of fighting, which means you have to keep going and plow through your enemies without pause.
Not all of these may be a good idea, or applicable, but it's food for thought. What kind of magical side-effects would you find cool in a game?
Addendum: Come to think of it, this really doesn't work as stated. Making you worse off each time you cast will just make the player play in a really conservative style to avoid having to cast spells - which makes the game really boring.
Instead, it might be better that you gain new spells or improvement to existing spells as part of a bargain with some dark gods. Want to be able to cast Blinding Flash? Well, the Eyeless One is happy to oblige, but it'll cost you a third of your vision range.
There could also be bargains which don't take anything away but constrain your actions:
- You are put under a geas that makes certain conversation choices unavailable. (You can choose them, but that permanently breaks the bargain, and you lose the powers.)
- The god of reckless courage requires you to never run away from a fight. (The UI will give you plenty of warning.)
- The god of idiotic pacifism requires you to never shoot first: you may only attack enemies that have attacked you first. (You can just not target enemies that haven't. Again, you can choose to break the pact at any time.)