So, I've been talking about this for quite a time but it always went out without saying, without answering this simple question: "What do I mean when I say improvising a game?" Or in other words, what does the GM do when the GM is improvising a game?
The truth is of course far simpler than it seems. When improvising a game, we don't come with a blank mind; we come with a few things for the start and roll from there. We don't forget what we've prepared because it can prove useful in a future thing. It doesn't mean to say the first thing that pops into our mind, although it sure can be useful sometimes. It means a simple thing: Going with the flow.
When we improvise a game, we usually come with a few dots of info, whether written or not, and roll from there. A great way to start an improvised game is by asking the players a few things about what they look for in the game (genre, scope, and the like) and then asking them a few questions about their characters' past. It is not mandatory, of course.
The most important thing, though, when improvising a game, is to go with the flow. It is made of three things:
1) Always say yes.
Not that surprising is the fact that saying no draws the game into a halt. Sure, there are things that deserve a no, but unless it is that far from normality, say yes. Say a lot of yeses, actually. The players will come with interesting things, very interesting most of the times, and letting them go with it will enrich the story.
Think about this situation: The characters are standing against a guard that prevents them from entering the castle to save the prince (princesses don't need to be rescued; they'll kill the dragon by themselves...). They have to find a way to get past the guard.
The GM may expect the characters to kill the guard, to persuade the guard or even to sneak past the guard. Another group will try to persuade the guard into joining their team ("we all know that Mr. Dragon doesn't pay you enough..."), teleport past the dragon (solving the problem without facing it) and so on. A railroady GM might stop the latter 2, as they destroy what the GM planned, but isn't it far more interesting if the guard will join them? Isn't creating a "no-teleportation-zone" like cheating?
2) Don't say just yes.
I've mentioned it once, and I'm mentioning it again. Adding the and/but spectrum to the answer will enrich the story with a little addition. A really little addition.
What will be more interesting, "you get to the place just in time" or "you get to the place just in time, but the place is crowded with many many cultists?" Adding a little and/but to every answer will make you seen like a master GM (s/he always has cool ideas and stories) 'cause everything will become far more interesting.
3) Understand that you're not the only storyteller
This is important, really really important. If you want to improvise well, you've got to understand that you're only a part of the group, a very important one, like a director, but not more than that. When improvising, the GM is more the one that shows the direction, and the players are more the ones who create the story.
The idea is to make it seem like a story of your own, but it isn't the truth. You show the direction by the way that you speak about things, by the way that you describe or play NPCs or anything else, but the players are there to do the real thing. Morpheus described it well: "[I] can only show you the door. You're the one that has to walk through it."