Unless you have a certain type of group, that's tough to pull of in a PnP RPG. The spotlight gets shared all the time. And because these are real, live people at the table, the more of them there are, the harder it gets.
Some of my favorite moments in gaming have been with two players and a GM. [I don't remember ever playing solo (player+GM) but I'm certain that can be very fun, too.] Frodo and Sam, the Elric brothers, Fafhfrd and the Mouser. It can be good gaming, but it can get lonely. It's great for a very focused, very specific kind of campaign, I think.
Three players is probably my ideal setup. It creates a triangle, gives you more to work with, shakes up the inter-party dynamics up a bit. It works best for me.
Four is almost the same, but a little harder to handle. To me it always feels like someone is being left out, even if just for a few moments, as if the spotlight only had room for three at a time and you have to shuffle it around constantly.
With five, for me, things start to seriously break down. We were playing Pathfinder yesterday and as it is with later-edition D&D, each individual turn in combat can take quite a while. The more people in the group, the more you have to wait for your turn and the longer the combat lasts (4e can be better or worse with this, if you're indecisive it's terrible, because there are, on average, more options, but 4e also means you have to pay more attention during other people's turns and get to act out of turn, so you're more involved). Out of combat it doesn't get any better. When the characters hit town, each with their own shopping list and agenda, aiming the spotlight around, trying to reconcile time and geography as everyone wanders off on their own is exhausting. I did not envy the GM yesterday but I didn't really do anything to help either.
I don't think I've ever gamed with more than 5+GM, but I've often heard of groups of size 6+, I think a friend of mine said they used to play Vampire with 10+ people at the table (I should ask her to confirm) (As an aside: Original D&D vaguely reefers to the number of players in double digits as if it were the expected number.). I know groups of this size exist, but I can't really imagine it at the table. I remember John Stavropolous talking about the dynamics of one such group. It's not really a game anymore but a social gathering. I think in many groups over 4 (and that's certainly a rule for us) and almost certainly in any group over 8, there will inevitably be people fiddling with their phones, playing Angry Birds, hogging up the computer to check their mail, wandering off to the kitchen for long periods of time, doodling and stacking dice...in short: not playing. If you're there mainly to hang out with your friends that's cool, but it can also be quite tiresome.
If you're reading: What's your average group size? How many people do you play with? What is your ideal group size?