Considering how many games exist for the promise of bringing you exciting action and adventure, it's a shame how few of them have memorable villains. Like a good story in general, a proper villain can help motivate a player to push his way through a game, past the difficulty spikes and through the inevitable rough patches, by adding meaning and context to the game mechanics as well as the promise of more variety and surprises. There are a few reasons why villains are hard to do in games, but games like Portal, Legacy of Kain: Soul Reaver, Final Fantasy 7, System Shock 2, or even Super Mario Bros. and Mike Tyson's Punch-Out!! are evidence of why investing in a proper villain is worth it.
One reason there are so few proper villains in games is implied by the word itself: The concept of villainy is kind of dumb. It's not how the world works. In reality, what happens is that when two people want opposite and mutually-exclusive things, they enter into an antagonistic relationship. Villainy is just an extreme form of antagonism where, most often, either the antagonist's motives are not rational or simply not well-developed. Video games' misguided attempts at villains usually hinge on grandiose schemes such as destroying the world or other sadistic, evil acts. They're bad guys who overcompensate for their flat desires with huge lifebars. But it's impossible to relate to their motivations so these villains are doomed to obscurity.
Instead, a proper antagonist gets under your skin and makes things personal in a way you could understand, even appreciate. Portal's GlaDos, despite being a machine, has the attitude of a spurned lover taking passive-aggressive revenge on a relationship that's slipped from her grasp. In Super Mario Bros., Bowser wants the Princess just as much as you. It's ironic that inhuman characters such as these turn out to be much easier to empathize with than the dime-a-dozen megalomaniacs waiting for you at the end of most games. But that's the key — if the antagonist is impossible to empathize with, then he's just another villain, and more than likely doesn't have the substance to be memorable as a character.
There's a more-practical reason why it's tough to have a proper antagonist in a game, which is that most games in the action or action-adventure genres are designed around kill-or-be-killed scenarios, leaving little room for character development. When they present you with an antagonist character and a combat situation, one of you needs to be defeated and it's not going to be you if you keep trying. So then, either the antagonist is knocked out of the game or you get the cliché of the antagonist escaping just in the nick of time, or even worse, the one where he beats you up in a cutscene after you kick his ass in-game.
What many games do to counteract this is they present you with a hodgepodge of disposable antagonists, in the form of different boss characters and such. But the narrative consequence is that the forces of antagonism in the game are diluted. Unless it's Metal Gear Solid, the story likely doesn't make time to develop most of these characters, and the artists and combat designers have to carry the burden of making them interesting when the fiction should be holding up its end of the bargain.
Conversely, the reason why games like Portal, System Shock II, BioShock, Batman: Arkham Asylum, and even Punch-Out!! succeed with their antagonists is that their stories are structured around an ever-present-but-physically-inaccessible antagonist, someone you're always aware of but can't get to until the climax of the game.
If a gameworld cannot support the idea of an ever-present but physically-inaccessible antagonist, then the burden is on the enemy faction to be empathetic, assuming this is compatible with the aesthetic of the gameworld. The enemy faction or factions you're fighting — the various goons that populate the gameworld and are the predicates of the gameplay — might as well be interesting. There's really no downside. And giving them empathetic qualities is a good way to make them interesting in most cases.
In Geometry Wars, the evasive little green shapes are the best, most memorable enemies in the game because they have an empathetic sense of self-preservation. In Halo, the little Grunts freak out when you kill one of the Elites leading them into battle. In Panzer Dragoon Orta, the hordes of enemies forcing you to dodge a hailstorm of bullets panic as you gun them down, making you think about what it must be like to be on the bad guys' side in a shoot-'em-up. In Plants vs. Zombies, the zombies are sincerely hungry for brains — you'd want to give them your brains if you didn't need them.
Since most traditional games revolve around violent conflict, in these games, the forces of antagonism ought to express empathetic behaviors, even in the strict confines of a combat encounter. It's totally doable and relatively inexpensive in many cases, just the cost of writing and audio in many cases (plus a high premium in scripting, animation, and artificial intelligence for all the shooters out there). Even F.E.A.R.'s genetically cloned supersoldiers in their full body armor show a shred of humanity in their final moments simply by saying "Oh, shit…!" as you strafe around the corner in slow motion with your shotgun drawn.
While memorable antagonists are rare, antagonists are some of the most memorable characters in games. That's because they often present far greater opportunities for character development than protagonists do. Look at games like BioShock and Portal, whose protagonists primarily serve as vessels for the player to immerse themselves into the experience, yet whose antagonists are extremely well-crafted, remarkable characters.
Part of why the combination of invisible-protagonist and ever-present-antagonist works so well in these games is that, when the climactic moments of the story crop up, the antagonists' escalating actions feel very personal. And when these highly motivating personal affronts are coming from characters whose own motives you can empathize with on some level — characters for whom the old "we're not so different, you and I" speech goes without saying — you're more likely to be playing a game that's going to stick with you after you're finished playing it. Which are the kinds of games I most like and most want to make.