Mother Jones has a cool chart of bank mergers explaining how the big four banks (Citi, Chase, Bank of America and Wells Fargo) got where they are. (Click on chart for larger image.)
Consolidation has certainly shrunk the number of banks, BUT there have also been 2,315 new banks chartered in the past 20 years. (source: FDIC)
Here's what I tell bank executives on the subject: when the big guys are firing on all cylinders, they have a cost advantage over smaller banks. Size does help lower costs. However, they rarely fire on all cyclinders. There seems to always be some service failure in some part of a large organization. They have trouble digesting an acquisition, or a computer system conversion fails badly, or a key executive is too focused on golf and empire-building to ensure good customer service, or the CFO finds a new way to boost short-term earnings at the expense of long-term relationships.
When the big boys have a sick division, they don't heal very quickly.
In contrast, community bank management knows their problems immediately, because they know their customers much better. They usually fix their service issues quickly. If not, they lose market share quickly. Although smaller banks have higher cost structures, their quick self-healing balances their higher costs.
The mid-sized banks really have to be on top of their game. They don't quite have the economies of scale of the largest banks, but they have enough layers of management that problems may not be addressed immediately. On the positive side, they have the size to better capitalize on the weaknesses of the big banks.
In other words, there is a niche for all sizes of banks. And when the big banks fail to serve their customers well, there will always be a small bank (or credit union) ready to steal those relationships.