Organizing contacts by category tells you nothing about who matters; prioritizing tells you where to spend your time.
The idea
I cook a lot, and I own way too many spices. The ones I reach for constantly (garlic powder, chili powder, salt, paprika) sit up front, and saffron and ras el hanout (whatever that is) live in the back. Alphabetizing the shelf or sorting it by region wouldn't help me find anything faster. Contacts work the same way. Labels like Leads, CEOs, People in DC or Investors are accurate, but they can't tell you who matters more. Hot leads, Top past clients or Current investors tell you who to spend time on first.
How to do it
Look at your current groups or tags. For each one, ask: does this tell me who to talk to first?
Keep the descriptive ones you actually use, like city, which helps when you travel.
Add a priority layer on top. Today I keep it to three tiers: Sphere A, B and C.
When a name comes up, ask whether this relationship matters to your current goals. If you can't answer, your setup isn't prioritized.
Example
Say you're flying to Washington and filter your contacts by "DC." Organization gets you the list of everyone there. Only prioritization tells you who you care most about seeing while you're in town.
Watch out for
Putting relative weight on people feels uncomfortable. Do it anyway, or you'll misallocate your time.
A/B/C Spheres: Sort everyone into three tiers by gut feeling (love, like, don't know) and spend more time on A's than B's, and more on B's than C's.
CAPITAL: Prioritize: The P in CAPITAL - decide who matters most right now and focus there, because you can't give everyone in your database equal time.
Groups vs. Tags: Put each person in one group by how you know them (client, past client, sphere, prospect, vendor, peer) and use tags for everything else.