The buyer’s scorecard
How to choose a marketing agency (without getting seduced by the pitch)
Most agency searches reward the agency that is best at getting picked in an agency search, not the best agency.
If you’ve been through a few agency pitches, you recognize some of this. The deck is gorgeous. The logos slide has a recognizable name on it. The founder is charming and confident. The spec campaign is suspiciously perfect. Everyone leaves feeling like they just watched the trailer for a movie that’s going to be great.
Then the contract gets signed, the founder disappears, the spec campaign turns out to be unbuildable with your actual data, and the “integrated team” is a 23-year-old account coordinator and a Slack channel.
I’ve spent 25 years inside agencies: building them, selling them, and sitting on the other side of the table from thousands of buyers. I’ve won pitches I shouldn’t have won, and lost pitches even though I knew my agency was the best fit. In one of those losses, my team gave a technically brilliant SEO presentation. The client told us later that they never understood how it would work for their business, so another agency won. They came back six months later after that agency stalled out. Gratifying, sure. I’d still rather have won the business the first time.
We proved we knew SEO. What we didn’t prove was that we understood the prospect. It doesn’t matter how good the agency is, how slick its presentation is, or how impressive its testimonials are. If it doesn’t understand your business, it isn’t the right agency for you.
The TL;DR checklist
Choose the agency that gives you the strongest evidence across five areas:
- Services fit: The service mix matches the problems you’ve identified as primary.
- Client fit: The agency has worked with companies that share your economics and constraints, not merely your industry.
- Team: The people doing the work have the insight, capacity, and judgment to deliver.
- Pricing: The fee, scope, and incentives make commercial sense on both sides.
- Evidence: The agency can connect its work to a business outcome you can verify.
Fame, awards, and chemistry can break a tie. They shouldn’t build the shortlist.
1. Write your decision brief before you start the search
Agencies are very good at helping you define your problem in a way that happens to require their services. That isn’t sinister; it’s how sales works. Decide what your issues are and which services you are likely buying before some agency slickster with a $400 haircut starts charming you.
The ANA and 4A’s joint research on agency search found the same failures repeatedly: no budget disclosure, no clear success criteria, decision-makers hiding until the final round, and nobody agreeing on who owns the pitch work. TrinityP3, who run searches for a living, recommend agreeing on the assessment method before the search starts.
Your brief should fit on one page and answer:
- What business condition needs to change?
- What does meaningful progress look like in 12 months?
- Which constraints are fixed: budget, timeline, tech stack, or legal?
- What can your internal team handle well?
- Which capabilities, experience, and certifications must the agency bring?
- What is the target budget for fees, media, production, tools, and creators?
- Who is involved in the decision, who can say yes, and who can veto it?
- Why is the current approach failing?
- What has to happen in the first 90 days for you to continue?
2. Apply the hard gates before you score anything
A weighted scorecard can hide a fatal mismatch. An agency can score beautifully in four categories and still be unusable because it works for your biggest competitor. Filter these out first:
- Competitive conflicts you can’t live with
- Required privacy, security, or regulatory capabilities
- Geographic or language coverage
- A hard budget ceiling
- A mandatory platform credential
- The ability to start inside your window
- Contract, data, or IP terms your legal team will never sign
Keep the list short. Every item should be something that would end the relationship if it were missing. There is a large difference between a must-have and a preference wearing a fake mustache.
The 100-point agency scorecard
Score every agency from 0 to 5 in each category. Multiply by the weight. Add it up.
| Category | Weight | Multiplier | What a 5 looks like |
|---|---|---|---|
| Services fit | 25 | ×5 | A clear diagnosis, the right service mix, sensible exclusions, and a credible first-90-days plan |
| Client fit | 20 | ×4 | Relevant work with comparable business models, constraints, buyers, and growth stages |
| Team | 20 | ×4 | Named people, clear roles, real availability, appropriate seniority, and a continuity plan |
| Pricing | 15 | ×3 | Transparent assumptions, sustainable staffing, clear change control, and aligned incentives |
| Evidence | 20 | ×4 | Checkable results tied to business outcomes, supported by references and a real measurement plan |
Use the same evidence standard for every score:
- 0
- No answer, or no evidence.
- 1
- A claim with serious contradictions.
- 2
- A generic answer with little supporting detail.
- 3
- A credible answer with an important gap.
- 4
- Strong, relevant evidence with a minor caveat.
- 5
- Specific, checkable evidence that maps directly to your situation.
Everyone on your evaluation team scores independently before the group discussion. Don’t merely average the scores; compare the highest and lowest in each category. When your CFO gives an agency a 2 on pricing and your marketing director gives it a 5, you’ve found the conversation you needed to have.
These categories match the pieces an Agency Field Guide profile separates: services, named clients, team size, pricing disclosure, and case studies. You can check them individually instead of taking the whole package on faith.
Services fit: 25 points
A good agency makes your problem clearer before prescribing anything. A mediocre agency agrees with your diagnosis in the first meeting and starts talking about deliverables.
Listen for whether they:
- Restate your business problem in their own words—and make it sharper.
- Ask questions that expose assumptions in your brief.
- Separate symptoms from causes.
- Explain which services belong together and which you should skip.
- Draw a boundary around work they shouldn’t do.
- Name the risks and dependencies that could sink the engagement.
Ask directly:
- “What do you think is actually causing this?”
- “Which part of our brief would you push back on?”
- “What should stay with our internal team?”
- “Which services would you leave out, and why?”
- “What would you need to learn before committing to a plan?”
- “What would make this engagement fail?”
If an agency agrees with every part of your diagnosis after one conversation, either you wrote the most perfect brief in marketing history or they want the work. I’ve been on the agency side of that call. It is almost always the second one.
Specialist, full-service, or a mix?
Match the model to the shape of your problem and your ability to coordinate.
| Your situation | Likely model |
|---|---|
| One contained channel is underperforming | Specialist |
| Several channels share data, creative, and customer journeys | Integrated or lead agency |
| Strategy is clear, execution capacity is missing | Execution specialist |
| Strategy itself is uncertain | Strategy-led agency, or a paid diagnostic first |
| You have strong internal marketing leadership | Multiple specialists can work |
| Nobody internally can herd multiple partners | One lead agency with named specialist support |
“Full-service” deserves a follow-up every time. Sometimes it means a real integrated team under one roof. Sometimes it means a project manager with a good contractor list and a markup. Neither is inherently wrong. You need to know which one you’re buying, because the second comes with a coordination tax.
Client fit: 20 points
Industry experience matters when the learning curve is expensive. Regulated categories, weird seasonality, long sales cycles, complex buying committees, and technical products reward an agency that has already made the mistakes on someone else’s dime.
But the logo slide will lie to you if you let it.
A global beauty conglomerate and a founder-led beauty subscription brand share a category and almost nothing else. Look for similarity in the things that shape the work: business model, buyer, growth stage, sales cycle, price point, distribution, regulation, internal maturity, geography, and the specific problem being solved.
“Show us the client situation most comparable to ours. Then tell us the most important way it is different.”
A credible agency knows the differences. A less thoughtful one tells you it is basically the same.
The Agency Field Guide directory lets you filter by niche and inspect named-client experience. A roster proves exposure. The agency’s role, the relevance of the work, and the result still need to be established.
Team: 20 points
You are hiring an agency. More specifically, you are hiring an account team made of specific humans. Evaluate them that way.
Size tells you about capacity, access, specialization, and key-person risk. It does not tell you about quality. A five-person senior shop may be perfect for a narrow, deep remit and terrible at managing 20 markets with high-frequency message testing. A 500-person network has enormous depth, but your budget buys only a fraction of it.
Ask for a working-team map that includes:
- Each person’s name, experience, and role
- How many other accounts each person supports
- Who makes strategic decisions
- Who serves as the day-to-day contact
- Who handles escalation
Meet those people before signing. The founder and new-business director may sell the work; you live with the delivery team. The IPA’s chemistry-meeting guidance makes the same point: the delivery team belongs in the evaluation.
The pitch-team switch is the oldest trick in the book, and it is not even a trick. It is structural. The people great at winning business are rarely the same people great at doing it. Ask plainly who will lead the account and meet the likely primary contacts as early as possible.
Pricing: 15 points
Comparing retainers is not an apples-to-apples comparison. It is a scope comparison.
Build the expected annual cost for every agency using the same categories:
- Strategy and account-management fees
- Creative and production costs
- Media commissions or management fees
- Technology, data, and research costs
- Creator and influencer payments
- Travel and pass-through expenses
- Setup or onboarding fees
- Performance incentives and annual increases
- Likely out-of-scope work—be honest about this one
The lowest price often has the least well-scoped proposal. Ask what is included and excluded, what triggers a change order, which client delays affect cost, who approves additional spend, what happens when priorities change, and what termination looks like.
Outcome-based pricing can work, and I would rather see it than hourly billing. In Agency Inner Circle #103, I argued that agencies should sell outcomes and report company metrics instead of platform metrics. From the buyer’s side, a performance fee works only when both parties define the baseline, attribution method, authoritative data source, measurement window, external factors, client responsibilities, caps, floors, and payment timing—in writing.
Without those, a performance fee is an argument scheduled several months in advance.
Evidence: 20 points
Agency evidence has levels, and most of what you will see is near the bottom.
| Level | What the agency provides |
|---|---|
| 0 | Claims, awards, badges, adjectives |
| 1 | A logo with no explanation of the work |
| 2 | A named client and a defined scope |
| 3 | A baseline, timeframe, agency role, and measured outcome |
| 4 | A comparable client problem with the important constraints disclosed |
| 5 | Checkable results, a live reference, and an honest account of what the agency learned |
Every case study should answer:
- What was true before the work?
- What business result was the client chasing?
- What did the agency do, and what did the client do?
- How long did the change take?
- Which metric moved, and from what baseline?
- Where did the data come from?
- What outside factors affected the result?
- What would the agency do differently now?
A giant percentage without a baseline belongs in the decorative arts. “We grew organic traffic 400%” from 200 visits a month is a good week for a food blog.
Of the 4,392 agencies currently in Agency Field Guide, 78% publish at least one case study, 24% publish pricing, and 25% publish usable team-size information.Agency Field Guide index, recomputed from published profiles
We do everything we can to determine what agencies do, whom they have worked with, and what it is like to work with them. Some evidence remains trapped on the other side of a sales call. Our methodology explains how the published facts are assembled.
Platform badges get the same treatment. A Google Premier Partner badge or Meta Business Partner badge signals platform exposure. It does not prove strong work for a company like yours. Our partner-badge guide covers what to check next.
Replace the beauty parade with a working session
Instead of relying on spec work, pay for a working session on a real business problem. Give the agency something true—a dashboard, customer interview, or product constraint—and watch how they:
- Ask questions and separate knowledge from assumptions
- Handle incomplete information
- Challenge your team without being jerks about it
- Bring the right people into the conversation
- Explain technical ideas to non-technical people
- Change direction when new information appears
- Turn discussion into decisions and next steps
TrinityP3’s selection guidance recommends workshops, supported references, and a resourced scope naming the people who will work on the account. It also suggests taking only two or three agencies into the expensive final stage. That gives everyone room to behave like humans instead of contestants.
Ask reference questions that produce information
“Were you happy with them?” is a question polite adults can answer for ten minutes without saying anything. Ask these instead:
- What did you originally hire them to change?
- Who did most of the work after the contract was signed?
- What surprised you in the first three months?
- Tell me about a time they pushed back on your team.
- What happened when performance dipped or a plan failed?
- How did they handle work outside the original scope?
- Which part of the relationship required the most management?
- What kind of company should not hire this agency?
- Would you hire the same team again?
- What do you wish you had clarified before signing?
- What value did you receive that you did not know you were paying for?
Ask the agency for one more reference: a client who left. Every agency has them. How both parties discuss that relationship tells you more than three happy references combined. An agency claiming it has never lost a client is either brand new or not being straight with you.
Also ask about average client tenure. An agency whose clients stay three or four years has a different operating model from one that churns every 14 months, no matter what the case studies say.
Use a paid first engagement when you are still not sure
A paid diagnostic, audit, or 30-day planning sprint reduces risk on both sides and tests the relationship better than a pitch. Define the question it will answer, who is involved, the evidence it must produce, the decision it supports, ownership, price, timing, and the conditions for moving into a larger engagement.
The output should remain useful even if you hire someone else. You get direct evidence of how the agency thinks, communicates, and handles your organization. The agency gets enough access to replace pitch assumptions with a real plan. The total cost is usually less than one month of the retainer you were about to sign blind.
Score chemistry through behavior, not lunch
You will spend many hours with these people. Chemistry matters. Define it through behavior:
- They listen more than they present.
- They make your problem clearer, not fuzzier.
- They explain hard ideas without hiding inside jargon.
- They disagree without turning the room into Thanksgiving dinner.
- They show curiosity about your business, not just your budget.
- Their working pace matches yours.
- The people on their side visibly respect one another.
- You can imagine delivering bad news to them—and receiving it.
Shared hobbies, expensive sneakers, and a great lunch are lovely. They have a disappointing relationship with marketing performance.
Red flags worth a closer look
None is automatically disqualifying. Every one should provoke a harder question.
- The proposal arrives before discovery is finished.
- Every case study is for a company much larger or smaller than yours.
- They cannot name a prospect they were not right for.
- Reporting samples stop at impressions, clicks, and sessions.
- They want to own the ad accounts.
- They agree with everything.
Final checklist before you sign
- Every evaluator scored the agencies independently.
- The scope identifies client and agency responsibilities.
- Annual cost has been normalized across finalists.
- Case-study claims have been checked.
- References, including a departed client, have been called.
- Success metrics connect to the business, not the platform.
- Change-order rules are clear.
- Data, IP, ad-account, and termination terms are understood.
Then make the call.
Your gut gets one vote. The evidence gets the other 99.