← Case Study Interviews: Mini Case Studies  /  all explorations

Read the first article before you read this one

How to Recruit Better with Case Studies explains how to design, run and read a case study inside the interview. This article continues from there and does not repeat it.

Part two / before the room, and after the slip

Every single thing is testable, if you design it.

A case study is not an hour in an interview room. It is anything you arrange so you can watch someone do the work. Some take an hour. Some take one minute, and the candidate does not know one is running until you tell them.

So the interview does not start in the interview. It starts with your first email, and most of what you need to know about how organized someone is has already reached your inbox by the time you meet. Left unscored, that evidence costs you an hour with someone you could have ruled out on Tuesday. It also costs you a hire who sends the same sloppy invite to your clients.

01The job

Sell the role hard, then find out who it is for.

Every role is amazing or terrible, and the only thing that decides which is the person sitting in it. For the right person the job really is amazing. For someone else it is terrible, and no amount of talent changes that.

That makes the job description a sales page rather than an announcement. A hiring manager described the old habit after rewriting theirs:

I was thinking of it as, okay, I'm just informing people. But when you put it as, I'm selling it, then it needs a whole different approach.

Selling hard is not lying, because for the right person every word is true. It does give you a second job, since you have now made the role sound amazing to everyone. That second job has two halves:

  • Will this person be amazing at the role?
  • Will this role be amazing for them? Someone who is great at the work and hates it will leave, or do a poor job, or want to do something else.

Most hiring looks only at the first half. Both need testing. For a role like this, every single thing can be tested, as long as you design the test and then pay attention to every signal it sends back.

02The example

Let's do 10 AM. Mind sending an invite?

You email a candidate for a customer success role to set up the interview. They reply with three times, and one of them works. What do you write back?

What nearly everyone does

"Great, 10 AM works. I'll set it up."

Polite and efficient, and you have just done their homework. You also spent your own time and learned nothing.

What you write instead

"Let's do 10 AM. Mind sending an invite?"

The same length, and it saves you the work. It also starts a case study that runs by itself.

Part of a customer success job is getting people into meetings, so you just asked them to do a small piece of the job for someone who will read it closely. Here is what comes back from two different people.

The same thread, two candidates

The one you meet

5 minutes later

"Here are three times that work for me."Exact times, because that is how you set up a meeting without four more emails.

+1

You

"Let's do 10 AM. Mind sending an invite?"

Within the hour

The invite arrives.Fast, so the slot is held and nobody is double-booked.

+1

Same invite

Clear title, right time, CV attached.Organized, on a small thing, without being asked.

+1

You

"Please apply on the site before we meet."

Soon after

The application is in, and you did not chase it.

+1

Four points, and you have not spoken yet. You go into the interview already interested.

The one you never meet

Slow

"When are you available?"The question alone is neutral. The delay is not.

−1

You

"How about 10 AM?"

Slow again

"OK."

−1

You

"Mind sending an invite?"

A day later

The invite arrives.

−1

Same invite

Lowercase title, two first names, no details.

−1

Same invite

The time on it is wrong.

−1

Three was enough. You are done before you begin, so you cancel the interview.

Names and details in these examples are invented. Both threads are composites of real candidates.

Two tests in one invite

The invite gets two separate scores: did it come quickly, and is it good? Put two side by side and the second score needs no explaining.

The same 10 AM meeting

Scores a loss

noa / daniel

Tuesday, 3:00 PM

Scores a gain

1Interview: Noa Levi and Daniel Bar, 2Acme, Customer Success Lead

3Tuesday, 10:00 to 10:45 AM

4Daniel Bar CV.pdf

  • 1Both names, and the word "interview", with capitals where capitals go.
  • 2The company, and what the meeting is about.
  • 3The time you agreed. People get this wrong often, and it is a big deal.
  • 4The CV, unasked. Extra points.

If they mess it up now, when it's super important and visible, they're absolutely going to mess it up later.

A couple of hours is the line.

Speed matters because of what happens in real life. Someone who leaves the invite for later does not block the time, and then they are double-booked. Past a couple of hours you are not looking at a busy person. You are looking at someone who does not close small loops when they come up.

A booking link beats three good times.

Google Calendar can give you a link that lets the other person book the meeting. A candidate who sends one has removed the back-and-forth entirely, which is exactly the instinct you are hiring for.

03The score

Gain points, stay neutral, or lose points.

Every small signal lands in one of three places. Most of the skill is refusing to treat them as the same thing.

  • Gain. A reply in five minutes with three exact times, because that is the job.
  • Neutral. A quick reply asking when you are available. Not impressive, and not yet bad. You note it.
  • Lose. Forgetting to respond, or being messy about it. If that is the whole picture, you do not need to meet them.

Then you count. Fail three of these small tests and it is over. One slip is not a pattern, so how seriously you take a loss depends on how big the slip was and how many came with it.

If someone doesn't know how to send out good invites on time, quickly, you don't want to start there. It's too low of a starting point.

These look like small tests, and they are high value, because they measure the things you least want to deal with after the person joins. You can teach someone your product. You do not want to spend your first month teaching them to close loops.

04The buckets

Three buckets, named so the name does the work.

Before you can test anything you have to decide what you are testing for. That is the performance review you would hold this person to a year from now, reduced to three buckets. One manager came in with six, moved to three, and still got the three wrong the first time. The change that fixed them was in the names.

The names matter more than they look, because they are what people end up remembering. There is one test, and it is worth more than keeping them short:

If people only read the names, how close will they be to inferring what we believe?

Here is the same role, for a customer success manager, before and after applying that test.

Amazing professional

Reliable, productive professional

Gets the job done without you asking twice, and goes looking for tools that make them faster.

Commercial instinct

Trusted advisor with commercial focus

Understands the numbers and the client's goals, and tells the client what they need to hear.

Client connection and communication

Deep personal care and relationships

Cares about people's lives, their challenges and their hopes, whether or not any of it touches the business.

You often know the wrong name first.

Many times you know something is the wrong answer before you know what the right answer is. "Amazing" said nothing about what was inside the bucket, and that was enough to reject it with no replacement in hand. Keep rewriting until you read the name and can see the person. When a name is right it feels right, and that feeling has a lot of depth behind it.

Decide what the role does not want.

An earlier draft had "fast learner and promoter of change" as its own bucket. For this role that was wrong. The role needed someone open to change who follows the process, rather than someone who arrives wanting to redo everything. A bucket copied from a generic list of virtues will test for the wrong person.

Warm and trusted are two different boxes

The argument that fixed these buckets was whether warmth belongs inside "trusted advisor". It does not. People who are warm and caring are often far from being a trusted advisor. A trusted advisor will tell you, specifically, that what you are doing is terrible and that you are going to fail. That is not a warm sentence.

Personal care has nothing to do with any of that. It is asking about someone's kids, remembering the birthday, being interested in how it is going at home. If they stopped doing their job tomorrow, you would still care. It may not lead to better business results at all, and it still matters, because it is how people want to live their lives.

It also buys you something the other box cannot. When an advisor who is only an advisor messes up on the business side, there is no recourse. When the client also likes you as a person, they want you to succeed.

Set a separate bar on each one

You will rarely find a ten in everything, and a real ten as a trusted advisor is very hard to get. So decide the level you need in each bucket separately. For this role the bar was about a five out of ten as a trusted advisor, and an eight or nine on personal care.

Another role would flip that, because the bar comes from who you serve. Advise high-powered CEOs and caring about their families matters only to a degree, since they want to get to business. Serve people who run nonprofits, who chose their work for idealistic reasons, and the personal side moves up. Both kinds of client come back. They come back for different things.

Revenue goes in as an outcome, not an activity.

The first draft of the trusted advisor bucket listed retention numbers, pipeline and upsells. Those are the results of being a trusted advisor. They are not what the person does. Someone in that seat does not pitch. They say: "I think you really should use this. It's going to be amazing for you. Just try it, because you trust me, and then let's talk." Keep the numbers and the goals in view, because that is what "commercial focus" means. But if you describe the bucket as selling, you will hire someone who sells, and that is a different person.

Personal care includes knowing where the lines are.

Write this into the description: able to adjust to the type of person they are speaking with. Clients do let you in eventually, and you cannot start there. A question that is natural between two people of the same age and background can destroy the relationship if asked early across a gap in age, gender or culture. Someone who cares and cannot read that is a risk, however warm they are.

05The tests

Write the tests before you write the ratings.

The natural next step is to fill in what a 1, a 3 and a 5 look like in each bucket. Do this first. Each bucket gets a list of tests, and every one of them is a case study, including the ones that take a minute.

How long a case study takes

1231 min3 to 15 min30 to 60 min
  • 1One sentence to write, one minute to run."Did they send a good, organized invite quickly?" That is two tests and two scores.
  • 2A mini case study.Tests one skill in three, five or fifteen minutes. Writing it might take half a page.
  • 3A full case study.Half an hour or an hour, as in the first article.
Some tests you will not know how to write yet. Name the skill in the bucket, leave the gap visible, and solve it later. That is what the advisor at the end of this article is for.

For the reliable, productive professional you already have four tests before anyone has met: how they answer the early emails, whether they offer times, the invite, and the application. Then comes the test that separates people above a 3.

The test, said out loud

"Show me how you use AI to make yourself more productive."

Negative points

"I go on ChatGPT and use it like Google, to search for information."

Or nothing at all. If someone has nothing to show you, that tells you a lot about the bucket you named "productive".

What you are listening for

"Renewing my car license took 20 minutes of clicking on a government website. Now it's three chat messages with an agent."

Yes, that only saves 15 minutes. But there are 50 of those annoying renewals to do, and they know it.

You are not checking for a clever trick. You want to see that they push toward it by themselves.

"Why should I change?" just got far more expensive.

People who ask why they should change, why they should adopt something new, why they should push to be more productive have always been a problem. Before AI you could deal with it over a long period of time. Now, if you do not deal with it quickly, you become irrelevant.

Run this on your own product today.

Before you write this test, feel what you are testing for. Ask an AI agent to do a complete task inside your own product: log in, navigate the interface, do all the work. There may be an issue here and there. It will still change your mind about where this is headed. And if the agent turns out to be much better than the people doing that work today, the question stops being whether they want to improve. They have no chance.

06The reveal

When someone fails, reveal the test.

Say someone comes highly recommended, so you are not ready to rule them out. They took 24 hours and sent a poor invite. What do you do?

A good answer is to note it, then design another test to check whether it is a deeper problem or a one-off. That works, and it puts a lot of effort on you. The great answer is one email.

The email

"It took you 24 hours, and you sent me a very poor-looking invite. Please organize the invite so it looks highly professional. I'd also love to understand why that happened."

Tell them afterwards, plainly, that this was a test. You explain and ask, it costs you almost nothing, and it is the most revealing thing in the whole process.

Three ways it can go, and only one continues

1The reveal2Fights it3Excuses4Owns it, fixes it
  • 1The reveal.You name what happened, ask for the fix and ask why.
  • 2"Why are you so picky?"Great. You are picky, and they are not hired. That one is easy.
  • 3The dog ate my homework.Reasons about the kids, the week, the calendar. You do not care about excuses, and this is what they will bring you later.
  • 4"I'm sorry about that. Sending you a really organized invite now."Full ownership, and the fix in two to four hours. If it takes another 24, that is another knock.

The fight is the answer that should end it, because a candidate who argues here is arguing about the easiest possible case. On quality standards in your company, there is no outside truth. You decide what good is, you just told them, and at this point you hold 99 percent of the information and they hold 1 percent.

Of course we're right. Later on, when they're in the company, it all becomes more complicated.

"I didn't know what good looks like" is no defense.

Anyone can paste the email and the invite into ChatGPT and say "help me make this awesome." It will tell them what to add: a real title, an agenda, the CV. So the fix is available to everyone, and the second invite tells you whether they went and got it.

This is different from teaching in the case.

The first article has you teach when a candidate pauses, then re-test whether they can use what you handed them. That measures learning. The reveal measures something else: what happens when you tell this person they got it wrong. You will be doing that for as long as they work for you.

07The full answer

What a great answer to fifty people sounds like.

The first article gives you the setup: fifty people arrive tomorrow, and nothing is ready. It follows the candidate as far as the furniture shop. Here is the rest of a great answer, from the first sentence to the last table, with what each move shows you.

  1. "The moment you say it, I do a mini kickoff with you. What do you know, what is available, and what is missing?"They find out which problem they are solving before they solve it.
  2. "Who needs to be involved, and what budget do I have?"People and money, early, without being prompted.
  3. "Right after this call I phone every one of those people. It's tomorrow, so there is no time to set up a meeting."Their tempo matches the deadline.
  4. "I start a list of everything I need, maybe with AI: tables, chairs, refreshments, access to the building."Capture first, and a tool used by reflex.
  5. "While I start executing, I send the list to you and the other three people I spoke with, so I hear what I missed as quickly as possible."They involve others without stopping to wait for them.
  6. "Then I look for the longest and most difficult item. Tables. Nobody has a solution, and we can't ship them from headquarters. So I solve tables first."The hardest thing goes first.
  7. "I can go buy folding tables nearby for $200. I email you: this is the solution I came up with, do you have a better one?"They bring you a decision and leave you room to overrule it.
  8. "Once the hardest ones are solved, I go on to the medium ones and the easier ones."

Your candidates will not say all of this. Having it written out means you can hear how far along it they get, and which moves they never make.

08Your bar

The tests can only be as good as your standard.

A players hire A players. B players hire C players. You would expect C players to hire terribly, and they do not. C players hire randomly. So the worst person to put in charge of hiring is a B player, who reliably hires below themselves.

What moves someone up is mindset, and it is separate from capability. You can be an A in mindset and still a B in capability, and you are fine as long as you are climbing. The tell is what you consider normal. A B player thinks of a completed checklist as something to manage. An A player says: of course this is what I expect, and there is no reason anyone should not do it.

Designing these tests is how that standard gets out of your head. One manager on a team of eight asked for a specific report by the end of the day. Six sent it, with no questions and no reminders. The two who did not were the two the manager already knew had an issue, and the explanation from one of them said only that the report had not been a priority.

Every problem on the team ends up being the manager's fault. Either you fixed it, or you fire the people who aren't fixing it, and then it's fixed.

Two years earlier nobody on that team would have sent the report. That is real progress, and it is also why the door matters. What you test for at the door is what you will not have to fix later.

09The mistakes

Four ways to waste a good test.

1You send the invite yourself.

You offered to set it up, out of good manners, and the test never ran. Any small task you take off a candidate's plate is a case study you just cancelled. You will usually notice right after you hit send.

2You note it, and then you meet them anyway.

Noticing is only worth something if it can end things. Three small losses and a calendar slot still on your schedule means you scored the test and ignored the score. Rule them out. It can be very quick.

3You build a bigger test when one email would do.

A loss makes you want more evidence, so you go and design a whole new exercise. Reveal the one you already ran. You get the fix, the reason and the reaction, and all it costs you is two lines.

4You ask AI before you understand it yourself.

If you do not really get it yourself, you cannot guide the AI in the right direction. Getting it means you can say: if I do X, and I look for this and this and this, I will learn something real. Write your first tests by hand. They will be rough, and you will know why.

10The advisor

Then build yourself a case study advisor.

Writing tests is the hard part, and it stays hard. Once you have written a few by hand, this prompt turns Claude into something you can ask every time you are stuck. Paste it in and let it run. It does a lot of research, and it can take about half an hour. Save what comes back as a skill. From then on you ask it: "How would I design a case study for this?"

The prompt
I want your help designing case studies and mini case studies.

Please run a workflow to bring in best practices and approaches from McKinsey and other great case study companies, and create a case study advisor for me.

Case studies, for me, are ones that test a specific skill in a candidate I am recruiting. A mini case study also tests a skill, but in a much faster way: maybe 3 minutes, or 5 minutes, or 15 minutes, where a full case study usually takes half an hour or an hour.

Please bring in all the best practices. Here is an example of a case study.

If I am hiring a project manager, a case study could be: "Imagine you are hired and on the job tomorrow. In the morning I tell you that we had a mess-up. Tomorrow morning we have 50 people coming for a full day of sessions and classes. We have the lectures, but we are completely unorganized on where exactly and how it will happen. What do you do?"

The answer shows the way they think and how they capture everything, from figuring out what they need to solve to actually doing the solution. A great answer would be: "The moment you said it, I do a mini kickoff with you. What do you know, what is available, and what is missing? Who needs to be involved, and what budget do I have? Right after this call, because there is no time to set up a meeting, I call every one of those people on the phone and get as much information as I can. Then I understand roughly where the location will be, and I start creating, maybe with AI, a list of everything I need: tables, chairs, refreshments, access to the building. While I start executing, I send the list to you and the other three people I talked with, to see if I missed anything, so I find out as quickly as possible. Then I go down the list and look for the longest and most difficult thing to execute. Tables: no one has a solution, and we can't ship them from headquarters. So I solve tables first. One option is to go buy folding tables nearby. They cost $200. I email you: this is the solution I came up with, do you have a better one? Once I solve the hardest ones, I go on to the medium ones and the easier ones." Asking questions to clarify is really important, so they know which problem they are solving, and so is involving others.

Here is a mini case study for a customer success person. Before the interview, I ask them to send me the invite for the time we scheduled. I want to see how quickly they send it and how well structured the invite is. Does the title include my name, their name, the company and the fact that it is an interview, or does it just say two first names and no additional details? Are the words capitalized? If they attach their CV, that is extra points.

Now use a workflow. Do deep thinking, spend a bunch of tokens, and create this advisor for me.