Simply Broken

← Case Study Interviews: Mini Case Studies  /  all explorations

Read the first article before you read this one

How to Recruit Better with Case Studies explains how to design, run and read a case study inside the interview. This article continues from there and does not repeat it.

Part two / before the room, and after the slip

Recruiting mastery, case studies tell you the truth

A case study is not an hour in an interview room. It is anything you arrange so you can watch someone do the work. Some take an hour. Some take one minute.

You start by "selling the role", but you should also open and directly say that every role is not for everyone so the mutual job for you and the candidate is to make sure this will be a win win. A win for your company because you verified the candidate will do an awesome job, and a win for the candidate because you helped them really understand the role and verify they would love doing it.

01The job

Sell the role hard, then find out who it is for.

Every role is amazing or terrible, and the only thing that decides which is the person sitting in it. For the right person the job really is amazing. For someone else it is terrible, and no amount of talent changes that.

That makes the job description a sales page rather than an announcement. A hiring manager described the old habit after rewriting theirs:

I was thinking of it as, okay, I'm just informing people. But when you put it as, I'm selling it, then it needs a whole different approach.

Selling hard is not lying, because for the right person every word is true. It does give you a second job, since you have now made the role sound amazing to everyone. That second job has two halves:

  • Will this person be amazing at the role?
  • Will this role be amazing for them? Someone who is great at the work and hates it will leave, or do a poor job, or want to do something else.

Most hiring looks only at the first half. Both need testing. Every single thing can be tested, as long as you design the test and then pay attention to every signal it sends back.

02The example

Let's do 10 AM. Mind sending an invite?

Imagine you are introduced to a candidate and email them to connect regarding a customer success role and to set up the interview. They reply with three times, and one of them works. What do you write back?

What nearly everyone does

"Great, 10 AM works. I'll set it up."

Polite and efficient, and you have just done their homework. You also spent your own time and learned nothing.

What you write instead

"Let's do 10 AM. Mind sending an invite?"

The same length, and it saves you the work. It also starts a case study that runs by itself.

Part of a customer success job is getting people into meetings, so you just asked them to do a small piece of the job for someone who will read it closely. Here is what comes back from two different people.

The same thread, two candidates

The one you meet

5 minutes later

"Here are three times that work for me."Exact times, because that is how you set up a meeting without four more emails.

+1

You

"Let's do 10 AM. Mind sending an invite?"

Within the hour

The invite arrives.Fast, so the slot is held and nobody is double-booked.

+1

Invite structure

Clear title, right time, CV attached.Organized, on a small thing, without being asked.

+1

You

"Please apply on the site before we meet."

Soon after

The application is in, and you did not chase it.

+1

Four points, and you have not spoken yet. You go into the interview already interested.

The one you never meet

Slow

"When are you available?"The question alone is neutral. The delay is not.

−1

You

"How about 10 AM?"

Slow again

"OK."

−1

You

"Mind sending an invite?"

A day later

The invite arrives.

−1

Invite structure

Lowercase title, two first names, no details.

−1

Same invite

The time on it is wrong.

−1

Three was enough. You are done before you begin, so you cancel the interview.

Names and details in these examples are invented. Both threads are composites of real candidates.

Pay attention to the details of the invite, it's a mini-case study.

The invite gets two separate scores: did it come quickly, and is it good? Put two side by side and the second score needs no explaining.

The same 10 AM meeting

Scores a loss

noa / daniel

Tuesday, 3 PM

Scores a gain

Interview: Noa Levi / Daniel Bar, (Acme - Customer Success Lead)

Tuesday, 10 to 10:45 AM

Daniel Bar CV.pdf

  • 1Both names, and the word "interview", with capitals where capitals go.
  • 2The company, and what the meeting is about.
  • 3The time you agreed. People get this wrong often, and it is a big deal.
  • 4The CV, unasked. Extra points.

If they mess it up now, when it's super important and visible, they're absolutely going to mess it up later.

A couple of hours is the line.

Speed matters because of what happens in real life. Someone who leaves the invite for later does not block the time, and then they are double-booked. Past a couple of hours you are not looking at a busy person. You are looking at someone who does not close small loops when they come up.

A booking link beats three good times.

Google Calendar can give you a link that lets the other person book the meeting. A candidate who sends one has removed the back-and-forth entirely, which is exactly the instinct you are hiring for.

03The score

Gain points, stay neutral, or lose points.

Every small signal lands in one of three places. Most of the skill is refusing to treat them as the same thing.

  • Gain. A reply in five minutes with three exact times, because that is the job.
  • Neutral. A quick reply asking when you are available. Not impressive, and not yet bad. You note it.
  • Lose. Forgetting to respond, or being messy about it. If that is the whole picture, you do not need to meet them.

Then you count. Fail three of these small tests and it is over. One slip is not a pattern, so how seriously you take a loss depends on how big the slip was and how many came with it.

If someone doesn't know how to send out good invites on time, quickly, you don't want to start there. It's too low of a starting point.

These look like small tests, and they are high value, because they measure the things you least want to deal with after the person joins. You can teach someone your product. You do not want to spend your first month teaching them to close loops.

04The buckets

Three buckets, named so the name does the work.

Before you can test anything you have to decide what you are testing for. That is the performance review you would hold this person to a year from now, reduced to three buckets. One manager came in with six, moved to three, and still got the three wrong the first time. The change that fixed them was in the names.

The names matter more than they look, because they are what people end up remembering. There is one test, and it is worth more than keeping them short:

If people only read the names, how close will they be to inferring what we believe?

Here is one role, a customer success manager, before and after that test. The role is only the example. The test works on any role you hire for.

Amazing professional

Reliable, productive professional

Gets the job done without you asking twice, and goes looking for tools that make them faster.

Commercial instinct

Trusted advisor with commercial focus

Understands the numbers and the client's goals, and tells the client what they need to hear.

Client connection and communication

Deep personal care and relationships

Cares about people's lives, their challenges and their hopes, whether or not any of it touches the business.

You often know the wrong name first.

Many times you know something is the wrong answer before you know what the right answer is. "Amazing" said nothing about what was inside the bucket, and that was enough to reject it with no replacement in hand. Keep rewriting until you read the name and can see the person. When a name is right it feels right, and that feeling has a lot of depth behind it.

Decide what the role does not want.

An earlier draft had "fast learner and promoter of change" as its own bucket. For this role that was wrong. The role needed someone open to change who follows the process, rather than someone who arrives wanting to redo everything. A bucket copied from a generic list of virtues will test for the wrong person.

Set a separate bar on each bucket.

You will rarely find a ten in everything. So decide the level you need in each bucket separately, before you meet anyone. The bar comes from who this person will serve, so the same bucket gets a different bar in a different role. A test is only useful when you know the score that passes it.

05The tests

Write the tests before you write the ratings.

How long a case study takes

1231 min3 to 15 min30 to 60 min
  • 1One sentence to write, one minute to run."Did they send a good, organized invite quickly?" That is two tests and two scores.
  • 2A mini case study.Tests one skill in three, five or fifteen minutes. Writing it might take half a page.
  • 3A full case study.Half an hour or an hour, as in the first article.

Some tests you will not know how to write yet. Name the skill in the bucket, leave the gap visible, and solve it later. That is what the advisor at the end of this article is for.

For the reliable, productive professional you already have four tests before anyone has met: how they answer the early emails, whether they offer times, the invite, and the application. Then comes the test that separates people above a 3.

The test, said out loud

"Show me how you use AI to make yourself more productive."

Negative points

"I go on ChatGPT and use it like Google, to search for information."

Or nothing at all. If someone has nothing to show you, that tells you a lot about the bucket you named "productive".

What you are listening for

"Renewing my car license took 20 minutes of clicking on a government website. Now it's three chat messages with an agent."

Yes, that only saves 15 minutes. But there are 50 of those annoying renewals to do, and they know it.

You are not checking for a clever trick. You want to see that they push toward it by themselves.

06The reveal

When someone fails, reveal the test, see how they react

Say someone comes highly recommended, so you are not ready to rule them out. They took 24 hours and sent a poor invite. What do you do?

A good answer is to note it, then design another test to check whether it is a deeper problem or a one-off. That works, and it puts a lot of effort on you. The great answer is one email.

The email

"It took you 24 hours, and you sent me a very poor-looking invite. Please organize the invite so it looks highly professional. I'd also love to understand why that happened."

Tell them afterwards, plainly, that this was a test. You explain and ask, it costs you almost nothing, and it is the most revealing thing in the whole process.

Three ways it can go, and only one continues

1The reveal2Fights it3Excuses4Owns it, fixes it
  • 1The reveal.You name what happened, ask for the fix and ask why.
  • 2"Why are you so picky?"Great. You are picky, and they are not hired. That one is easy.
  • 3The dog ate my homework.Reasons about the kids, the week, the calendar. You do not care about excuses, and this is what they will bring you later.
  • 4"I'm sorry about that. Sending you a really organized invite now."Full ownership, and the fix in two to four hours. If it takes another 24, that is another knock.

The fight is the answer that should end it, because a candidate who argues here is arguing about the easiest possible case, imagine what they will fight about once inside the company.

"I didn't know what good looks like" is no defense.

Anyone can paste the email and the invite into ChatGPT and say "help me make this awesome." It will tell them what to add: a real title, an agenda, the CV. So the fix is available to everyone, and the second invite tells you whether they went and got it.

This is different from teaching in the case.

The first article has you teach when a candidate pauses, then re-test whether they can use what you handed them. That measures learning. The reveal measures something else: what happens when you tell this person they got it wrong. You will be doing that for as long as they work for you.

07The full answer

What a great answer to fifty people sounds like.

The first article gives you the setup: fifty people arrive tomorrow, and nothing is ready. It follows the candidate as far as the furniture shop. Here is the rest of a great answer, from the first sentence to the last table, with what each move shows you.

  1. "The moment you say it, I do a mini kickoff with you. What do you know, what is available, and what is missing?"They find out which problem they are solving before they solve it.
  2. "Who needs to be involved, and what budget do I have?"People and money, early, without being prompted.
  3. "Right after this call I phone every one of those people. It's tomorrow, so there is no time to set up a meeting."Their tempo matches the deadline.
  4. "I start a list of everything I need, maybe with AI: tables, chairs, refreshments, access to the building."Capture first, and a tool used by reflex.
  5. "While I start executing, I send the list to you and the other three people I spoke with, so I hear what I missed as quickly as possible."They involve others without stopping to wait for them.
  6. "Then I look for the longest and most difficult item. Tables. Nobody has a solution, and we can't ship them from headquarters. So I solve tables first."The hardest thing goes first.
  7. "I can go buy folding tables nearby for $200. I email you: this is the solution I came up with, do you have a better one?"They bring you a decision and leave you room to overrule it.
  8. "Once the hardest ones are solved, I go on to the medium ones and the easier ones."

Your candidates will not say all of this. Having it written out means you can hear how far along it they get, and which moves they never make.

08Your bar

The tests can only be as good as your standard.

A players hire A players. B players hire C players. You would expect C players to hire terribly, and they do not. C players hire randomly. So the worst person to put in charge of hiring is a B player, who reliably hires below themselves.

What moves someone up is mindset, and it is separate from capability. You can be an A in mindset and still a B in capability, and you are fine as long as you are climbing. The tell is what you consider normal. A B player thinks of a completed checklist as something to manage. An A player says: of course this is what I expect, and there is no reason anyone should not do it.

Designing these tests is how that standard gets out of your head. Each test says, in writing, what you consider normal. What you test for at the door is what you will not have to fix later.

09The mistakes

Four ways to waste a good test.

1You send the invite yourself.

You offered to set it up, out of good manners, and the test never ran. Any small task you take off a candidate's plate is a case study you just cancelled. You will usually notice right after you hit send.

2You note it, and then you meet them anyway.

Noticing is only worth something if it can end things. Three small losses and a calendar slot still on your schedule means you scored the test and ignored the score. Rule them out. It can be very quick.

3You build a bigger test when one email would do.

A loss makes you want more evidence, so you go and design a whole new exercise. Reveal the one you already ran. You get the fix, the reason and the reaction, and all it costs you is two lines.

4You ask AI before you understand it yourself.

If you do not really get it yourself, you cannot guide the AI in the right direction. Getting it means you can say: if I do X, and I look for this and this and this, I will learn something real. Write your first tests by hand. They will be rough, and you will know why.

10The advisor

Then build yourself a case study advisor.

Writing tests is the hard part, and it stays hard. Once you have written a few by hand, this prompt turns Claude into something you can ask every time you are stuck. Paste it in and let it run. It does a lot of research, and it can take about half an hour. Save what comes back as a skill. From then on you ask it: "How would I design a case study for this?"

The prompt
I want your help designing case studies and mini case studies.

Please run a workflow to bring in best practices and approaches from McKinsey and other great case study companies, and create a case study advisor for me.

Case studies, for me, are ones that test a specific skill in a candidate I am recruiting. A mini case study also tests a skill, but in a much faster way: maybe 3 minutes, or 5 minutes, or 15 minutes, where a full case study usually takes half an hour or an hour.

Please bring in all the best practices. Here is an example of a case study.

If I am hiring a project manager, a case study could be: "Imagine you are hired and on the job tomorrow. In the morning I tell you that we had a mess-up. Tomorrow morning we have 50 people coming for a full day of sessions and classes. We have the lectures, but we are completely unorganized on where exactly and how it will happen. What do you do?"

The answer shows the way they think and how they capture everything, from figuring out what they need to solve to actually doing the solution. A great answer would be: "The moment you said it, I do a mini kickoff with you. What do you know, what is available, and what is missing? Who needs to be involved, and what budget do I have? Right after this call, because there is no time to set up a meeting, I call every one of those people on the phone and get as much information as I can. Then I understand roughly where the location will be, and I start creating, maybe with AI, a list of everything I need: tables, chairs, refreshments, access to the building. While I start executing, I send the list to you and the other three people I talked with, to see if I missed anything, so I find out as quickly as possible. Then I go down the list and look for the longest and most difficult thing to execute. Tables: no one has a solution, and we can't ship them from headquarters. So I solve tables first. One option is to go buy folding tables nearby. They cost $200. I email you: this is the solution I came up with, do you have a better one? Once I solve the hardest ones, I go on to the medium ones and the easier ones." Asking questions to clarify is really important, so they know which problem they are solving, and so is involving others.

Here is a mini case study for a customer success person. Before the interview, I ask them to send me the invite for the time we scheduled. I want to see how quickly they send it and how well structured the invite is. Does the title include my name, their name, the company and the fact that it is an interview, or does it just say two first names and no additional details? Are the words capitalized? If they attach their CV, that is extra points.

Now use a workflow. Do deep thinking, spend a bunch of tokens, and create this advisor for me.