Leadership Stories from Slack by Michael Lopp

Leadership Stories from Slack by Michael Lopp

You can’t simply learn a handful of rules that will make you a great leader. Everything depends on the context. The best way to build a real understanding about leadership - except for making the mistakes yourself - is to look at leadership stories.

The more context you get, the more you learn about the nuances of leadership.

This blog post brings you leadership stories from Michael Lopp, also known as Rands. He looks back on his time as VP of Product Engineering at Slack, and explains what he’d learned along the way, some mistakes he fixed and others he made.

Check out the full interview for more details and leadership stories on episode 1 of Level-up Engineering Stories hosted by Karolina Toth.

This post covers:

• Leadership takeaways from Slack
• The darkest day of Slack
• Tactical vs strategic focus as VP of Product Engineering
• Retaining your direct reports
• Creating processes before they're necessary
• The experience of Slack going public
• Leadership story about telling the truth

About Michael Lopp

Michael’s passion is engineering leadership. When he became an engineering manager at Netscape, he couldn’t find useful literature for engineers to develop leadership skills. This is why he started talking to other leaders. He wrote down the ideas he found and turned them into a blog.

Some people consider him famous or “valley famous,” but he thinks of himself as just a guy who writes about engineering leadership and has been recognized for it. He’s built a leadership Slack community over the past 5 years. He enjoys the collective learning aspect and helping others develop.

He has worked as VP of Product Engineering at Slack, Head of Engineering at Pinterest, and occupied engineering leadership positions at other notable companies.

Biggest leadership takeaway from Slack

I only really figured out how to be an executive about three years into the role, which was a year after I joined Slack as VP of Engineering. I was in an executive role at Pinterest for two years before, adding up to three years combined.

We humans tend to be hard on ourselves when we get into a new role. The roles are often significantly different, whether it’s going from junior to senior engineer, transitioning to management, or managing managers for the first time.

I’ve recently come up with this phrase: you’re aspirationally qualified.

This means you believe you can deal with the new role and you’re excited about it. That’s essential, but it doesn’t actually qualify you for the role. Realistically, new managers aren’t qualified to be managers when they’re promoted. This is just the way it is.

It took me three years to figure out my executive role, and it’s probably true for most roles. No one wants to hear this. When you’re starting in a role, your previous experiences are built around using a different set of skills, and it takes time to adapt to your new role.

I wish I knew this earlier. My role at my current company is different. I’m about a year and four months in, and I’ve got it about 50% figured out.

You also need to learn about the culture and the rules. The unspoken rules are especially important, because no one writes them down, but everyone expects you to follow them. You need to watch carefully and ask around to figure out the expectations and why they’re a part of the culture.

The story of Slack going down twice on the same day from a leadership POV

Arrival at Paradise

When I arrived at Slack, any engineer could push anything to production at any time. We were releasing Slack 100 times a day.

Slack was proud of the fact that whenever a customer complained about a bug, a person from customer service would go to engineering and have it fixed in a matter of hours. The customers were happy about bugs being fixed too.

This seemed like a dream.

The downside kicked in because Slack was in hypergrowth and went from 60 software engineers to 600.

Setting up staging areas, regression, pre-submission testing, chaos testing, monkey testing, and a long list of things took a lot of work. Unfortunately, we only learned this when we already needed them.

The darkest day of Slack

On the darkest day of Slack, we were down twice on the same day. I told the team, “If Slack’s down, the New York Times isn’t going to publish.” I’m sure the NYT was fine, I just wanted to get their attention. I needed to make the point that our service isn’t just nice to have, but it’s mission critical.

Many people are blocked at work when we’re down, so we needed to take this seriously. It was caused by too many releases, so we went from releasing 100 times a day to 3-6 daily rollouts. Saying it like that makes it sound easy, but it took two years to dig ourselves out of the technical debt in the infrastructure and all around.

The CTO at Slack, Cal Henderson, sent me a text that day saying this but with stronger wording, “We’re not going to release anything until we figure this out.” I said, “Got it,” and I gathered everyone in a conference room, and let them know, “We’re not going to release anything until we change this,” so a many-hours-long whiteboard remediation commenced.

It took us getting to a dire place to change the release process. We managed to fix it; it’s no mistake you don’t hear about Slack going down anymore. This happened years ago, and it made us go through a lot of hard work and learn lessons the hard way.

How much was your focus tactical vs strategic as VP of Product Engineering at Slack?

CTO vs VP of Engineering

Let’s start with the difference between a CTO and a VP of Engineering. In this scenario, the CTO, Cal, built the machine, and my job as VP of Engineering was to run the machine. This includes all the humans who work on Slack.

My specialty is the people side of the equation. Cal had written a lot of the code, and he knew the tech stack incredibly well. This meant that I got to focus on the process, the people, and the product in my day-to-day work.

Putting out fires often turns into strategy

My focus often shifted between strategy and tactics depending on where we were in the product cycle. At times, I focused on the tactical level to help push a big feature across the line. Other times I was working with the other VPs to plan out the next 6-12 months and to make decisions about the product, the people, and the technology.

Another important part of the job is dealing with the unexpected stuff. The things that land on your plate have usually passed through your ICs, their managers, their leaders, and everyone else to get to you. These things are often on fire, because if anyone else along the way could have handled these, they would have.

You spend about 25% of any day dealing with unexpected things that show up randomly. These often turn into strategies, as you start looking into what caused the fire. You may have a leadership issue on a team, or someone not doing their job because that turned into a disaster.

In this case, you tactically navigate through the problem, then turn it into a long-term strategy to stop the same from happening again.

A story of a random discussion turning into strategy

It often happens that people talk to you about one thing, but their real problem is something else entirely. This happened to me literally this morning. I can’t go into details, so I’ll keep it abstract.

A person tells me about one of their reports being cranky about topic X. When I hear that, I instantly get the impression it’s not the real problem, so I ask, “What about A, B, and C?” They tell me, “A is a thing, but B or C is not a problem.”

This tells me it’s all about A, which is completely different from X, which we started talking about originally. At this point, I need to figure out how to teach this person to recognize these situations and to focus on the important part. It’s part of the job.

It’s a real problem I need to spend hours dealing with out of the blue. The solution won’t be as obvious as writing a test. I need to teach this person a lesson, and make sure they learn it.

How do you keep your direct reports from leaving the company?

Give them a new challenge

In my experience, bored people quit. The moment the people I want to keep working with show the slightest hint of boredom, I change what they’re doing. I put something new, interesting, and big on their plate.

Your goal is to detect their boredom before they’re bored. When I’m in a one-on-one meeting with Craig, a senior lead for Team X, and he says something like, “When this is done, I’m not sure what I want to do next,” what I hear is, “I’m bored.” In my head, the clock is ticking because I expect he’s about three to six months away from quitting on me.

What can I do in this situation to improve retention?

I put him on a different team, and I change our one-on-ones from fortnightly to weekly, where we start discussing architectural or product strategy. The moment he says something like, “I’d love to prototype that thing,” I say, “Okay, let’s do that.” He can’t just throw everything away and do that, because he has a lot of things on his plate, so I need to make sure others take those over.

Delegating those tasks is a lot of work, and it’s difficult to notice that someone is bored before they do. The point is that I’m going to clear the way for Craig to get to the next thing that gets him excited. This will keep him learning and engaged, so he won’t be thinking about boredom and quitting.

Create a testing ground

Another common occurrence is an architect telling me they want to become an engineering manager. I used to say, “That’s a horrible idea.” I don’t do that anymore.

I created a tech lead manager role at Slack as a way of training engineering managers. They get to manage three people, we teach them the rules and provide them with training, but they’re still coding 50% of the time. They get to try before they buy.

If they love it, they can start to learn more, scale their engineering team, and become a full-time manager.

If it doesn’t go well, we start to scale the coding back up to 60% and so on, and if that looks better, we take the other direction. When that happens, the story will be that Craig got to try management and didn’t like it for some reason, not that he failed as a manager. Then you can give him another interesting project to make him an even better architect or a more senior tech lead.

Do you create processes that aren’t necessary right now but will be in the future?

In this situation, I try to incept those ideas.

If you hire me to a 10-person company, you probably know I’m good at dealing with hypergrowth. If I join the company, they will have engineering managers, and I will define what their roles, powers, and responsibilities are. It’s great to define this template early.

It saves everyone from a lot of debates you would have had otherwise. You won’t have to tell everyone that you don’t want to go with a holacracy, because everyone will know that you’ve already decided to go with managers. You also don’t have to debate what the managers’ responsibilities are, because you already have a template you just need to fine-tune as you go.

The hardest part of being a leader is that your success is the absence of drama. You do your job well when nothing is happening. It’s not satisfying because no one gives you a compliment for nothing blowing up.

Releasing Slack 100 times a day was a mistake. There were a million good reasons for us to get to that point, but we should have been thinking ahead and doing preventative maintenance to be able to scale. This decision should have been made two years prior to Slack going down, and it was a leadership responsibility.

What was your experience of Slack going public?

From my position, it wasn’t a big deal. Obviously, it was a big deal for Slack.

They did a good job by making sure one team drove this incredibly complex process to get the company in shape to become a publicly traded company. It only slightly affected my work, in so far as it was a minor distraction.

It was an amazing day in New York when we went public. We were all excited to see what was going to happen. It was an amazing experience to walk around Wall Street and see your company logo up on a flag.

The people who were working on making it happen would tell a different story. They would probably say, “This was my life for nine months and it was hell.”

Executives need to tell the truth - A leadership story from Slack

I suggest executives in growing companies tell the truth as quickly as possible.

It’s great advice, but this story is about how it backfires. Really, this was a failure on my part as a senior leader, and it flowed from this approach. If I get a question, I answer it. If you ask, “How are we doing on hiring women in engineering?” I’ll say, “Here’s the answer and here’s the data.”

At Slack, we had a lot of data come in about hiring in a big spreadsheet. It’s data like the number of people hired, diversity in recruiting, and so on.

My admin asked me, “Someone asked for this data. How are we going to share this information?” While literally running from one meeting to another, I answered, “Just send it to them.”

This was a terrible idea.

It was just a spreadsheet with numbers, including no personal data. What could go wrong?

Someone should have gone through that data to gather what we can learn from it and frame it right. When you give raw data to a person who doesn’t have context, they’re likely to read the worst possible story out of it.

I learned this lesson an hour later.

Someone came to me and told me, “We’re not doing X, because the spreadsheet says…” I was very surprised, so I started to put it in context, “That’s not what it says. You’re reading one row out of 27 rows, and…”

I use a phrase that came out of this situation: don’t YOLO communication. Senior leaders have an added burden about what they say and how they're saying it.

I still advise executives to tell the truth as quickly as possible. I’d also remind them that they need to be thoughtful about everything they say, and it needs to stand up to scrutiny. They need to explain well, especially when it comes to complex concepts like diversity, hiring, or engineering culture.

This may go against the part of doing it fast, but just do it as quickly as possible. Sometimes I just say, “That’s a great question. I need to gather the data and put it in context for you, and that’s going to take some time.” I need to be able to tell the story, explain the strategy, and say why we succeeded or failed clearly so it makes sense and doesn’t create confusion.

In this instance, I created confusion. Once we got everyone together and I explained what we could learn from the data and how they all go together, everyone realized it was not a disaster. I didn’t give them the framing.