I just got back from a week-long restorative holiday, where I found some time to go back digging into my Kindle library.
Based on Faraz’s suggestion, this time I picked up Working Backwards, a book that revolves around Amazon’s culture, its unique approach to solving organizational challenges and its pragmatic (albeit innovative) approaches to product development.
Little did I know this would be a great read, even though the style of the book is definitely unconventional.
The book is really strange, in the sense that there doesn’t really seem to be an agenda and is instead a bunch of chapters about what the authors think created great competitive advantage for Amazon, loosely tied together. To be honest, when you have to handpick a bunch of winning strategies for a company, it’s probably hard to write a unidirectional story and instead should focus on a bunch of anectodes (or mini-stories), which is what the authors do here, and do extremely well.
It was incredibly insightful — for me, working at a high-growth, large-scale company in a similar space, it provided a lot of key information about problems we face while going through the same exact journey (even though amazon’s problems are at planet-scale, while we mostly limit to our region).
I really enjoyed reading about the single-threaded leadership model (re-inforced my thoughts), as well as the journey from a XML product feed to AWS (it’s brief, so don’t expect a lot around AWS in this book). I also liked the practical examples around 6-pagers as well as PR/FAQ documents: while I do believe too much narrative can slow things down, I also recognize how important is to think things through, and the fact that too many times I hit the ground running without a coherent and cohesive plan. Another key takeway for me is focusing on input, as opposed to output, metrics.
One anectode that got me cracking is the first in-person feedback session on the earlier amazon APIs (aka XML product feeds), where someone who would then go on to have a brilliant career at AWS showed up:
Since we did not have much experience creating programs for software developers, we sought in-person feedback from heavy users of the service. We decided to host an Amazon software developer conference in our Seattle headquarters. The first one attracted a grand total of eight people. We flew two of them in from Europe. I discovered, just a week before the conference, that one of the European attendees was a teenager. I had to check with our legal department if that was okay—fortunately we didn’t need permission from his parents, and he was able to join us at the conference. We worked out the logistics and set up a full day of sessions. Tim O’Reilly and Rael Dornfest from the O’Reilly Media, who were both avid supporters of the web services movement and taught us a lot about this new field, were there too. Another attendee was an avid customer who happened to live in Seattle. His name was Jeff Barr. He commented: The attendees were outnumbered by the Amazon employees. We sat and listened as the speakers talked about their plans to build on their success and to expand their web service offering over time. One speaker (it may have been Colin Bryar but I am not sure) looked to the future and said that they would be looking around the company for other services to expose in the future. This was the proverbial light-bulb moment for me! It was obvious that they were thinking about developers, platforms, and APIs and I wanted to be a part of it. Jeff Barr joined Amazon a few weeks later and is still with the company, serving as VP and chief evangelist for AWS.
Some other interesting quotes from the book:
bias for separable teams run by leaders with a singular focus that optimizes for speed of delivery and innovation
When I write about what led to Jeff making key decisions in this book, I can do so because I often directly asked him for his specific thinking behind his insights, as the reasoning behind them was often more illuminating than the insights themselves.
Before we start building, we write a Press Release to clearly define how the new idea or product will benefit customers, and we create a list of Frequently Asked Questions to resolve the tough issues up front. We carefully and critically study and modify each of these documents until we’re satisfied before we move on to the next step.
The meeting begins with everyone reading all the interview feedback. Afterward, the Bar Raiser may kick off the meeting by asking the group, “Now that everyone has had a chance to read all the feedback, would anyone like to change their vote?”
At many companies, the hiring manager has the recruiter make the offer. This is another mistake. The hiring manager should personally make the offer and sell him/her on the role and company. You may have chosen the candidate, but that doesn’t mean the candidate has chosen you. You must assume that good employees are being actively pursued by other companies, including their current employer. There is always the risk that you could lose the candidate. Nothing is certain until the day they report to the office.
employees should be able to say to themselves, “I’m glad I joined when I did. If I interviewed for a job today, I’m not sure I’d be hired!”
The best way to fail at inventing something is by making it somebody’s part-time job.
During this phase, we became aware of another, less positive trend: our explosive growth was slowing down our pace of innovation. We were spending more time coordinating and less time building. More features meant more software, written and supported by more software engineers, so both the code base and the technical staff grew continuously. Software engineers were once free to modify any section of the entire code base to independently develop, test, and immediately deploy any new features to the website. But as the number of software engineers grew, their work overlapped and intertwined until it was often difficult for teams to complete their work independently.
At last we realized that all this cross-team communication didn’t really need refinement at all—it needed elimination. Where was it written in stone that every project had to involve so many separate entities? It wasn’t just that we had had the wrong solution in mind; rather, we’d been trying to solve the wrong problem altogether. We didn’t yet have the new solution, but we finally grasped the true identity of our problem: the ever-expanding cost of coordination among teams. This change in our thinking was of course nudged along by Jeff. In my tenure at Amazon I heard him say many times that if we wanted Amazon to be a place where builders can build, we needed to eliminate communication, not encourage it.
The leader must have deep technical expertise, know how to hire world-class software engineers and product managers, and possess excellent business judgment.
While each two-pizza team crafted its own product vision and development roadmap, unavoidable dependencies could arise in the form of cross-functional projects or top-down initiatives that spanned multiple teams. For example, a two-pizza team working on picking algorithms for the fulfillment centers might also be called upon to add support for robotics being implemented to move products around the warehouse. We found it helpful to think of such cross-functional projects as a kind of tax, a payment one team had to make in support of the overall forward progress of the company. We tried to minimize such intrusions but could not avoid them altogether. Some teams, through no fault of their own, found themselves in a higher tax bracket than others. The Order Pipeline and Payments teams, for example, had to be involved in almost every new initiative, even though it wasn’t in their original charters.
The original idea was to create a large number of small teams, each under a solid, multidisciplined, frontline manager and arranged collectively into a traditional, hierarchical org chart. The manager would be comfortable mentoring and diving deep in areas ranging from technical challenges to financial modeling and business performance. Although we did identify a few such brilliant managers, they turned out to be notoriously difficult to find in sufficient numbers, even at Amazon. This greatly limited the number of two-pizza teams we could effectively deploy, unless we relaxed the constraint of forcing teams to have direct-line reporting to such rare leaders. We found instead that two-pizza teams could also operate successfully in a matrix organization model, where each team member would have a solid-line reporting relationship to a functional manager who matched their job description—for example, director of software development or director of product management—and a dotted-line reporting relationship to their two-pizza manager. This meant that individual two-pizza team managers could lead successfully even without expertise in every single discipline required on their team. This functional matrix ultimately became the most common structure, though each two-pizza team still devised its own strategies for choosing and prioritizing its projects.
Jeff has an uncanny ability to read a narrative and consistently arrive at insights that no one else did, even though we were all reading the same narrative. After one meeting, I asked him how he was able to do that. He responded with a simple and useful tip that I have not forgotten: he assumes each sentence he reads is wrong until he can prove otherwise. He’s challenging the content of the sentence, not the motive of the writer. Jeff, by the way, was usually among the last to finish reading.
Leadership and management are often about deciding what not to do rather than what to do. Bringing clarity to why you aren’t doing something is often as important as having clarity about what you are doing.
Input metrics measure things that, done right, bring about the desired results in your output metrics.
We soon saw that an increase in the number of detail pages, while seeming to improve selection, did not produce a rise in sales, the output metric. Analysis showed that the teams, while chasing an increase in the number of items, had sometimes purchased products that were not in high demand. This activity did cause a bump in a different output metric—the cost of holding inventory—and the low-demand items took up valuable space in fulfillment centers that should have been reserved for items that were in high demand. When we realized that the teams had chosen the wrong input metric—which was revealed via the WBR process—we changed the metric to reflect consumer demand instead. Over multiple WBR meetings, we asked ourselves, “If we work to change this selection metric, as currently defined, will it result in the desired output?” As we gathered more data and observed the business, this particular selection metric evolved over time from number of detail pages, which we refined to number of detail page views (you don’t get credit for a new detail page if customers don’t view it), which then became the percentage of detail page views where the products were in stock (you don’t get credit if you add items but can’t keep them in stock), which was ultimately finalized as the percentage of detail page views where the products were in stock and immediately ready for two-day shipping, which ended up being called Fast Track In Stock.
Charlie Bell, an SVP in AWS and a great operational guru at Amazon, put it aptly when he said, “When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero, because it turns out that behind every issue there’s a very interesting story.”
The deck is usually owned by someone in finance. Or more accurately, the data in the deck are certified as accurate by finance.
People like talking about their area, especially when they’re delivering as expected, and even more so when they exceed expectations, but WBR time is precious. If things are operating normally, say “Nothing to see here” and move along. The goal of the meeting is to discuss exceptions and what is being done about them. The status quo needs no elaboration.
He said, “Amazon has a decent chance of being the last place to buy CDs. The business will be high-margin but small. You’ll be able to charge a premium for CDs, since they’ll be hard to find.” Jeff did not take the bait. We were their guests and the rest of the meeting was uneventful. But we all knew that being the exclusive seller of antique CDs did not sound like an appealing business model. While it is tempting to suggest the meeting impacted Jeff’s thinking, only Jeff can speak to that. What we can say is what Jeff did and did not do afterward. What he didn’t do (and what many companies would have done) is to kick off an all-hands-on-deck project to combat this competitive threat, issue a press release claiming how Amazon’s new service would win the day, and race to build a copycat digital music service. Instead, Jeff took time to process what he learned from the meeting and formed a plan. A few months later, he appointed a single-threaded leader—Steve Kessel—to run Digital, who would report directly to him so that they could work together to formulate a vision and a plan for digital media. In other words, his first action was not a “what” decision, it was a “who” and “how” decision.
We applied the new two-pizza structure to every part of the org chart below Steve and his direct reports. The two-pizza structure became more complicated at the top of the org chart. For example, should product, engineering, and business functions all report to a single leader? Or should each one be run by its own leader, with those leaders in turn working as a team on the product, engineering, and business details? We decided that there would be separate leaders for business and tech for each digital product category—books, music, and video. Each of these category leaders would hire leaders for each business function, such as product management, marketing/merchandising, and vendor/content management (licensing digital content from publishers, studios, and record companies). Each general manager (GM) category leader had a corresponding peer leader on the engineering side. Each engineering category had a two-pizza team for each major component of the software services (e.g., content ingestion and transformation) and for client application software. This was mostly a pragmatic decision based on the skills of the leaders. For example, I had no experience at that time managing an engineering organization. The same was true of my peers on the engineering side with respect to business. This would change in the years to come.
Customer expectations are not static. They rise over time, which means you cannot rest on your laurels.
Jeff would say something like this to a leader who had just laid an egg: “Why would I fire you now? I just made a million-dollar investment in you. Now you have an obligation to make that investment pay off. Figure out and clearly document where you went wrong. Share what you have learned with other leaders throughout the company. Be sure you don’t make the same mistake again, and help others avoid making it the first time.”
Definitely recommended: there are some overlooked problems embedded in amazon’s ruthless culture, but this book does a great job at explaining some of the things that worked really well.