Agile 101

Hi I’m Coach Takeshi. Welcome to my agile 101 session. In this not so long video, we will go through a remarkably wide variety of complex topics, each brain twisting, perspective shifting contents on its own.

Yet what I want you to experience is that, despite this sheer complexity, you will find simplicity and even elegance in it. Nature is full of examples like this – things that work, however complex, are beautifully simple.

I hope you experience the same with agile.

Okay, now let’s get started with, what is agile.

Let’s start with clarifying the taxonomy. Agile, Lean, Scrum, Sprint, Design Thinking – we all hear these terms and they seem to relate to, or compete with agile, quite confusing right?

First, agile and lean are technically different things emerging from different origins, but they share a lot of commonalities so just consider them as same. They are both experimental and iterative approaches for highly uncertain challenges.

And then we have Lean, which we have the good’ol Lean TQM or Lean Manufacturing, made famous by the car manufacturer Toyota, and the more recent Lean Startup phenomena. But for now, just consider them both in the large bucket of Lean.

Then we have Scrum and Sprint, probably terminologies you hear a lot and wondering what their relationship is with Agile. Well, Scrum is one of the most popular implementation frameworks in Agile. And Sprint is a Scrum terminology. It’s what the iteration cycle in Scrum is called.

Now as a matter of practicality, I’d like us to loop Design Thinking in the agile bucket. I use Design Thinking a lot, often as a synthesis with Scrum and Lean Startup. They all share the same spirit of experimentation and iteration, and it just makes sense.
So, to summarize, consider Lean and Agile as same, and Scrum and Sprint as part of Agile. And Design Thinking as an element of Agile.

Clear? Now there are more related frameworks like Extreme Programing and Kanban and the many scaled Agile and Scrum of Scrum frameworks like SAFe, LeSS, Nexus and even Prince2, but that’s too much for now, so we’ll talk about it on another occasion.

So now that we’ve got our glossary clarified, let’s take a step back and address the often-heard skepticism on Agile. Is Agile a fad? It’s a really valid question.

See, Agile did not invent trial and error. In fact, you’ve probably heard things like PDCA, plan-do-check-act, also known as the Deming’s circle used in Lean, and the OODA loop, observe-orient-decide-act, an iterative decision-making framework developed by a USAF colonel back in 1976. These loopy way of thinkings are not new notions, in fact PDCA dates back to 70 years ago.

And then when we think of our philosophical fathers like Socrates and Confucius whom their writings are essentially all about learning from trial and error, the teaching hasn’t changed.

It’s all about experimentation and iteration.

And that begs the question, why after two and a half millennia of trial and error teaching, we’re still talking about it?
See, it’s because by default, we humans don’t like trial and error. It’s coded in our biology – we hate change and uncertainty. It’s our survival instincts to resist change and avoid uncertainty. It’s that strong. But at the same time, it’s also universal knowledge and wisdom that risk taking is rewarding. Risk taking by trial and error is simply the best strategy out there for evolution. This battle between two opposing forces, resistance to change versus trial and error, we’ve been doing that for a few thousand years and probably longer, and it continues till today.

Agile is simply just one such latest attempt to get trial and error, experimentation and iteration right. It doesn’t matter what it’s called, the spirit is same.

So, PDCA, OODA, Lean Startup, Design Thinking and Scrum, they all share a loopy, iterative characteristics. This contrasts to waterfall, the linear, deterministic ways of doing things.

All of these Agile frameworks are different variations of the same thing: experimentation and iteration. And they share the common objective of addressing shortcomings of what we know as waterfall.

Let’s take a more detailed look at this in our next section, Why Agile?

Waterfall is a linear, sequential form of project management where a predictive big plan is made upfront, and phase-gate executed all the way to the end.

Now don’t get me wrong, I’m not saying waterfall is bad. In fact, waterfall is good. We owe modernity to waterfall. Without waterfall, we wouldn’t have high-rises, bridges, subways, safe air travel, our favourite packs of Danone yogurt and Kellogg cornflakes on supermarket shelves, not even fast food chains like McDonald’s and convenience store franchises like 7-11.
It’s a clean, transparent form of project planning and execution that produces stable results in mass quantity.

The challenge here is that, waterfall is not the best approach to what we call complex adaptive challenges – projects and endeavors of uncertain and ambiguous nature.

Pretty much every new product development today, is a complex adaptive challenge. Thanks to globalization, industry 4.0 and the service economy, consumers are offered more choices today. And thanks to social media and the internet, consumers tastes and preferences shift and diversify at unbelievable paces.

Simply put, it’s hard to predict what people want, nowadays. So, if we develop a new product or service with a hard assumption upfront, there’s a good chance that we may miss product market fit by the time we get the product out to the market. In waterfall, we can only find out if our assumption was correct at the very end. It’s an expensive way to fail.

Hence we should reserve waterfall only for projects of certainty. And for projects of uncertainty, better do it in iterations; i.e. agile.

For example, in Lean Startup, we build-measure-learn in MVPs. We come up with a product hypothesis, build a minimum viable product to test the hypothesis, and learn if there’s product market fit.

If there is evidence of product market fit, great, we continue the development, i.e. persevere, if not, we pivot to another idea, and most typically, we learn good things and bad things about our MVP, so we keep the parts that work and improve on the shortcomings in the next iteration, i.e. tweak. So persevere, pivot or tweak, repeat.

It’s a much faster and cheaper way to fail. And eventually, as you circle into product market fit from the learnings, your probability of success goes exponentially higher.

Agile is a safer way for developing complex products and services.

BTW, note that I highlight “product development” under the Agile header. Agile is not used for “managing” processes, it’s used for “developing” solutions.

It’s not easy to adopt Agile product development though.

Most people, particularly those who are used to waterfall project management will find the ambivalent nature of agile product development very uncomfortable. Like very.

In waterfall, there’s a clean plan and a target output, and as long as there is quality execution, there’s the comfort and confidence of building an end product exactly as planned.

Meanwhile, with Agile, we actually don’t know what exactly we’re going to build. We sort of know it and we do have an idea of a target outcome and it’s somewhere around here, but we don’t know exactly what’s it’s going to look like. This can be a tough sell to stakeholders. Convincing funding for a product development project with a vague idea of what’s being built, can be challenging.

And then this: first step we’re clear on what we want to do, second step we have a general idea of what to do, but thereafter… This, takes a serious leap of faith.

The cure to this is to find the right balance of freedom within a framework. Later on I’ll use the example of Scrum, and demonstrate that Agile actually is quite disciplined when it comes to implementation.

Now let’s look at the organizational aspects of why agile.

In a traditional vertical, hierarchical organization, strategic decisions are made by leaders, and then handed down to middle managers sitting in functional silos.

The middle managers are then responsible for making tactical decisions within their silo, and further passing on instructions to the rank and file.

The rank and file are typically evaluated individually on the quality of execution against the instructions handed down.
Pure vertical, hierarchical organizations function very well in environments of certainty, but increasingly less so in today’s complex world.

What if the leader makes the wrong strategic decision, and the managers give wrong instructions on how to do things?
In response, in agile organizations, the teams who are closest to the customer, market, production facility etc., are empowered to make decisions on their own. It’s a concept from Lean called “Gemba”; the idea is that frontline people have the best information, so why not let them make the decisions? Makes sense, right?

And from there, we follow the natural order of human organizing – if you trust in people’s ability to self-organize, and with a little facilitation help from leadership, quite remarkably people get together to form highly functioning teams.

And what about complexity handling beyond the individual level? That’s the ingenuity of organizations as systems – people naturally start reaching out to each other to compliment their skills; i.e. organic emergence of cross-functional teams.

Now, why is this so important? It’s because we are now in the age of the networked knowledge worker.

We are going through a global service economy transformation, and in a service economy, the competitive advantage is knowledge and connectivity.

And at the individual professional level, like you and me, our professional development is naturally gravitating to these two attributes – learning and developing new skills and expertise, and increasing our access to a wide variety of outside resources that will aid to our capabilities.

Meanwhile, at the organizational level, leaders are facing the reality of innovate or die from becoming obsolete. This is how fast our world is evolving.

Gone are the days of a single master mind leader steering the entire organization. It’s way too complex now – survival of the organization can’t do without tapping into the collective intelligence and drive of today’s networked knowledge worker.

This is what leadership is about today. The new organization needs to follow the amorphous nature of the knowledge workers’ network. It’s time to evolve from the traditional vertical, hierarchical organization, to a more cross-functional, networked organization.

There’s one more soft reason, people reason, why I believe agile is important.

So I’m also a behavioral coach, and my training was in positive psychology. And in positive psychology, we make heavy use of motivation theory.

This is a visualized interpretation of Maslow’s hierarchy of needs. According to Maslow, there are two higher order intrinsic motivations that propel us to do extraordinary things. The first is from our strong desire of personal mastery, which leads to self-actualization.

We take pride in our work, and without anyone telling us, we strive to be the best at what we do. And even after we receive the recognition as master of one’s trade, we still keep on striving. This is self-actualization, and it’s so fulfilling.

On top of self-actualization, is self-transcendence. This is about going over and beyond for the collective good. We feel great when we contribute to others, to the team, organization, society, humanity and, beyond. Our strong sense of belonging is where self-transcendence stems from, and elevates.

An agile organization is fertile bed for culturing people’s desire to self-actualize and self-transcend. In agile organizations, we respect people’s ability to work autonomously, and to self-organize.

And why does this work? Because, trust, is at the heart of agile. We know we don’t have to use extrinsic incentives such as carrots and sticks to motivate people. Our people will naturally flourish, when we believe in them.

Nonetheless, enterprises still have to make money. So let’s look at the economic case of agile.

Let’s take this example of a hypothetical high risk, high return project. The probability of success is low at only 20%, but expected return on investment is high at 20x. Actually, it’s not a bad investment opportunity. If you look at the weighted average ROI, it’s $100 times 20% times 20, which comes out to $400, a quadruple expected return. This is actually not an uncommon risk-return profile; startup projects are like this and venture capital firms invest in these kind of projects day-in-day-out.

Now, the $400 ROI is based on the assumption that investment is made in one go. Let’s look at the scenario where we invest in an incremental, agile way. So we invest in five goes, $20 each, and every iteration we see continuous improvement that contributes to an increased probability of success, say at a rate of 20%.

These are hard assumptions but not necessarily untrue from reality – as a startup, if we spend money wisely in increments to build-measure-learn with MVPs; every iteration, we are likely going to get closer to product market fit.

So if we tally up the improving weighted average ROI from each iteration, we end up in this example with an ROI of $1200. That’s three times the ROI compared to investing in one-go, waterfall like.

And the good news is, it doesn’t stop there. As by the time of the last iteration, on the assumption that we’ve nailed product market fit, we get to keep on milking the cow until the market shifts. So, agile de-risks, and agile makes money. Cool, right?

Okay, time to give you an example of how to do Agile. Let’s take a look at Scrum, the most popular implementation framework in Agile.

Scrum started as a software development framework, but today it’s widely applied in business, operations, etcetera.

Scrum is a time-constraint development framework, where work is delivered in short, one to four weeks, typically two-week Sprints.

Scrum is team based; in a Scrum Team you have 3 to 9 Development Team members, and two servant leaders, the Product Owner as the “what” guy, and the Scrum Master as the “how” guy.

Very importantly, the Product Owner and Scrum Master are not managers nor bosses. There’s no hierarchy in a Scrum Team, and everyone self-organizes.

Scrum is a lightweight framework for developing complex products.

The rules, roles, values, pillars, artifacts and events are defined in a 19 page Scrum Guide published on the web.

Scrum is pretty robust and disciplined, but it is not a fixed process, prescriptive step-by-step methodology. The Scrum framework makes use of various methods, tools and practices, but again, Scrum itself is not a methodology.

Across the years, many useful tools, methods and techniques that compliment the Scrum framework have emerged from the Scrum community.

For building the Product Backlog, which is the master list of everything that’s needed in the product, and the Sprint Backlog, which is the set of Product Backlog items that are going to be developed in a given Sprint, a common practice is to break down the Product Backlog items into a collection of Epics, which is a collection of Stories, which in turn is a collection of Tasks.

It’s a pretty nifty way to breakdown and manage complex blocks of work.

Each Sprint begins with Sprint Planning. Prior to Sprint Planning, the Product Backlog needs to be refined, complete with prioritization and estimates for each Product Backlog item.

Using User Stories – As a [user], I want [whatever feature or function] so that [whatever value is realized] – is a good way to visualize the value of a Product Backlog item.

In Sprint Planning, the Development Team agrees on a Sprint Goal, pulls work from the Product Backlog into the Sprint Backlog, together with a plan for delivering the product increment.

The Sprint Backlog is commonly visualized in the form of a Scrum Board, also known as a Kanban Board, a concept originating from Lean.

The Scrum Board is not an artifact mentioned in the Scrum Guide, but is now ubiquitously seen in all Scrums. It’s a simple board consisting of to do, work in progress and done boxes, but it’s a powerful tool that provides persistent visualized sharing of information to the Scrum Team and stakeholders.

It’s also a good tool for observing the practice of Definition of Done on Product Backlog items, which is a requirement by the Scrum Guide.

Here’s a variation of a Scrum Board. Often during a Sprint, Development Team members are approached by a stakeholder with a “priority request.” These non-planned items are a major source of disruption in a Sprint.

Creating a triage box in the Scrum Board solves the problem. When a priority request happens, the Development Team member who receives the request can visually put it into the triage box, and tell the stakeholder “We’ll let the Product Owner know and get back to you what we can do.” It’s a fair process that the stakeholder can’t really say no. Thereafter, the Development Team can discuss with the Product Owner and decide if the request can be accommodated, and if there’s no capacity, a judgement call has to be made whether or not to bump out other tasks.

The Daily Scrum, more commonly known as the daily stand-up, is a Scrum Guide required event and cannot be skipped.

It is officially time-boxed to 15 minutes, and needs to happen at the same time and place everyday. To keep it from “drifting,” for example persistently starting a few minutes late, I often suggest to the Scrum teams to set an odd time to start, like 8:43am, finishing at 8:58am. It registers on everyone’s mind, and keeps the team committed.

A typical format of a stand-up is to go around and share what work was done yesterday, what work is planned today, and if there’s any impediments. A common misconception is that the Scrum Master needs to facilitate the stand-up. Even worse, some teams do stand-ups as a status update to the Scrum Master. Again, the Scrum Master is not the boss nor manager. The Development Team self-organizes and takes care of the Daily Scrum themselves.

At the end of the Sprint, for a two-week Sprint that would typically be the last Friday every fortnight, the Scrum Team invites stakeholders, partners and even customers for a Sprint Review meeting.

This is an end of Sprint “what” meeting to showcase what was built and released, and the main purpose is to elicit feedback from the stakeholders. The main question is, “are we building the right product?” The Product Owner is responsible for facilitating the meeting.

The final event of a Sprint is the Sprint Retrospective. This is an end of Sprint “how” meeting facilitated by the Scrum Master. Only the Scrum Team attends, and is typically held right after the Sprint Review.

The focus is on continuous improvement, and while the standard what went well and not, and what can we do better kind of discussions are fine, it’s often more fun to use one of the many Sprint retro games created by the Scrum community.

So that’s pretty much it on what Scrum is.

The Scrum Guide says Scrum is simple to understand but difficult to master.

Very true.

Here’s a blog post I wrote on some of the sad states of failed Scrum teams I’ve come across or heard. Some of them are horror stories.

So, my advice, if you’re going to do Scrum, do it properly.

A lot, can go wrong with Scrum.

And here are a couple more blogs on my website that would help getting Scrum right.

Okay, last chapter and we’re rapping up. Back to the wider notion of agile and what we need to be careful about in doing and becoming agile.

Question: would you agree with me that agile is not a methodology?

It is, understandably debatable, but typically, a methodology comes with specific instructions and a somewhat systematic application of a process.

Agile is process driven too, but not fixed process. I believe agile is a much wider notion than a methodology; more an approach, framework, modality, mindset, a way, spirit, culture.

In fact, if I summarize what I consider as the spirit of agile, it definitely does not look like a methodology.
Agile is about relentlessly pursuing customer value.

We accept uncertainty head-on with agile.

We build-measure-learn and iterate incrementally.

We trust in autonomy – this is a big statement because it means we’re giving up control over people.

And because no trial and error comes without failure, we welcome failure.

And to welcome failure, we need the premise of a no-blame culture agreed in the organization.

A more facilitative and coaching style of leadership and management works better in agile, and we definitely need that to help people self-organize into cross-functional teams.

It’s quite a change isn’t it, becoming an agile organization.

As an Organization Development professional, my job is to provide reality to agile organizational transformation.

One reality is the notion of capital A Agile, and lower case a agile – or to easier differentiate, agility. Capital A Agile is the type of Agile that is applied to disciplined frameworks such as Scrum, while lower case a agile and agility applies to a desired mindset and culture.

In this table, I listed up all values, behaviors and practices characteristic to capital A Agile, and classified them by learning, organizational and process attributes.

And then I considered each element against what I believe is the minimally necessary mindset and culture for lower case agile and agility.

This is the same table resorted.

Agile and agility, my belief is that not all parts of the organization needs to be capital A Agile, but the whole of the organization can adopt lower case a agile and agility.

As you can see, most of the “Encouraged” items listed under agility requires process and structural changes to the organization and some significant skills upgrades, while the “Yes” items under agility are mindset and cultural shift target elements.

This makes agile organizational transformation more realistic and potentially applicable to the whole organization, while making capital A Agile adoption selectively applicable to where necessary in the organization.

And why is it necessary for the whole organization to adopt agility?

Agile adoption success at the team or business unit level, as shown in the lower right quadrant here, is a fairly common occurrence. The challenge here is that the teams eventually lose steam as they don’t receive the right support from leaders that are aligned with them.

Leaders who don’t embrace agile can’t give proper credit, recognition and support to Agile teams.

What is needed, is success like in the upper right quadrant, where there is alignment with leadership.

I don’t think leaders have to operate at the same level of capital A Agile as with the Agile teams, but at the minimum, I believe they need to commit and abide to an agility mindset and behavior.

And if leaders don’t honestly and sincerely learn and adopt the agility mindset and behavior, we will see the worst tragedy of agile transformation – what I call, waterfall agile.

Delivering agile organizational transformation with waterfall project management, it sounds moronic, right?

Well, it happens, a lot. As a matter of fact, saving people from this trap is a large part of my organization development work.

People subtly fall into the waterfall trap without realizing it. We are so ingrained in waterfall project management culture, it’s really hard to shake off the linear, predictive planning style of work. It’s so hard to de-learn waterfall.

I’d like to close this session with this slide, which should help raise the consciousness of our waterfall tendencies, and set the tone for an agile mindset.

Simple, complicated, complex, chaotic – this is known as the Cynefin framework. The first issue here is that we often confuse complicated, with complex.

Complicated tasks are things where many expert steps are required, yet as long as you follow the steps, you get predictable results.

A good example is a car engine – 10,000 parts and expert skills are required to assemble a car engine, but as long as the right steps are followed, you can make identical engines one after another.

Compared to that, a complex task is something where there are known ways of doing it and you have confidence that you’ll get what you want, but you can’t really tell what exactly you will get. So, sales, marketing, and Agile product development, right?

Okay, so, to solve a complicated task, we can handle it algorithmically. That is, like solving a math problem, step-by-step in a linear way.

Now as we learned today, the right way to solve a complex challenge is to do trial and error, right? Yes, and it’s called heuristic problem solving, and as a matter of fact we can call Agile, heuristic problem solving.

Now here comes the real big problem, like waterfall agile. It’s solving complex problems algorithmically. This is the crux of the problem. We’re trying to solve complex problems, the wrong way. This is the simple main cause of the many dysfunctions we see in our work today.

The right way, again, to solve complex adaptive challenges is to use heuristics, which is trial and error, which, in iteration is agile.

Thank you very much.