Extreme Programming Explained by Kent Beck

Summary

Extreme programming a methodology developed by Kent Beck. It mostly relies on best practices which spread out as more agile approaches to software development became the norm across the industry, but it tries to unify them in a cohesive set of values, principle ans practices which are based on a common ideology around what is good software development and how to approach it.

Extreme Programming Explained is the reference book gathering all the basics to understand where the methodology comes from, what to expect from it and how to make the best of it for your own context. Some of the practices might have aged a bit and can be replaced by new best practices more adapted to the latest tools and infrastructure available in software development. But, the values and principles are rather timeless and likely to remain relevant for quite a long time!

A good read to look at your approach to software development and see what could be challenged or adapted for better results.

Quotes from the book

What is XP?

The old habit of holding something back in order to be safe doesn’t really work. Holding back that last 20% of effort doesn’t protect me. When my project fails, the fact that I didn’t give my all doesn’t actually make me feel better. It doesn’t protect me from a sense of failure that I couldn’t make the project work. If I do my very best writing a program and people don’t like it, I can still feel justly good about myself. This attitude allows me to feel safe no matter the circumstance. If how I feel is based on an accurate read on whether I did my best, I can feel good about myself by doing my best.

Learning to Drive

This is the paradigm for XP. Stay aware. Adapt. Change.

The problem isn’t change, because change is going to happen; the problem, rather, is our inability to cope with change.

Values, Principles, and Practices

Practices are the things you do day-to-day.

Values are the large-scale criteria we use to judge what we see, think, and do.

Values and practices are an ocean apart. Values are universal.

Bridging the gap between values and practices are principles. Principles are domain-specific guidelines for life.

Values

What matters most in team software development is communication.

Simplicity is the most intensely intellectual of the XP values.

Achieving simplicity gives you that much less to communicate about.

No fixed direction remains valid for long; whether we are talking about the details of software development, the requirements of the system, or the architecture of the system. Directions set in advance of experience have an especially short half-life. Change is inevitable, but change creates the need for feedback.

Feedback comes in many forms: Opinions about an idea, yours or your teammates’ How the code looks when you implement the idea Whether the tests were easy to write Whether the tests run How the idea works once it has been deployed

XP teams strive to generate as much feedback as they can handle as quickly as possible. They try to shorten the feedback cycle to minutes or hours instead of weeks or months.

Courage is effective action in the face of fear.

Sometimes courage manifests as a bias to action. If you know what the problem is, do something about it. Sometimes courage manifests as patience. If you know there is a problem but you don’t know what it is, it takes courage to wait for the real problem to emerge distinctly.

The previous four values point to one that lies below the surface of the other four: respect. If members of a team don’t care about each other and what they are doing, XP won’t work. If members of a team don’t care about a project, nothing can save it.

Other important values include safety, security, predictability, and quality-of-life.

Principles

Often, software development doesn’t meet human needs, acknowledge human frailty, and leverage human strength.

What do people need to be good developers? Basic safety—freedom from hunger, physical harm, and threats to loved ones. Fear of job loss threatens this need. Accomplishment—the opportunity and ability to contribute to their society. Belonging—the ability to identify with a group from which they receive validation and accountability and contribute to its shared goals. Growth—the opportunity to expand their skills and perspective. Intimacy—the ability to understand and be understood deeply by others.

There are other human needs; such as rest, exercise, and socialization; that don’t need to be met in the work environment. Time away from the team gives each individual more energy and perspective to bring back to the team.

Limiting work hours allows time for these other human needs and enhances each person’s contribution while he is with the team.

Software development that doesn’t acknowledge economics risks the hollow victory of a “technical success”. Make sure what you are doing has business value, meets business goals, and serves business needs. For example, solving the highest priority business need first maximizes the value of the project.

The time value of money says that a dollar today is worth more than a dollar tomorrow. Software development is more valuable when it earns money sooner and spends money later.

Another source of economic value in software development is its value as options for the future.

Every activity should benefit all concerned.

Mutual benefit in XP is searching for practices that benefit me now, me later, and my customer as well. Win-win-win practices are easier to sell because they relieve some immediate pain.

try copying the structure of one solution into a new context, even at different scales.

Self-similarity isn’t the only principle at work in software development. Just because you copy a structure that works in one context doesn’t mean it will work in another. It is a good place to start, though. Likewise, just because a solution is unique doesn’t mean it’s bad. The situation may really call for a unique solution.

In software development, “perfect” is a verb, not an adjective.

The cycle is to do the best you can today, striving for the awareness and understanding necessary to do better tomorrow. It doesn’t mean waiting for perfection in order to begin.

Teams need to bring together a variety of skills, attitudes, and perspectives to see problems and pitfalls, to think of multiple ways to solve problems, and to implement the solutions.

Two ideas about a design present an opportunity, not a problem. The principle of diversity suggests that the programmers should work together on the problem and both opinions should be valued.

Good teams don’t just do their work, they think about how they are working and why they are working. They analyze why they succeeded or failed. They don’t try to hide their mistakes, but expose them and learn from them. No one stumbles into excellence.

Flow in software development is delivering a steady flow of valuable software by engaging in all the activities of development simultaneously. The practices of XP are biased towards a continuous flow of activities rather than discrete phases.

The more things are deferred, the larger the chunk, the higher the risk. In contrast, the principle of flow suggests that for improvement, deploy smaller increments of value ever more frequently.

The practices of XP are biased towards a continuous flow of activities rather than discrete phases.

To reach excellence, problems need to turn into opportunities for learning and improvement, not just survival.

Sometimes the desire for more time is a mask worn to protect from the fear of the consequences of getting going. Sometimes, though, patience solves a problem by itself.

The critical, difficult problems in software development should be solved several different ways. Even if one solution fails utterly, the other solutions will prevent disaster. The cost of the redundancy is more than paid for by the savings from not having the disaster.

While redundancy can be wasteful, be careful not to remove redundancy that serves a valid purpose. Having a testing phase after development is complete should be redundant. However, eliminate it only when it is proven redundant in practice by not finding any defects several deployments in a row.

If you’re having trouble succeeding, fail. Don’t know which of three ways to implement a story? Try it all three ways. Even if they all fail, you’ll certainly learn something valuable.

This is not intended to excuse failure when you really knew better. When you don’t know what to do though, risking failure can be the shortest, surest road to success.

Quality isn’t a purely economic factor. People need to do work they are proud of. I remember talking to the manager of a mediocre team. He went home on the weekends and made fancy ironwork as a blacksmith. He met his need for quality; he just met it outside of work.

A concern for quality is no excuse for inaction. If you don’t know a clean way to do a job that has to be done, do it the best way you can. If you know a clean way but it would take too long, do the job as well as you have time for now. Resolve to finish doing it the clean way later.

Baby steps acknowledge that the overhead of small steps is much less than when a team wastefully recoils from aborted big changes. Baby steps are expressed in practices like test-first programming, which proceeds one test at a time, and continuous integration, which integrates and tests a few hours’ worth of changes at a time.

Responsibility cannot be assigned; it can only be accepted. If someone tries to give you responsibility, only you can decide if you are responsible or if you aren’t. The practices reflect accepted responsibility by, for example, suggesting that whoever signs up to do work also estimates it. Similarly, the person responsible for implementing a story is ultimately responsible for the design, implementation, and testing of the story.

The practices reflect accepted responsibility by, for example, suggesting that whoever signs up to do work also estimates it. Similarly, the person responsible for implementing a story is ultimately responsible for the design, implementation, and testing of the story.

Primary Practices

Develop in an open space big enough for the whole team. Meet the need for privacy and “owned” space by having small private spaces nearby or by limiting work hours so team members can get their privacy needs met elsewhere.

Does the practice of sitting together mean that multisite teams can’t “do XP”? Chapter 21, “Purity,” explores this question in more depth; but the simple answer is no, teams can be distributed and do XP. Practices are theories, predictions. “Sit Together” predicts that the more face time you have, the more humane and productive the project. If you have a multisite project and everything is going well, keep doing what you’re doing. If you have problems, think about ways to sit together more, even if it means traveling.

Whole Team Include on the team people with all the skills and perspectives necessary for the project to succeed. This is really nothing more than the old idea of cross-functional teams.

Include on the team people with all the skills and perspectives necessary for the project to succeed. This is really nothing more than the old idea of cross-functional teams.

Twelve is the number of people who can comfortably interact with each other in a day. With more than 150 people on a team, you can no longer recognize the faces of everyone on your team. Across both of these thresholds it is harder to maintain trust, and trust is necessary for collaboration. For larger projects, finding ways to fracture the problem so it can be solved by a team of teams allows XP to scale up.

Make your workspace about your work. An interested observer should be able to walk into the team space and get a general idea of how the project is going in fifteen seconds.

Work only as many hours as you can be productive and only as many hours as you can sustain. Burning yourself out unproductively today and spoiling the next two days’ work isn’t good for you or the team.

Write all production programs with two people sitting at one machine. Set up the machine so the partners can sit comfortably side-by-side.

An issue that has come up and requires comment is the close contact in pair programming. Different individuals and cultures are comfortable with different amounts of body space.

Working effectively together feels good. It may be a new experience in the workplace for some. When programmers aren’t emotionally mature enough to separate approval from arousal, working with a person of the opposite gender can bring up sexual feelings that are not in the best interest of the team. If these feelings arise when pairing, stop pairing with the person until you have taken responsibility for and dealt with your feelings. Even if the feelings are mutual, acting on them will hurt the team. If you want to have an intimate relationship, one of you should leave the team so you can build a personal relationship in a personal setting without confusing the team’s communication with a sexual subtext. Ideally, emotions at work will be about work.

Plan using units of customer-visible functionality.

Early estimation is a key difference between stories and other requirements practices. Estimation gives the business and technical perspectives a chance to interact, which creates value early, when an idea has the most potential.

Plan work a week at a time. Have a meeting at the beginning of every week.

Plan work a quarter at a time. Once a quarter reflect on the team, the project, its progress, and its alignment with larger goals.

In any plan, include some minor tasks that can be dropped if you get behind. You can always add more stories later and deliver more than you promised. It is important in an atmosphere of distrust and broken promises to meet your commitments. A few met commitments go a long way toward rebuilding relationships.

Automatically build the whole system and run all of the tests in ten minutes. A build that takes longer than ten minutes will be used much less often, missing the opportunity for feedback. A shorter build doesn’t give you time to drink your coffee.

I had been watching first-hand the incredible waste generated by their habitual overcommitment: unmanageable defect loads, dismal morale, and antagonistic relationships. Meeting commitments, even modest ones, eliminates waste. Clear, honest communication relieves tension and improves credibility.

Integrate and test changes after no more than a couple of hours. Team programming isn’t a divide and conquer problem. It is a divide, conquer, and integrate problem. The integration step is unpredictable, but can easily take more time than the original programming. The longer you wait to integrate, the more it costs and the more unpredictable the cost becomes.

Write a failing automated test before changing any code.

test, code, refactor.

Invest in the design of the system every day. Strive to make the design of the system an excellent fit for the needs of the system that day. When your understanding of the best possible design leaps forward, work gradually but persistently to bring the design back into alignment with your understanding.

The advice to XP teams is not to minimize design investment over the short run, but to keep the design investment in proportion to the needs of the system so far.

Getting Started

To do XP, you start where you are and adapt by adding practices that meet your goals and express your values.

Corollary Practices

Make people whose lives and business are affected by your system part of the team. Visionary customers can be part of quarterly and weekly planning.

The point of customer involvement is to reduce wasted effort by putting the people with the needs in direct contact with the people who can fill those needs.

When replacing a legacy system, gradually take over its workload beginning very early in the project.

You’ll have to find a way to run both programs in parallel, splitting and merging files or training some users to use both programs. This scaffolding, technical or social, is the price you pay for insurance.

Keep effective teams together.

Value in software is created not just by what people know and do but also by their relationships and what they accomplish together.

Once you gel, once you earn and offer trust, nothing short of shared calamity can pull the team apart.

As a team grows in capability, keep its workload constant but gradually reduce its size. This frees people to form more teams.

Every time a defect is found after development, eliminate the defect and its cause. The goal is not just that this one defect won’t ever recur, but that the team will never make the same kind of mistake again.

Anyone on the team can improve any part of the system at any time. If something is wrong with the system and fixing it is not out of scope for what I’m doing right now, I should go ahead and fix it.

Maintain only the code and the tests as permanent artifacts. Generate other documents from the code and tests. Rely on social mechanisms to keep alive important history of the project.

There is only one code stream. You can develop in a temporary branch, but never let it live longer than a few hours.

Put new software into production every night. Any gap between what is on a programmer’s desk and what is in production is a risk. A programmer out of sync with the deployed software risks making decisions without getting accurate feedback about those decisions.

Daily deployment is a corollary practice because it has so many prerequisites. The defect rate must be at most a handful per year. The build environment must be smoothly automated. The deployment tools must be automated, including the ability to roll out incrementally and roll back in case of failure. Most importantly, the trust in the team and with customers must be highly developed.

Write contracts for software development that fix time, costs, and quality but call for an ongoing negotiation of the precise scope of the system. Reduce risk by signing a sequence of short contracts instead of one long one.

With pay-per-use systems, you charge for every time the system is used. Money is the ultimate feedback. Not only is it concrete, you can also spend it. Connecting money flow directly to software development provides accurate, timely information with which to drive improvement.

Even if you can’t implement pay-per-use, you might be able to go to a subscription model, in which software is purchased monthly or quarterly. With the subscription model, the team at least has the retention rates (the number of customers that continue to subscribe) as a source of information about how the team is doing.

The Whole XP Team

Testers on an XP team help customers choose and write automated system-level tests in advance of implementation and coach programmers on testing techniques.

On XP teams much of the responsibility for catching trivial mistakes is accepted by the programmers.

The role of testers shifts to early in development, helping define and specify what will constitute acceptable functioning of the system before the functionality has been implemented.

Interaction designers on an XP team choose overall metaphors for the system, write stories, and evaluate usage of the deployed system to find opportunities for new stories.

Architects on an XP team look for and execute large-scale refactorings, write system-level tests that stress the architecture, and implement stories.

Another task for architects on an XP team is partitioning systems. Partitioning isn’t an up-front, once-and-for-all task, though. Rather than divide and conquer, an XP team conquers and divides. First a small team writes a small system. Then they find the natural fracture lines and divide the system into relatively independent parts for expansion.

Project managers on an XP team facilitate communication inside the team and coordinate communication with customers, suppliers, and the rest of the organization. Project managers act as team historians, reminding the team how much progress it has made.

In XP, product managers write stories, pick themes and stories in the quarterly cycle, pick stories in the weekly cycle, and answer questions as implementation uncovers under-specified areas of stories. A product manager doesn’t just pick a bunch of stories at the beginning of the project and then sit back. A plan in XP is an example of what could happen, not a prediction of what will happen.

Executives provide an XP team with courage, confidence, and accountability. The strength of an XP team, shared progress toward shared goals, can also be a weakness.

What if the team’s goal doesn’t align with corporate goals? What if the goal drifts because of the pressures and excitements of success? Articulating and maintaining large-scale goals is one job for executives sponsoring or overseeing XP teams.

The role of technical publications on an XP team is to provide early feedback about features and to create closer relationships with users.

Users on an XP team help write and pick stories and make domain decisions during development. Users are most valuable if they have broad knowledge and experience with systems similar to the one being built and if they have strong relationships with the larger user community that will use the system once it is fully deployed.

Programmers on an XP team estimate stories and tasks, break stories into tasks, write tests, write code to implement features, automate tedious development process, and gradually improve the design of the system. Programmers work in close technical collaboration with each other, pairing on production code, so they need to develop good social and relationship skills.

Two challenges have been reported for human resources when teams begin applying XP: reviews and hiring.

In XP, valuable employees: Act respectful. Play well with others. Take initiative. Deliver on their commitments.

Roles on a mature XP team aren’t fixed and rigid. The goal is to have everyone contribute the best he has to offer to the team’s success.

The Theory of Constraints

The Theory of Constraints says that in any system there is one constraint at a time (occasionally two). To improve overall system throughput you have to first find the constraint; make sure it is working full speed; then find ways of either increasing the capacity of the constraint, offloading some of the work onto non-constraints, or eliminating the constraint entirely.

The Theory of Constraints says there is always a constraint. When we eliminate one constraint we create another. Micro-optimization is never enough. To improve our results we must look at the whole situation before deciding what to change.

Planning: Managing Scope

The state of a shared plan provides clues about the state of the relationship between the people affected by the plan. A plan out of touch with reality betrays an unclear, unbalanced relationship. A mutually agreed-upon plan, adjusted when necessary to reflect changing reality, suggests a respectful, mutually valuable relationship.

Planning in XP is like shopping for groceries. Imagine that you go into a grocery store with $100 in your pocket.

Plans are not predictions of the future. At best, they express everything you know today about what might happen tomorrow. Their uncertainty doesn’t negate their value. Plans help you coordinate with other teams. Plans give you a place to start. Plans help everyone on the team make choices aligned with the team’s goals.

Lowering the quality of your work doesn’t eliminate work, it just shifts it later so delays are not clearly your responsibility.

Satisfaction comes from doing quality work.

If we make scope explicit, then: We have a safe way to adapt. We have a way to negotiate. We have a limit to ridiculous and unnecessary demands. Plan at every timescale with these four steps: 1. List the items of work that may need to be done. 2. Estimate the items. 3. Set a budget for the planning cycle. 4. Agree on the work that needs to be done within the budget. As you negotiate, don’t change the estimates or the budget.

Testing: Early, Often, and Automated

Defects destroy the trust required for effective software development.

Without trust, people spend much of their time defending themselves against the possibility that someone else may have made a mistake.

Here is the dilemma in software development: defects are expensive, but eliminating defects is also expensive.

The XP practices are aimed at communicating clearly so defects don’t arise in the first place, and when they do, making sure the team uses them to learn how to avoid similar problems in the future.

Testing in XP is a technical activity that directly addresses defects. XP applies two principles to increase the cost-effectiveness of testing: double-checking and the Defect Cost Increase.

Software testing is double-checking. You say what you want a computation to do once when you write a test. You say it quite differently when you implement the computation. If the two expressions of the computation match, the code and the tests are in harmony and likely to be correct.

Defect Cost Increase (DCI) is the second principle applied in XP to increase the cost-effectiveness of testing. DCI is one of the few empirically verified truths about software development: the sooner you find a defect, the cheaper it is to fix.

Code and tests can be written in either order. In XP, when possible, tests are written in advance of implementation.

Tests can provide a measure of progress.

Designing: The Value of Time

Incremental design is a way to deliver functionality early and continue delivering functionality weekly for the life of the project.

There is an art to designing just enough to get feedback, and then using that feedback to improve the design enough to get the next round of feedback.

In software, however, reversing a day’s work is trivial.

Design quality doesn’t ensure success, but design failure can ensure failure.

The most powerful design heuristic I know is Once and Only Once: data, structure, or logic should exist in only one place in the system.

Design is deferred until it can be made in the light of experience and the decisions can be used immediately.

Scaling XP

People often ask how XP scales. One hundred people can’t plan their work in detail in a single meeting once a week. One hundred people can work together in a spirit of communication, feedback, simplicity, courage, and respect. Creating and maintaining a community of one hundred is a much different job than creating and maintaining a community of twelve, but it is done all the time.

At some point they realized that the way problems were solved when there were two programmers no longer worked.

1. Turn the problem into smaller problems.
2. Apply simple solutions.
3. Apply complex solutions if any problem is left.

In summary, faced with the apparent need for a large team, first ask if a small team can solve the problem. If that doesn’t work, begin the project with a small team, then split the work among autonomous teams.

Be sure to maintain communication with the rest of the organization in the forms they are accustomed to.

Long-running XP projects work well because the tests prevent many of the common maintenance mistakes and tell the story of the development process.

XP is ideally suited to projects requiring the close cooperation of specialists. One challenge at the beginning of such projects is getting everyone to work in concert

Sometimes systems grow big and complicated, out of proportion to the problem they solve. The challenge is to stop making the problem worse. It is difficult for a struggling team to keep going when every defect fixed creates three more.

The XP strategy for dealing with excess complexity is always the same: chip away at the complexity while continuing to deliver. Brighten the corner where you are. If you are fixing a defect in an area, clean up while you are there.

Taylorism and Software

The first step of social engineering in Taylorism is the separation of planning from execution. It is the educated engineers who decide how work is to be done and how long it will take. The workers must faithfully execute the assigned task in the assigned way in the allotted time and all will be well. Workers are cogs in a machine. No one wants to think of himself as a cog.

The second step of Taylorist social engineering is the creation of a separate quality department. Taylor assumed that workers would “soldier” whenever possible (work slowly or badly but not so slowly or badly as to be noticed). He created a separate quality control department to ensure that workers not only worked at the right pace but in the specified way, in order to achieve the right level of quality.

Toyota Production System

Their alternative social structure of work is critical to the success of TPS. Every worker is responsible for the whole production line. If anyone spots a defect he pulls a cord that stops the whole line.

Waste is eliminated through kaizen (continuous improvement) events. Workers identify a source of waste, either quality problems or inefficiency. Then, they take the lead in analyzing the problem, performing experiments, and standardizing the results.

Software development is full of the waste of overproduction: fat requirements documents that rapidly grow obsolete; elaborate architectures that are never used; code that goes months without being integrated, tested, and executed in a production environment; and documentation no one reads until it is irrelevant or misleading. While all of these activities are important to software development, we need to use their output immediately in order to get the feedback we need to eliminate waste.

Conclusion

The key to XP is integrity, acting in harmony with my true values.

With XP, I work to become worthy of respect and offer respect to others. I’m content to do my best and strive always to improve. I hold values I am proud of and act in harmony with those values.