293개의 교훈으로 이루어진 책. > Any one of these tiny lessons could be worth the price of the book -- [Ward Cunningham](/pages/Ward%20Cunningham.txt)의 [Framework for integrated test](/pages/Framework%20for%20integrated%20test.txt)도 이 책에서 말하는 의 영향을 많이 받았다고 한다: > I particularly like the section on automation because it states so clearly the basis on which automation should be judged. I'm working up what I think of as "reference designs" for [Extreme programming](/pages/Extreme%20programming.txt) . I will do my best to be sure they reflect nicely in the light of this book. Let me also say that I like very much the format of your book. It is what I had hoped the patterns community would routinely produce. The last time I saw a book I liked so much I asked to write the foreword. That was for and 's "". -- [Ward Cunningham](/pages/Ward%20Cunningham.txt) ## Chapter 1. The Role of the Tester Lesson 8. You focus on failure, so your clients can focus on success. Lesson 9. You will not find all the bugs Lesson 10. Beware of testing "completely" Lesson 11. You don't assure quality by testing ## Chapter 2. Thinking Like a Tester Lesson 16. Testing is applied : > Epistemology is a branch of philosophy that helps you test better. … Epistemology is the study of how you know what you know. It's the study of evidence and reasoning. It establishes the foundations of scientific practice. … Applied to software testing, epistemology asks questions like the following: > > - How do you know the software is good enough? > - How would you know if it wasn't good enough? > - How do you know you've tested enough? Lesson 17. Studying epistemology helps you test better: > Here are three very readable books to get you started:

- : Metathoughts for Psychology

-

- | 이번엔 인지심리학(). Lesson 18. Testing is grounded in > Some of its topics relating to testing include: > > - Reliability of your senses and memory. > - Where beliefs come from. > - How your beliefs affect your behavior. > - Biases and short cuts you use to make decisions. > - How you learn and share what you know. > - How you think about complex things. > - How you think under pressure. > - How you recognize patterns. > - How you sort ideas and things into categories. > - How you notice differences between things. > - Distortions in remembered events. > - How you reconstruct partially remembered events (such as nonreproducible bugs). Lesson 38. Use heuristics to quickly generate idea for tests. > Here are some examples of testing heuristics: > > - Test at the boundaries. Boundaries are more linkely to reveal ambiguities in the specification > - Test every error message. Error-handling code tends to be weaker than mainstream functionality > - Test configurations that are different from the programmer's. The programmer will already be biased toward making sure his own configuration works. > - Run tests that are annoying to set up. Easy-to-set-up tests are more likely to be performed, all other things being equal. > - Avoid redundant tests. If a test is truly duplicating another test, what new value is it giving you? ## Chapter 3. Testing Techniques ## Chapter 4. Bug Advocacy Lesson 65. Never use the bug-tracking system to monitor programmers performance. Lesson 66. Never use the bug-tracking system to monitor testers performance. Lesson 74. A failure is a symptom of an error, not the error itself. > The failure migh look minor, but the underlying problem might be much more serious. … Therefore, whenever you see an error that looks minor. > > - Do follow-up testing to find more serious symptoms, which will make for a more compelling report. > - Do follow-up testing to show that the problem occurs under a wider range of circumstances and will be seen by lots of people. Lesson 76. Always report nonreproducible errors; they may be time bombs. > Programmers have tools that you don't have. If you report the symptoms clearly, the programmer can often trace through the code, asking how you could have gotten a certain message or what could possibly have happened when you were looking at a given dialog or clicking on a given control. … > > Using PrintScreen, a screen recorder like Spector, or even video recording can help you prove the existence of UFOs (Unidentified Funny Objects) that some programmers would otherwise say could never have appeared. Lesson 77. Nonreproducible bugs are reproducible. Lesson 80. Ask before reporting bugs against prototypes or early private versions. Lesson 83. The summary line is the most important line in the bug report. Lesson 85. Report the problem clearly, but don't try to solve it. > Your job is to report problems, not to identify root causes and not to push for specific solutions. Without studying the underlying code, you aren't in a position to know the root cause of a failure. … > > Some programmers are likely to reject reports whose "solutions" they know to be invalid, without considering the underlying problem. Don't make it easy to reject your reports. … > > Reports like these sometimes bring a lot of laughs to the programming group. You might not want to be the person at whose ignorant suggestions they're laughing.| Lesson 87. Make your reports readable, even to people who are exhausted and cranky. > A remarkable number of bugs are fixed (and deferred) in the last weeks of the project, by programmers who are working heavy overtime to get the product finished. On tough projects, the programmers are often sleep-deprived, under stress, and over-caffeinated. Write reports that these programmers can understand. Lesson 88. Improve your reporting skills. > - Compare closed bugs that were fixed and not fixed. Look for differences in the way they were reported. If you want yours fixed, report them like the ones that historically do get fixed. > - Read programmers' ( and others' ) answers to bug reports. What makes them confused? Angry? Unreceptive? Appreciateve? Lesson 91. Meet the programmers who will read your reports. Lesson 93. When the programmer says it's fixed, make sure it isn't still broken Lesson 94. Verify bug fixes promptly Lesson 95. When fixes fail, talk with the programmer Lesson 96. Bug reports should be closed by testers Lesson 101. When you decide to fight, decide to win! > Talk with other stakeholders, such as Technical Support, Documentation, Sales, and so on. Find out whose budget will be mostaffected if the ubg is left in the product, how much it will cost them, or how bothered they are by it. ## Chapter 5. Automating Testing Lesson 109. Don't estimate the value of a test in terms of how often you run it. > The value of a test comes from the information it provides. Lesson 117. Automated regression tests die. > The biggest problem facing automated regression tests is decay and premature death. Lesson 126. Don't build test libraries simply to avoid repeating code. > Standard programming wisdom recommends avoiding repeating code by placing the repeated code in a function that is called from each place where the repeating code resided. In test automation, this approach often leads to trouble. The opposite approach, which leaves the repeated code in place, is called *open coding*. … > > If you simply relocate the code you see repeated, you end up with a hodge-podge library.| Lesson 127. Data-driven test automation makes it easy to run lots of test variants. > To test different inputs and input combinations with a common test procedure, use data-driven test automation. … After you've put together a data-driven test procedure, you can use it again and again to execute new tests. The technique is most powerful with product workflows that have lots of different data options. Use a more complex variant, *keyword-driven automation*, to support tests composed of varying sequences or several alternate paths. > > Data-driven test automation supports nonprogramming testers. The automators create the data-driven test procedures, and the testers create the test data. In some cases, you might find it hard to automate the verification of the test results. Have your test procedure collect the results and present them in the context of the input data to simplify the manual results analysis. Lesson 128. Keyword-driven test automation makes it easy for nonprogrammers to create tests. > Keyword-driven test automation builds on the data-driven technique. However, the tables contain directives (keywords), not just data. … > > Using it *avoids complex logic in your test scripts (Lesson 125)*. Because the tests are stored in spreadsheets, they're often easy for nonprogrammers to create and review. Bucause the tasks to be used and tested are all that the tester specifies, he can concentrate on the tests, not the control language. > > In contexts that require many nonprogrammers to create automated tests, we believe this is one of the better solutions. One drawback of this approach is that you can't write a tes unless supported keywords already exist for the tasks you require. Defining and implementing the task functions can become a major undertaking. > > Reviewer reports, "Keyword driven approaches, like my own action words, can provide a good basis. However, over the years I have learned that testing and test automation remain very challenging area and need the involvement of experienced specialists to do it right." … > > We also have reports of teams that found that this approach required too much overhead and was unsustainable. ## Chapter 6. Documenting Testing ## Chapter 7. Interacting with Programmers > Many testers have told us that programmers come back with excuses. "It will compromise the security of software." "It will hurt performance." These are occasionally valid concerns for test code. But usually, we believe, these are code phrases for "we don't want to think about it." --p150 ## Chapter 8. Managing the Testing Project Lesson 162. There are always late changes. > Many traditional [project management](/pages/Project%20management.txt) approaches are designed to limit and control change, but others embrace it (for example, see Weinberg 1992 ( Volume 1), Beck 1999 (), Beck et al. 2001 (), and Krutchen 2000 () ). All project management approaches must nonetheless deal with change. … > > Requirements are the result of ongoing struggles between what we want and what we can have. As the project progresses, the requirements change. Lesson 177. "Project documents are interesting fictions: Useful, but never sufficient." > Consider error handling. We've seen estimates that more than 80 percent of the code of a modern software project deals with error handling and less than 20 percent deals with primary flows. Yet even supposedly thorough specifications dedicate less than 20 percent of their space to error handling. That means that 80 percent of the code is designed by programmers as they write it. Lesson 182. Great test planning makes late changes easy. > Here are a few suggestions: > > - Rather than developing a large suite of tests in advance of testing, develop tests as you need them. If later changes to the product obsolete these tests, at least they will have been useful for a while. > - Don't create enormous test documents that have high maintenance costs, such as detailed manual test scripts. Keep your paperwork as lean as psooible. > - Don't tie manual or automated tests to specifics of the user interface, other than tests specifically intended for the user interface. Even if you do , which is necessarily through the user interface, don't tie your test to the fnie details of the UI, because they will change. > - Seriously consider using [Extreme programming](/pages/Extreme%20programming.txt) methods to develop automated tests. … Lesson 192. Try testing in pairs. ## Chapter 9. Managing the Testing Group ## Chapter 10. Your Career in Software Testing Lesson 249. Extend your career beyond Software testing Lesson 250. Extend your career beyond your company Lesson 269. Improve your writing skills ## Chapter 11. Planning the Testing Strategy ## Appendix. The Context-Driven Approach to Software Testing The seven basic principles of the context driven school: 1. The value of any practice depends on its context. 2. There are good practices in context, but there are no best practices. 3. People, working together, are the most important part of any project's context. 4. Projects unfold over time in ways that are often not predictable. 5. The product is a solution. If the problem isn't solved, the product doens't work. 6. Good software testing is a challenging intellectual process. 7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.