一旦一個程序通過了測試,那么以后就不太可能不通過該測試了。這種說法導出了一些論述(沒有一個被明確贊成),即自動化測試有危險,因為它給我們一種很舒服的錯覺,認為程序沒有問題。即使程序今天沒有出現昨天出現的問題,但是依然還有很多情況可以使程序出問題。而如果你總是在沒有bug的地方尋找,你就永遠找不到它們。
6. Of the bugs found during an automated testing effort, 60%-80% are found during development of the tests. That is, unless you create and run new test cases under the automation tool right from the start, most bugs are found during manual testing. (Consensus)
在一次自動化測試工作中找到的bug里,60%到80%是在開發測試中被發現的。也就是說,除非你從一開始就用自動化工具創建并運行新的測試用例,不然大多數bug都會在手動測試中被發現。(一致同意)
Most of us do not usually use the automation tool to run test cases the first time. In the traditional paradigm, you run the test case manually first, then add it to the automation suite after the program passes the test. However, you can use the tool more efficiently if you have a way of determining whether the program passed or failed the test that doesn’t depend on previously captured output. For example:
我們大多數人都不是第一次使用自動化工具來運行測試用例。在傳統規范中,你首先要手動運行測試用例,然后在程序通過測試后把它添加到自動化套件中。但是,如果你有一種辦法不依靠先前的截取的輸出結果就能確定程序是否通過了測試,那么你就能更加有效的使用工具。例如:
Run the same series of tests on the program across different operating system versions or configurations. You may have never tested the program under this particular environment, but you know how it should work.
要在不同的操作系統版本或配置上運行相同系列的測試。你可能從未在一種特殊配置環境下測試過程序,但是你知道它應該怎樣工作。
Run a function equivalence test. [11] In this case, you run two programs in parallel and feed the same inputs to both. The program that you are testing passes the test if its results always match those of the comparison program.
要運行一個功能等價測試。這時,你可以并行地運行兩個程序,并且給它們同樣的輸入。如果程序的結果總是與對照程序一樣,就說明它通過了測試。
Instrument the code under test so that it will generate a log entry any time that the program reaches an unexpected state, makes an unexpected state transition, manages memory, stack space, or other resources in an unexpected way, or does anything else that is an indicator of one of the types of errors under investigation. Use the test tool to randomly drive the program through a huge number of state transitions, logging the commands that it executes as it goes. The next day, the tester and the programmer trace through the log looking for bugs and the circumstances that triggered them. This is a simple example of a simulation. If you are working in collaboration with the application programming team, you can create tests like this that might use your tool more extensively and more effectively (in terms of finding new bugs per week) than you can achieve on your own, scripting new test cases by hand.
要在測試中操作代碼,這樣就能在出現異常情況時生成日志,比如,程序到達了一個異常狀態、做了一個異常狀態轉換、以異常方式管理內存、堆;蚱渌Y源、或者做了任何一種錯誤的事情。先通過大量的狀態轉換以及記錄執行的命令使用測試工具隨機地驅動程序。然后在第二天,測試人員和程序員通過跟蹤日志來尋找bug和觸發它們的環境。這是模擬出的一個簡單例子。如果你正在與應用程序團隊合作,那你就能像這樣,相對自己手工編寫新的測試用例腳本,更加擴展地、有效地(每周都可以找到新的bug) 用你的工具創建測試。
7. Automation can be much more successful when we collaborate with the programmers to develop hooks, interfaces, and debug output. (Consensus)
當我們與程序員合作開發鏈接、接口和調試輸出時,自動化會更加成功。(一致同意)
文章來源于領測軟件測試網 http://www.kjueaiud.com/