非功能性屬性的測試

　　4. 非功能性測試的若干問題

　　· One key issue is when to do NFT? The problem with non-functional defects is that most of them will be related to the design and fixing them will need a fix in the design and therefore effort in fixing may be high. This warrants that NFT betaken up as early in the test cycle as possible or the fix may adversely affect meeting the delivery schedules. Another reason for advocating an early execution of NFT is that some of the NFT defects like usability issues maybe perceived to be of a lower priority and the probability of such defects being left open as limitations is quite high. The project management tends to postpone these since changing the code may potentially affect even the working functionality.

　　一個關鍵的問題是何時進行非功能性測試。大部分非功能性的缺陷與設計有關，故要修復它們就需要修改設計，因此修復代價是很高的。這就說明非功能性測試需要在測試周期中盡可能早的進行，否則修復活動將反過來影響到提交進度的實現。提倡盡早實施非功能性測試的另一個原因是：類似可用性這樣的非功能性屬性在測試中出現的缺陷可能被認為是低優先級的，并且這種缺陷成為一個公開限制的可能性非常高。項目管理者傾向于把這些缺陷的修復推后，因為改變代碼可能對系統功能潛在地產生影響。

　　But the problem with scheduling NFT early, is that in many cases the system need to attain a minimum level of stability and functionality completeness for the test cases to run. Examples are load, performance and configuration testing. This will mean that the system functionality testing has to be done and any defects found from this need to be fixed before NFT can be carried out. If the system is not robust and stable enough before performance or load testing is attempted, the test team maybe wasting a lot of effort in restarting the system etc..

　　然而，把非功能性測試安排到早期也存在問題。那就是在很多情況下系統需要獲得一個最低程度的穩定性和功能完整性來運行測試用例。例如載入測試、性能測試和配置測試。這就意味著在非功能性測試完成之前，系統功能性測試應該已經完成并且從中找到的任何缺陷都應該得到修復。假如在性能測試和載入測試進行之前系統達不到足夠的健壯和穩定的話，測試團隊很可能在重起系統等方面等待很長時間。

　　Considering the above mentioned factors, the following approach is suggested:

　　考慮到以上因素，這里提出如下建議：

　　Initial Rounds : Functionality testing, Usability testing, Security testing

　　第一輪：功能性測試、可用性測試、安全性測試。

　　Intermediate Rounds : Functionality (Remaining and regression), Configuration Testing, Recovery testing

　　中間輪：功能性測試(剩余的和回歸的)、配置性測試、恢復性測試。

　　Final rounds : Volume, Stress and performance

　　最終輪：容量測試，壓力測試和性能測試。

　　This is a general guideline for sequencing and will need to be tailored for individual projects.

　　這只是一個對測試順序的大概說明，在具體項目中還可以自行取舍。

　　If reliability assessment is done through analysis, it will continue throughout test execution.

　　如果已經通過分析完成了可靠性評估，那么還需要通過繼續測試來完成最終的評估。

　　· There are a few issues related to regression testing of NF requirements to be considered. 100% regression is neither practical nor advisable. Examples of issues are

　　還有幾點是關于對非功能性需求的回歸測試的，100%的回歸測試既不現實也不被提倡。以下是兩個例子：

　　– Repetition of time consuming volume testing can affect release schedule.

　　– It will not be possible to get the desired feedback if the same subject is used again for doing usability testing of same procedures

　　— 容量測試中大量時間的消耗會影響項目的進度

　　— 如果在可用性測試中對于同樣的過程使用同樣的主題，是不可能得到想要的反饋的。

　　Therefore normally only defect fixes are verified for most cases unless there are major changes in implementation, UI design etc.. Exceptions can be performance and stress testing.

　　因此，除非在實現，用戶接口設計等方面有重大變化，通常情況下僅僅校驗缺陷的修復。而性能測試和壓力測試又屬例外情況。

　　5. Performance Testing

　　5. 性能測試

　　The importance of performance of the system to the customer is somewhat better understood than other NF requirements. Testing for the stated requirement also is performed in many projects. But this testing is sometimes limited to the measurement of one value, say, throughput rate or response time. But this need to be done under differing processing and configuration conditions.

　　對于客戶來說，系統性能的重要性比其他非功能性需求要好理解一些。但在很多項目中，對規定需求的測試還是要進行。然而，測試往往局限于對某個值的測量，比如吞吐率或者響應時間。盡管如此，在不同的處理和配置條件下，這些測試還是要去實施。

　　The performance of the system depends on various factors including configuration of the system, external factors and current state of the system. Therefore, test team should take into aclearcase/" target="_blank" >ccount these while doing the performance benchmarking [6]. Configuration of the system includes all hardware and software configurations like number of processors, RAM, DB server software used etc.. External factors may include things like network load and switching system used. Internal factors may include transactions processed since startup, current load on the system, any failure conditions etc.. The various steps in performance testing are listed below:

　　系統的性能取決于很多因素，這些因素包括系統的配置，外部條件以及系統的當前狀態。因此，當測試團隊在做性能基準的時候需要考慮這些因素。系統配置包含所有的硬件和軟件，例如處理器數量、RAM、數據庫服務器軟件等等。外部因素包含網絡負載和交換機等。內部因素包含啟動后的業務處理、系統當前負載以及任何失敗條件等等。以下列出了性能測試有關的若干步驟：

　　1) Identify the parameters to be measured

　　Examples are call setup time, TPS etc. This may be taken from requirements book.

　　1) 確定要測量的參數。例如調用啟動時間，TPS等等。這些可以從需求說明書中得到。

　　2) Identify the different external factors, system configuration parameters and internal factors which affect performance. It will be a good idea to brainstorm and come up with a fish-bone diagram.

　　2) 確定不同的外部條件、系統配置參數以及能夠影響性能的內部因素。使用頭腦風暴法以及魚骨圖會是個不錯的主意。

　　3) Identify discrete values each of these parameters can take and shortlist the factors and values for which you plan to carry out the testing. It is essentially identifying interesting profiles for testing.

　　3) 確定離散值，使每一個參數都能反映出影響性能的因素并且可以對這些值可以進行測試。確定那些引起興趣的方面并進行測試很重要。

　　4) Identify a default, most common value for each factor.

　　4) 為每一個因素確定一個默認的，最常用的值。

　　5) Identify the measurement method

　　This may require tools, special build of the product, special setup etc.. Plan and implement these

　　5) 確定測量方法。這可能需要使用工具、產品的特殊構建、特殊的啟動等等。規劃并實現它們。

　　6) Do the measurement varying one parameter at a time keeping all the others at their default values.

　　6) 在保持其他所有默認值不變的情況下，每次測量的時候只改變一個參數。

　　7) Plot the results so that the effect of each parameter on performance is identified.

　　7) 對測試結果進行劃分，這樣可以確定每一個參數對性能的影響程度。

　　Steps 1-4 are done during test planning phase, step 5 is done as part of test requirements, step 6 relates to test execution and step 7 is part of test reporting. The advantage of this is that marketing or customer support has data to advise on the ideal configuration for a customer based on his performance and other requirements.

　　步驟1到4在測試計劃階段實施，步驟5作為測試需求的一部分，步驟6與測試執行有關而步驟7是測試報告的一部分。這樣做的好處是可以基于性能和需求來為市場部或者客戶支持部提供數據，以便讓他們為客戶提供理想的配置。

　　Performance testing can be accomplished in parallel with Volume and stress testing because you want to assess performance under all load conditions. Volume testing is performed to find weaknesses of the system with respect to its handling large amounts of data over time. Stress testing is carried out to find out the system behavior when subjected to large numbers of transactions during peak period.

　　性能測試可以和容量測試以及壓力測試并行地完成，這是因為你想在所有的載入條件下都到達性能指標。容量測試通過系統對大量數據的處理時間來找到系統的弱點。而壓力測試則是通過系統在高峰期的大量的業務處理來找出系統的缺陷。

　　If the performance does not measure up to the requirements / expectations, profiling tools are used to find the performance bottle necks in the system. Test team will need to assist development team in this as well.

　　如果系統性能不能滿足需求的期望值，那么就要使用剖析工具來找出制約系統性能的瓶頸。測試團隊此時同樣要協助開發團隊。

　　Other activities normally associated with performance testing are resource utilization measurements. This normally includes memory and CPU utilization and the testing is carried out under volume and stress conditions.

　　其他與性能測試協同進行的過動是資源利用測量。它通常包含內存和CPU的利用率并且是在容量和壓力的條件下進行測試。

　　6. Usability Testing

原文轉自：http://www.kjueaiud.com