How to write a performance test case

Chuyên đề thảo luận về Performance Testing
Forum rules
Chuyên đề này chỉ thảo luận về Performance Testing.
Để có kết quả nhanh, các bạn nên search trước khi tạo chủ đề mới.
Post Reply
tvn
Admin
Posts: 4792
Joined: Tue 10 Aug, 2010 10:11 am
Location: HCM
Contact:

How to write a performance test case

Post by tvn » Sat 24 Nov, 2012 1:13 pm

I have decided to release an early draft of this document so that others may provide feedback. Please let me know what you think.

Writing test cases for performance testing requires a different mindset to writing functional test cases. Fortunately it is not a difficult mental leap. This article should give you enough information to get you up and running.

First, lets set out some background and define some terms that are used in performance testing.

Test case is a test case is the same as a use case or business process. Just as with a functional test case, it outlines test steps that should be performed, and the expected result for each step.
Test script is a test script is a program created by a Performance tester that will perform all the steps in the test case.
Virtual user is a virtual user generally runs a single test script. Virtual users do not run test scripts using the Graphical User Interface (like a functional test case that has been automated with tools like WinRunner, QuickTest, QARun or Rational Robot); they simulate a real user by sending the same network traffic as a real user would. A single workstation can run multiple virtual users. [Would a diagram be useful here?]
Scenario is a performance test scenario is a description of how a set of test scripts will be run. It outlines how many times an hour they will be run, how many users will run each test script, and when each test script will be run. The aim of a scenario is to simulate real world usage of a system.

Writing a test case for performance testing is basically writing a simple Requirements Specification for a piece of software (the test script). Just as with any specification, it should be unambiguous and as complete as possible.

Every test case will contain the steps to be performed with the application and the expected result for each step. As a performance tester will generally not know the business processes that they will be automating, a test case should provide more detail than may be included in a functional test case intended for a tester familiar with the application.

It is important that the test case describes a single path through the application. Adding conditional branches to handle varying application responses, such as error messages, will greatly increase script development time and the time taken to verify that the test script functions as expected. If a test script encounters an error that it does not expect, it will usually just stop. If the Project Manager decides that test scripts should handle errors the same way a real user would, then information should be included on how to reproduce each error condition, and additional scripting time should be included in the project plan.

The main reason a user may be presented with a different flow through the application is the input data that is used. Each test case will be executed with a large amount of input data. Defining data requirements is a critical part of planning for a performance test, and is the most common area to get wrong on a first attempt. It is very easy to forget that certain inputs will present the user with different options.

The other important data issues to identify are any data dependencies and any potential problems with concurrency. Is it important that data is used in some business functions before they are used in others? And, will data modified by virtual users cause other virtual users to fail when they try to use the same data? The test tool can partition the data used by each virtual user if these requirements can be identified. It can be difficult for a performance tester to debug test script failures with little knowledge of the application, especially if the failures only occur when multiple virtual users are running at once.

One of the most important pieces of information a performance test is designed to discover is the response time of the system under test is both at the overall business function level and at the low level of individual steps in the test case, such as the time it takes for a search to return a result set. Any test cases provided to a performance tester should clearly define the start and end points for any transaction timings that should be included in the test results.

It is important to remember that the test script is only creating the network traffic that would normally be generated by the application under test. This means that any operations that happen only on the client do not get simulated and therefore do not get included in any transaction timing points. A good example would be a client application that runs on a user™ PC, and communicates with a server. Starting the client application takes 10 seconds and logging in takes 5 seconds but, since only the login is sending network traffic to the server, the transaction timing point will only measure 5 seconds.

Operations that only happen on the client, including the time users take to enter data or spend looking at the screen is simulated with user think time is an intentional delay that is inserted into the test script. If no think time is included, virtual users will execute the steps of the test case as fast as they can, resulting in greater (and unrealistic) load on the system under test. Depending on the sophistication of the performance test tool, the user think time may be automatically excluded from the transaction timing points. Think times are generally inserted outside of any transaction timing points anyhow.

While a functional test case will be run once from start to finish, a performance test case will be run many times (iterated) by the same virtual user in a single scenario. Information on how the test steps will be iterated should be included in the test case. For example, if a test case involves a user logging in and performing a search, and the entire test case is iterated by the virtual user; then a test scenario may be generating too many logins if the real users generally stay logged into the application. [Is there a way to break up this sentence?] A more realistic test case may have the virtual user log in once and then keep doing the same action for as long as the virtual user is run.

When a script is iterated, consideration should be given to the non-obvious details of how it is iterated. A good example would be a test script simulating users using an Internet search engine. When the test script is iterated, simulating a new search operation, should the virtual user establish a new network connection and empty their cache or should every iteration simulate the same user conducting another search?

As all performance test tools have different default behaviours, a good performance tester should clarify this type of detail with business and technical experts. Some performance test tools make these details easier to change than others. If it is not practical to emulate all the attributes of the expected system traffic with a particular tool, then it should be noted as a limitation of the performance test effort, and a technical expert should assess the likely impact.

Hopefully this article has provided some insight into the extra considerations that must be given when writing a performance test case, rather than a functional test case. As with any software specification, a performance test case may need to be refined as questions are raised by the performance tester.

An example test case:
example-performance-test-case.jpg
Nguồn: http://www.myloadtest.com/how-to-write- ... test-case/
You do not have the required permissions to view the files attached to this post.



phailamsao263
Fresher Tester
Posts: 26
Joined: Sun 23 Sep, 2012 4:20 pm
Contact:

Re: How to write a performance test case

Post by phailamsao263 » Sat 15 Dec, 2012 11:24 am

Mọi ơi anh có tài liệu chuẩn quy định về kiểm thử hiệu năng(Website)không cho mình xin với để mình viết đặc tả cho website, rùi viết testcase cho website
ví dụ: quy định thời gian trả ra kết quả tìm kiếm, đăng nhập,...khi người dùng yêu cầu
chuẩn của khối lượng: hệ thống có thể lưu được bao nhiêu dữ liệu, bao nhiêu người dùng tối đa, tối đa bao nhiêu người dùng có thể cùng đăng nhập một lúc...



tvn
Admin
Posts: 4792
Joined: Tue 10 Aug, 2010 10:11 am
Location: HCM
Contact:

Re: How to write a performance test case

Post by tvn » Mon 17 Dec, 2012 7:05 pm

phailamsao263 wrote:Mọi ơi anh có tài liệu chuẩn quy định về kiểm thử hiệu năng(Website)không cho mình xin với để mình viết đặc tả cho website, rùi viết testcase cho website
ví dụ: quy định thời gian trả ra kết quả tìm kiếm, đăng nhập,...khi người dùng yêu cầu
chuẩn của khối lượng: hệ thống có thể lưu được bao nhiêu dữ liệu, bao nhiêu người dùng tối đa, tối đa bao nhiêu người dùng có thể cùng đăng nhập một lúc...
Những cái này không ai qui định chuẩn là bao nhiêu cả bạn ạ. Tùy khách hàng, loại website (tải dữ liệu nhiều không, có media nhiều hay chỉ text thôi,...) và cấu hình hệ thống server của họ mà họ đưa ra yêu cầu. Bạn có thể tham khảo một yêu cầu performance test ở đây.



phailamsao263
Fresher Tester
Posts: 26
Joined: Sun 23 Sep, 2012 4:20 pm
Contact:

Re: How to write a performance test case

Post by phailamsao263 » Sun 23 Dec, 2012 11:18 am

Vâng, website của em không có media. Chỉ là post thông tin tour du lịch, khách hàng đăng ký tour và gửi ý kiến phản hồi lên ạ.
Em tự viết đặc tả cho website và viết test case thì cô giáo em bảo số liệu không hợp lý, mà phải tìm chuẩn thì mới được. Em thì không rõ yêu cầu về thời gian đáp ứng, khả năng lưu trữ hay chịu tải của một website thì thường như thế nào hì :D



tvn
Admin
Posts: 4792
Joined: Tue 10 Aug, 2010 10:11 am
Location: HCM
Contact:

Re: How to write a performance test case

Post by tvn » Sun 23 Dec, 2012 7:47 pm

Chào bạn,

Vậy thì bạn nên tìm hiểu thêm ở đây nha, đây là mẫu yêu cầu phi chức năng cho website. Bạn tham khảo thêm về yêu cầu performance ở đâ nha http://msdn.microsoft.com/en-us/library/bb924365.aspx

Mình cũng có tìm thử về các tiêu chuẩn hiệu năng của website nhưng không có, và mình nghĩ cái này cũng không có tiêu chuẩn nào cả. Bạn chỉ có thể tham khảo hiệu năng - thời gian phản hồi, số lượng user online cùng lúc,... - của các website lớn, các website tương tự với ngành của bạn hoặc phỏng vấn khách hàng tìm năng về mong muốn thời gian phản hồi của website là bao nhiêu giây thì chấp nhận được, để từ đó bạn đưa ra con số tốt nhất và phù hợp nhất với website của mình (và không phải bàn cãi, thời gian phản hồi càng nhỏ càng tốt).



Shoe
Hoc Tester
Posts: 1
Joined: Tue 01 Mar, 2016 10:40 am
Contact:

Re: How to write a performance test case

Post by Shoe » Thu 27 Apr, 2017 10:50 pm

Cho mình hỏi kia là đối với website, vậy ở winform có không bạn?



tvn
Admin
Posts: 4792
Joined: Tue 10 Aug, 2010 10:11 am
Location: HCM
Contact:

Re: How to write a performance test case

Post by tvn » Fri 28 Apr, 2017 6:27 pm

Nếu winform mà cũng có server, thì có thể nên kiểm thử.
Nhưng JMeter không hỗ trợ nên bạn có thể sử dụng Loadrunner hoặc test tool trong Visual Studio.



Post Reply

Return to “Performance Testing - Kiểm thử hiệu năng”