Verification vs Validation

[tvn]

Verification
Are we building the product right?

Validation
Are we building the right product?

Verification vs Validation
The design requirements (section 7.3) for ISO 9001:2000 require that designs be verified and validated. This requirement has been in the ISO 9000 series requirements from their inception. It has also been a source of confusion. To discuss the subject we will nee some definitions. Verification is the conformation that a product meets identified specifications. Validation is conformation that a product appropriately meets its design function or the intended use.

So far the distinction is just words and not helpful determining what to do. The following example may be helpful. One of our clients makes an epoxy material that is used to form the head gasket of an engine. There are a variety of specifications including formulation, specific gravity, flow characteristics, and temperature resistance that apply to the epoxy. Testing that assures conformance to these specifications is verification. When the epoxy is applied to an engine properly it must withstand the working pressures of an engine and perform as a head gasket. If the epoxy leaked when the engine was pressurized it would fail validation. It may have met all the material specifications (verification) but it did not work as a head gasket (validation). Obviously, almost all properly designed products will pass validation testing if they pass verification testing. But, some products are difficult or impossible to verify by the manufacturer. For example, the engine mounts on a car. Their design function is to physically support the engine and decouple engine vibration from the chassis. The only way to validate the engine mount is to assemble it in a car and determine if it isolates engine vibration. None of the specification testing (verification) can absolutely assure that the mount will provide sufficient vibration decoupling (validation).

The engine mount manufacturer cannot perform validation testing because they usually do not have access to a test vehicle. The vehicle manufacturer is reticent to take responsibility for the validation activity and provide objective evidence of the validation process even though they are the only organization that can validate the product.

In a ISO/QS-9000 registered system this leads to difficulty. There is a clear requirement for validation but it may be impossible for the manufacturer to perform. In the case of the engine mount, even if the manufacturer could get a test vehicle, their judgment about the adequacy of the vibration decoupling would usually not satisfy the customer; the customer would want to make that judgment (perform the validation).

Although the distinction may now be clear, a solution may not be evident in cases where the manufacturer does not have the ability to perform validation. One solution would be to consider the final customer as a subcontractor for the purpose of design validation. This would require customer agreement. A second solution would be to identify verification testing that is logically linked to the specific design characteristics that require validation. Establish a test series beyond verification testing that logically would cause you to have confidence that the usage requirements can be met. This approach is also difficult, because, strictly speaking, it only approximates validation. You will need to convince the registrar that this is appropriate and all that is possible.

I believe most of the confusion between verification and validation does not result from a lack of understanding of the difference as much as the difficulty of implementing appropriate validation in some circumstances.

[tvn]

Verification
Are we building the product right?

Validation
Are we building the right product?

Following is some discussion about Verification and Validation

VER: Look at the situation like this.
For any activity in a project, there is an input as what need to be achieved; there is a process/way of working to be followed and an Output. Verification is ANY activity that checks the Output against the Input specification. Any work product can be verified. A design doc is trying to achieve what the FRS wants, a piece of code is what the detailed design wants, review is an effective means of work items verification. Requirement based testing is a Verification of FRS as functional level.

VAL:
Here the focus is on USER Needs and the user's Environment. Any activity done with the product/components for the User needs so that the product will work in the user environment is a Validation activity.
Beta testing a well known Validation activity. High level requirements and prototype review by end customers is another instance of Validation. High level requirements testing by application specialists and end users is another Validation activity.

IEEE 1012 explains it nicely.

~~~

Verification: Testing of work products done internally
Verification: Testing of work products done by the customer

~~~

Verification always takes place before the process of validation.

>>Verification: Testing of work products done internally

In the above statement 'Internally' means work done before starting actual product testing, i.e. some work which is internal to the requirementgathering/development/coding phase. It will include evaluation and review of documents, plans, code, requirements, and specifications. This evaluation is done by using various metrices, checklists, walkthroughs, inspections and review meetings. Verification is also called static testing because no software program is executed to check it. Eg: Each requirement specification is checked for correctness, feasibility, testability etc. If some fault is identified, then it is rectified at this requirements stage itself. This ensures that the next phase of 'designing' will not be built on faulty requirements.

>>Validation: Testing of work products done by the customer

Customer may or may not be involved in Validation process. The statement actually means that the validation is a kind of actual testing of the product in the way the customer would use it. It includes evaluating the whole software and checking that each functionality works as expected by the customer. Validation is also called dynamic testing as in this the software is tested in running mode. Eg: A user should be able to register himself on a website by providing a valid username and email id. If the website does not behave as per requirement specification, then it is a defect.

~~~

"Customer may or may not be involved in Validation process. The statement actually means that the validation is a kind of actual testing of the product in the way the customer would use it. It includes evaluating the whole software and checking that each functionality works as expected by the customer. "

When doing those tests, how do you know if the program is doing what it was supposed to do by the customer, when the customer is not involved? Validation is getting the customer to confirm by asking if the actions taken and the results returned are valid. So the process of validating is asking the customer: "I did this, and the result is that. Is this what you expected it to do?". If the customer confirms (or disconfirms) it, you validated it.

Comparing to written down results, like requirements, doesn't hold the same value as getting it validated by the customer. A requirement is still an interpretation of a statement made at a certain point in time, in a certain context (and most likely said by the customer). When there is room for the slightest misconception, the results may differ from what the customer intended it to be. If you would validate the results against requirements or any form of written down expectations, there is still a possibility you would confirm (or disconfirm) the result. Even if the customer would not.

~~~

Verification: Testing of work products done internally
>> Testing is one of the means of Verification. There is review/inspection which is another important method.

Validation: Testing of work products done by the customer
>> May be by the customer or may not be. For example, when you sell a MRI machine, customer uses it only after the purchase. The thing to keep in mind is - Meeting Customer NEEDS in customer like environment. Any activity done for this purpose is Validation.

The problem is that since few of the authorities responsible for teaching this are not giving a standard answer, we are not sure who to believe when referring to?

[tvn]

Objectives

• To introduce software verification and validation and to discuss the distinction between them
  To describe the program inspection process and its role in V & V
  To explain static analysis as a verification technique
  To describe the Cleanroom software development process

...

Detail:

Verification and validation.zip

[hoccachhoc]

* Kiểm chứng (Verification):

• + Có đúng đặc tả không? có đúng thiết không?
  + Phát hiện lỗi lập trình

* Thẩm định (Validation):

• + Có đáp ứng nhu cầu người dùng không?
  + Có hoạt động hiệu quả không?
  + Phát hiện lỗi phân tích, lỗi thiết kế (lỗi mức cao)

* V&V=Verification vs Validation

• + Mục điêu là phát hiện và sửa lỗi PM, đánh giá tính dùng được của PM
  + Thứ tự thực hiện: Verification --> Validation