Hierarchy definition

Normally, an investigator (facilitator), often supported by a focus group, divides the decisional problem in a set of criteria and sub-criteria and organized them into meaningful hierarchy.

The facilitator, basing on his experience of AHP, design the hierarchy, which is then reviewed with the other participants to check if it is accurate and comprehensive.

Questionnaires

Questionnaires are then designed to enable each respondent to compare the relative importance of each sub-criteria with all of the other sub-criteria within the same category.

For each pair of sub-criteria (i,j), responders will be asked the following question: “according to your experience, how important do you consider the sub-criteria i compared to the sub-criteria j?”. Responders is asked to answer by choosing one of the following judgments:

• extremely less important,
• much less important,
• less important,
• equally important,
• more important,
• much more important,
• extremely more important.

In accordance with the Saaty natural scale, an integer numerical value is given to each judgment: 1 if equally, 3 if more important, 5 if much more important, and 7 is extremely more important. The reciprocal values is given to the remaining judgments: 1/3 if less important, 1/5 if much less important and 1/7 if extremely less important. In-between numbers can be used for in-between judgments. Although several scales have been proposed for this process, the Saaty natural scale is often used as it is easier to understand for responders who are not skilled in complex mathematics or with the AHP method.

The process is then repeated, designing similar questionnaires to elicit the relative importance of each criteria.

Judgment matrix

For each criteria, a judgment matrix A_nxn is computed, where “n” is the number of sub-criteria included in this category. According to the Saaty theory, it is possible to prove that each matrix has the following properties:

• The generic element (a_ij) referred to the ratio between the relative importance of the sub-criteria “i” (SC_i) and “j” (SC_j);
• The element a_ji is the reciprocal of a_ij, assuming the reciprocity of judgment (if SC_i is 3 times more important than SC_j, then SC_j should be 1/3 of SC_i);
• The element a_ii is equal to 1 (SC_i is equal in importance to itself);
• The matrix A is assumed to be a transitive matrix, which means that “∀ i, j, k ∈ (1; n), a_ij = a_ik * a_kj” by definition of a_ij (see eq. 1).

This last property is called the transitivity property and reflects the idea that if “i” is considered twice as important as j (SC_i= a_ij * SC_j), and “j” is considered three times more important than “k” (SC_j= a_jk * SC_k), then “i” should be judged six times (two times three) more important than “k” (SC_i = a_ik * SC_k, with a_ik=a_ij* a_jk).

Local weights: the relative importance of sub-criteria within each criteria

It has been proved that, if a matrix A satisfies the properties described in section above then each column is proportional to the others and only one real eigenvalue (λ) exists, which is equal to “n”. The eigenvector associated with this eigenvalue is again proportional to each column, and represents the relative importance of each sub-criteria compared to each of the other sub-criteria in the same category. The relative importance (weight) of a sub-criteria i within the category m will be further recalled as LW_i^m or local weight.

In cases where the judgments are not fully consistent, the columns of the matrix are not proportional to one another. Thus, the matrix has more eigenvectors and none are proportional to all the columns. In this case, the main eigenvector, which is the one corresponding to the largest eigenvalue (λ_max), is chosen. Its normalized components represent the relative importance of each sub-criteria.

Consistency estimation

If the transitivity property is not respected, an inconsistency will be generated. This inconsistency is estimated by posing some redundant questions. Considering three sub-criteria (i, j, and k) the respondent is asked to perform the pair comparisons i–j and j–k, and then the redundant comparison i–k. The answer to the redundant question is compared with the one deduced from the first two, assuming the transitivity of judgment. The difference between the real answer and the transitive one represents the degree of inconsistency. The global effect of this inconsistency is estimated by measuring the difference between the major eigenvalue λ_max and “n”. The error is zero when the framework is completely consistent. Inconsistency is, in the majority of cases, due to loss of interest or distraction. If inconsistency occurs, the responders are required to answer the questionnaire again. Some inconsistency between responses is expected; using a scale of natural numbers will cause some systemic inconsistency because not all the ratios can be represented and because of the limited upper value (e.g. 3*2 gives 6, but the maximum value in the scale is 5). For this reason, an error less than a certain threshold is accepted in accordance with the literature. An error over this threshold should be considered too high for reliable decisions.

At each node, the responders’ consistence is estimated measuring the difference of the eigenvalue λ_max from “n” (number of elements in the node), normalized to “n”. This is defined as the consistency index (CI), and is zero when the framework is completely consistent (λ_max=n). According to literature, the CI is divided by the Random Consistency Index (R.I.), which is a tabled [57] value changing for n from 1 to 9. This ratio is called Consistency Ratio (CR=CI/CR) and a threshold of CR≤ 0.1 is generally considered appropriate, although some authors have proved that it is possible to increase this threshold to 0.2 when the hierarchy is complex and it is not practical for the responders to discuss the questionnaire results.

Critarie importance per responder

By applying the same algorithm to the criteria it is possible to evaluate their relative importance. The relative importance of a category m will be further recalled as criteria importance (weight) or Categorical Weight (CW^m).

Global-importance of each sub-criteria per responder

Finally, the relative importance of a sub-criteria i compared to all the others (not only those in the same category) is defined as global-importance (Global-Weight) of the sub-criteria i (GWi). GWs are calculated by multiplying the local (within category) importance of the sub-criteria by the importance of the root element (criteria) into the Hierarchy. For instance the global-weight of the sub-criteria i, which is in the category m, is calculated as the product of the local importance of the sub-criteria (LWik) and the importance of its category m (CWm) (Eq2).

User feedback

Finally, to fully understand the reasons behind the sub-criteria prioritization, the results obtained are discussed with the responders and with other domain experts.