With pairwise comparison, aka paired comparison analysis, you compare your options in pairs and then sum up the scores to calculate which one you prefer. In determining the criteria, the criteria and options should not be increased in their numbers, of course there are lots of pairwise comparisons which can lead to incompatibility. Data Format. If you need to handle a complete decision hierarchy, group inputs and alternative evaluation, useAHP-OS. Beginning Steps. Pairwise Comparison is a common research technique utilized by technology startups. InternationalJournal of Uncertainty, Fuzziness and Knowledge based systems, Vol 14, No 4, 445-459. You can use the following excel template for the same calculation as shown with this online tool. Disclaimer: artikel ini dibagi menjadi dua bagian, bagian pertama menjelaskan mengenai pairwise comparison in general dan bagian kedua menjelaskan cara menyusun pairwise comparison matrix Pairwise comparison atau perbandingan berpasangan adalah setiap proses membandingkan entitas berpasangan untuk menilai entitas mana yang lebih disukai atau memiliki jumlah properti kuantitatif yang lebih . In this study, the effect of different types of smiles on the leniency shown to a person was investigated. Edit Conditions. Note: Use calculator on other tabs for more or less than 4 candidates. common Pairwise Comparison technique is described below, followed by a description of the modifications applicable to each use. Rather than asking participants to vote on every possible head-to-head comparison, probabilistic pairwise comparison asks for a much smaller sample of pair votes and uses data science techniques to predict the answer that would have been given for the pairs that didnt get voted on. Slightly modify your comparisons, if you want to improve consistency, andrecalculatethe result, ordownloadthe result as a csv file. It is equal to \(2.65\). After the result is known, the following formulae are used to update the scores of each option: rating1 = rating1 + K*(Actual Expected); rating2 = rating2 + K*(Actual Expected); Kfactor = 32 (default number for Chess which can be altered). There are two types of Pairwise Comparison: Complete and Probabilistic. Consider the first row "Cost" and get the product of the values of this row. You can calculate the total number of pairwise comparisons using a simple formula: n (n-1)/2, where n is the number of options. Note: Use calculator on other tabs for more than 3 candidates. With pairwise comparison, aka paired comparison analysis, you compare your options in pairs and then sum up the scores to calculate which one you prefer. 'Pairwise Won-Loss Pct.' ^ The expected score of option1 and option2, respectively. We use Mailchimp as our marketing platform. comparisons to calculate priorities using 4) Cost. The assumptions of the Tukey test are essentially the same as for an independent-groups t test: normality, homogeneity of variance, and independent observations. The LibreTexts libraries arePowered by NICE CXone Expertand are supported by the Department of Education Open Textbook Pilot Project, the UC Davis Office of the Provost, the UC Davis Library, the California State University Affordable Learning Solutions Program, and Merlot. Pairwise comparisons simplified. Complete the Preference Summary with 3 candidate options and up to 6 ballot variations. As of 2022-23, OTs are all 3-on-3, and thus an OT win is only counted as 0.6666 of a win, and 0.3333 of a loss. Similarly, the non-significant difference between the miserable smile and the control does not mean that they are the same. This comparison ought to be predicted in the survey and in the analysis of the outputs data. ), Complete the Preference Summary with 6 candidate options and up to 10 ballot variations. Pairwise comparison of the criteria. Francisco used this data to calculate the financial impact of each segments top problem so that he could pick which one to focus on solving first. Complete each column by ranking the candidates from 1 to 6 and entering the number of ballots of each variation in the top row (0 is acceptable). Weighting by pairwise comparison. Enter the elements or criteria you want to compare in the field below, separated by commas. The more preferred candidate is awarded 1 point. Tournament Bracket/Info The left side of the above figure shows the original pairwise comparison matrix. It definitely gives us more confidence in our roadmap planning.". Pickedshares.com sends out newsletters regularly (1-4 times per month) by email. This will create filters for each column that you can select in the top row. Input number and names (2 - 20) OK Pairwise Comparison 3 pairwise comparison(s). The Pairwise Comparison Matrix and Points Tally will populate automatically. History, NCHC Most of us would agree that weighting of label appeal as the drinker of the beer would not be very important. In the General tab, select the Taste and Sweetness columns as dependent variables, and the Panelist and Product columns as explanatory qualitative variables. I would suggest csv format, as I can just drag and drop it onto QGIS window. Working with pairwise comparison tool is very simple: 2. For this experiment, \(df = 136 - 4 = 132\). If you are referring to some other kind of "PairWise comparisons," please. Please input the size of Pairwise Comparison Matrix ( the number of evaluation items or evaluation objects), n where 2 n 9. But even more commonly, its that our participants are better are picking the words that truly represent the problems, pain points and priorities they intimately know best. The more means that are compared, the more the Type I error rate is inflated. It reformatted how we thought about our whole approach Who knows where this project would have ended up if we didn't know about OpinionX." is the team's winning percentage when factoring that OTs (3-on-3) now only count as 2/3 win and 1/3 loss. false vs felt. Open the XLSTAT menu and click on XLSTAT-Modeling data / ANOVA . Compute the degrees of freedom error (\(dfe)\) by subtracting the number of groups (\(k\)) from the total number of observations (\(N\)). Enjoy using our free tool. AHP Criteria. Six Comparisons among Means. While the sliders are being set, a ranking list appears below, in which the weighting of the individual criteria is displayed. (Note: Use calculator on other tabs for more or less than 8 candidates. To do this, they are entered in the input field of the online tool for pairwise comparison. Therefore, \[dfe = N - k\], Compute \(MSE\) by dividing \(SSE\) by \(dfe\):\[MSE = \frac{SSE}{dfe}\]. Change the weightings here as you see fit. - Podcasts, Radio, Live Streams, TourneyWatch: All the Latest Articles and More, Atlantic Hockey = .05) then we . Based on these priorities, it is the car Element which seems to answer the problem. The Tukey HSD is based on a variation of the \(t\) distribution that takes into account the number of means being compared. History, CCHA Language: English In order to be able to make this decision, a benefit analysis is prepared. Within two or three weeks of launching a new roadmap, we're focused on the next one. The Pairwise Comparison Matrix, and Points Tally will populate automatically. In Excel 2008, choose Data | Data Analysis | . Business Performance Management Singapore, Subscribe to Newsfeed Tukey's Test Need Not be a Follow-Up to ANOVA. the false smile is the same as the miserable smile, the miserable smile is the same as the neutral control, and. Our breakthrough genome editing technologies let us bring exciting new products to market that are more enticing, more convenient and more likely to . What are you trying to use your pairwise comparison research to understand? Pairwise Comparison Ratings. Calculate priorities from pairwise comparisons using the analytic hierarchy process (AHP) with eigen vector method. But there was a problem; Francisco couldnt spot a clear pattern in the needs that customers were telling him about during these interviews. Therefore, if you were using the \(0.05\) significance level, the probability that you would make a Type I error on at least one of these comparisons is greater than \(0.05\). In the SpiceLogic ahp-software, whenever you perform a pairwise comparison or view the pairwise comparison matrix, you will notice the consistency ratio for that set of comparisons calculated and displayed at the bottom as shown below. Thanks to J-Walk for the terminology "Pairwise Comparison". Occasion: using a specific event or recurring circumstance to understand the needs that extend beyond product offerings (eg. difficulties running performance reviews). Having spent the last few years designing and managing hundreds of Pairwise Comparison projects for clients ranging from early-stage startup founders and product teams at scaling tech companies to government leaders and social scientists, Ive seen some really interesting research approaches. Tournament Bracket/Info We're here to change the story of fruits and vegetables by making them the most irresistible food on the planet. Note: Use calculator on other tabs for more or less than 5 candidates. By moving the slider you can now determine which criterion is more important in each direct comparison. 1) Less filling. When we ran our OpinionX survey, it came back as the most frustrating part for people. According to the Saaty scale, this means that the cost is judged to be very important compared to the style criterion. But that final step threw them quite the curveball "[Before our Pairwise Comparison study,] all of our other data was pointing to stuff at other points in the journey. challenges that arise at the financial year-end). A single word or phrase can change the entire meaning of the statement. Pairwise comparison, or "PC", is a technique to help you make this type of choice. For example, with a frustration ranking criterion and collaborating with teammates on our product as our activity of focus, we get the question Which option is more frustrating when trying to collaborate with teammates on our product?, This example is suited for a Pair Rank project, whereas an Order Rank question might start instead with Rank the options from most to least frustrating when trying to collaborate with teammates on our product.. Waldemar W Koczkodaj. The best research projects use Pairwise Comparison as the middle step of a broader discovery project. The XLSTAT AHP feature offers the possibility to test the data consistency by calculating two parameters: the index of coherence and the ratio of coherence. So, finalize the table before. 1) Though the maximum number of criteria is 15, you should always try to structure your decision problem in a way that the number of criteria is in the range 5 to 9. Regarding the math. To continue we take the weighted average of the columns of the original pairwise comparison matrix using the new weights: Next estimate. This procedure will be described in detail in a later chapter. ), Complete the Preference Summary with 10 candidate options and up to 10 ballot variations. In the above formulae, E(A) is equivalent to our E1 and R(A) is equivalent to our rating1. 3) Can or bottle. Micah Rembrandt, Senior Product Manager at Animoto. And should not carry as significant a ranking as, say, tastes great. In order to determine which groups are different from one another, a post-hoc test is needed. This tool awards two point to to the more important criteria in the individual comparison. He decided to run a quick Pairwise Comparison survey on OpinionX to add some measurable data to this unclear picture. Before I met the Kristina, the Gnosis Safe had a "pretty lengthy process" to decide on what they would prioritize each quarter: "We would look through our internal user research database and say, 'ok, I saw people mention X or Y more often, this seems like a big issue.' regards, Klaus, AHP Online Calculator Update 2013-12-20, New AHP Excel template with multiple inputs, Line 1: Date (yyyy-mm-dd)Time (hh:mm:ss) Title (text), Last line: eigenvalue and consistency ratio CR. Its just too much to take in, in my experience, so we wouldn't have done it given the scope and timing of this project. Micah Rembrandt, Sr. PM at Animoto. The Analysis ToolPak is an add-in provided on the Office/Excel installation. Moreover, for a consistent pairwise comparison matrix, it is well known, see e.g., , that the priority vector satisfying can be generated by either EVM or by GMM. Because Probabilistic Pairwise Comparisons use samples of the total options list, we can add new options to the list as we go. Not only would this be an extremely time-consuming and repetitive process, it also collects a lot more data than we actually need. If we had three conditions, this would work out as 3(3-1)/2 = 3, and these pairwise comparisons would be Gap 1 vs .Gap 2, Gap 1 vs. Gap 3, and Gap 2 vs. Grp3. Three are three different approaches you can take to run a Pairwise Comparison study and calculate your ranked results: Unless youre an Excel whizz, this approach only works for small, simple projects or childrens math class assignments. Do not use simple thing in the spectra of the question. Complete each column by ranking the candidates from 1 to 4 and entering the number of ballots of each variation in the top row (0 is acceptable). Learn more about Mailchimp's privacy practices here. Probabilistic Pairwise Comparison combines transitivity together with pattern recognition so that each participant only has to vote on a tiny sample just 10 to 20 pairs and then an algorithm analyzes the voting patterns over time to build a confidence model of how each opinion ranks in comparison to each other. Input the number of criteria between 2 and 20 1) and a name for each criterion. Note: This chart is updated as each game result comes in. Its lightweight, requiring just a handful of simple head-to-head votes from participants which are pretty low in cognitive load. For example, check out this detailed explanation of how multiple algorithms work together to power Probabilistic Pairwise Comparison on OpinionX. Input: Size of Pairwise Comparison Matrix; Input: Pairwise Comparison Matrix (The values of Pairwise Comparison) Display: Weights (Eigen Vector) and CI (Eigen Value) Output: Text File. To do that, participants need the same frame of context for considering each option. This generally takes the form of an activity of focus the overall action or objective that serves as context for participants when interpreting the options in your pairwise comparison list. The data is grouped in a table as follows: We are ready to proceed to convert the matrix to a pairwise column. > #read the dataset into an R variable using the read.csv (file) function. But sometimes we have a lot of options to compare, like 50+ different problem statements or 100+ different crowdsourced feature ideas. To compute pairwise op you can do the following trick: expand the vector to two 2-dimensional vectors: [n, 1] and [1, n], and apply the op to them. Kindly rate the software from 1 star (poor) to 5 stars (excellent) at the bottom of this post. The Pairwise Comparison Matrix and Points Tally will populate automatically. The Pairwise Comparison Matrix and Points Tally will populate automatically. B wins the pairwise comparison and gets 1 point. . This procedure would lead to the six comparisons shown in Table 1. In reality, the complexity of manually calculating the results of Pairwise Comparison studies means that most people dont end up using Pairwise Comparison as a research method at all. So instead of skipping over this step of his research, he used a Stack Ranking Survey to get the best of Pairwise Comparison without the complex analysis. For example, how important the criterion A is for you? Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. OpinionX does this for you by calculating the personal stack rank of each participant so that you can compare it to the overall results and pick the right interviewee with ease. 5- Strong importance, 7- Very strong importance, 9- Extreme importance ), Complete the Preference Summary with 5 candidate options and up to 10 ballot variations. To counteract this, the best Pairwise Comparison studies use simple multiple-choice questions to gather demographic data on participants like their gender, age, location or job title. These newsletters contain information about new content on pickedshares.com, thematically relevant information and advertising. disclaimer: artikel ini merupakan bagian kedua dari topik pairwise comparison, sebelum membaca artikel ini, diharapkan Anda membaca bagian pertama dengan judul: Pairwise Comparison in General Pada artikel sebelumnya, kita sudah membahas mengenai pengertian dan manfaat pairwise comparison serta langkah-langkah dalam melakukan Analytical Hierarchy Process. The pairwise comparison method (sometimes called the ' paired comparison method') is a process for ranking or choosing from a group of alternatives by comparing them against each other in pairs, i.e. Current Report The first results are tables and graphs presenting the mean values of the results obtained by the evaluator. Espaol The AHP feature proposed in XLSTAT has the advantage of not having any limitations on the number of criteria, of subcriteria and of alternatives and allows the participation of a large number of evaluators. The Method of Pairwise Comparisons Denition (The Method of Pairwise Comparisons) By themethod of pairwise comparisons, each voter ranks the candidates. Current Report The degrees of freedom is equal to the total number of observations minus the number of means. Pairwise comparison (also known as paired comparison) is a powerful and simple tool for prioritizing and ranking multiple options relative to each other. For example, Owen has evaluated the cost versus the style at 7. Pairwise Comparison is uniquely suited for informing complex decisions where there are many options to be considered. 0. Pairwise Comparisons Method . The Type I error rate can be controlled using a test called the Tukey Honestly Significant Difference test or Tukey HSD for short. Complete each column by ranking the candidates from 1 to 3 and entering the number of ballots of each variation in the top row (0 is acceptable). Pairwise: How Does it Work? In the General tab, choose a worksheet that contains a DHP design generated by XLSTAT, here AHP design. The Pairwise Comparison Matrix, and Points Tally will populate automatically. Current Report Next, do a pairwise comparison: Which of the criterion in each pair is more important, and how many times more, on a one to nine scale. Complete each column by ranking the candidates from 1 to 8 and entering the number of ballots of each variation in the top row (0 is acceptable). Best of all, its completely free to create a stack ranking survey. I created a guide to writing seeded options for some of the most common types of Pairwise Comparison studies, check it out for some inspiration. Legal. But opting out of some of these cookies may affect your browsing experience. Check out the full story to see how we did that. Instructions: On the "AHP Template" worksheet, select the number of criteria that you would like to rank (3 to 15) Enter the names of the criteria/requirements and a title for the analysis. With Check consistency you will then get the resulting priorities, their ranking, and a consistency ratio CR2) (ideally < 10%). If youre working with larger option sets or participant populations and still need to do calculations manually, I would recommend using an ELO Rating Algorithm. feature. Thousands of gyms around the world, from small family studios to national franchises, use Glofox to schedule classes, manage memberships, track attendance rates, automate payments, and more. For example, if the ratio of coherence is greater than 10% then it is recommended to review the evaluation of the comparison table concerned. The Pairwise Comparison Matrix and Points Tally will populate automatically. Complete each column by ranking the candidates from 1 to 5 and entering the number of ballots of each variation in the top row (0 is acceptable). (Ranking Candidate X higher can only help X in pairwise comparisons.) This page titled 12.5: Pairwise Comparisons is shared under a Public Domain license and was authored, remixed, and/or curated by David Lane via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request. If you or your instructor do not wish to take our word for this, see the excellent article on this and other issues in statistical analysis by Leland Wilkinson and the APA Board of Scientific Affairs' Task Force on Statistical Inference, published in the American Psychologist, August 1999, Vol. Before we started working together, Micahs team felt like they had understood the most important unmet needs and decided to run a quick stack ranking survey to validate their findings before moving on. However, the probabilistic method is often the most accessible. So in just one evening, we found 150 participants through Slack communities to participate for free in a quick Pairwise Comparison survey to stack rank 45 different problem statements. ; H A: Not all group means are equal. Plot. Pairwise comparison, or "PC", is a technique to help you make this type of choice. As the team completes each of the comparisons, the number of the preferred item is recorded in that square, until the matrix is completely filled in. Imagine a person is being asked to vote on three pairs consisting of Option A, B and C. If the person prefers A over B and also B over C. We wouldnt need to ask someone if they prefer Option A over Option C, instead we can just infer this. Once the entities are compiled into a group, the decision-makers run through all possible pairsgenerally ranking alternatives against each other . 3:Input: Pairwise Comparison Matrix Input the Pairwise Comparison Matrix; Do not use fractions; You can use negative number -a ij instead of fraction 1 / a ij; Example: 1/3 -3, 1/2.8 -2.8; Output Fig.4: Output C.I. AHP is a decision aid method based on a criteria hierarchization. { "12.01:_Testing_a_Single_Mean" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.02:_t_Distribution_Demo" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.03:_Difference_between_Two_Means" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.04:_Robustness_Simulation" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.05:_Pairwise_Comparisons" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.06:_Specific_Comparisons" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.07:_Correlated_Pairs" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.08:_Correlated_t_Simulation" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.09:_Specific_Comparisons_(Correlated_Observations)" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.10:_Pairwise_(Correlated)" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.11:_Statistical_Literacy" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12.E:_Tests_of_Means_(Exercises)" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()" }, { "00:_Front_Matter" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "01:_Introduction_to_Statistics" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "02:_Graphing_Distributions" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "03:_Summarizing_Distributions" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "04:_Describing_Bivariate_Data" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "05:_Probability" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "06:_Research_Design" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "07:_Normal_Distribution" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "08:_Advanced_Graphs" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "09:_Sampling_Distributions" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "10:_Estimation" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "11:_Logic_of_Hypothesis_Testing" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "12:_Tests_of_Means" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "13:_Power" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "14:_Regression" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "15:_Analysis_of_Variance" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "16:_Transformations" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "17:_Chi_Square" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "18:_Distribution-Free_Tests" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "19:_Effect_Size" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "20:_Case_Studies" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "21:_Calculators" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()", "zz:_Back_Matter" : "property get [Map MindTouch.Deki.Logic.ExtensionProcessorQueryProvider+<>c__DisplayClass228_0.b__1]()" }, [ "article:topic", "pairwise comparison", "Honestly Significant Difference test", "authorname:laned", "showtoc:no", "license:publicdomain", "source@https://onlinestatbook.com" ], https://stats.libretexts.org/@app/auth/3/login?returnto=https%3A%2F%2Fstats.libretexts.org%2FBookshelves%2FIntroductory_Statistics%2FBook%253A_Introductory_Statistics_(Lane)%2F12%253A_Tests_of_Means%2F12.05%253A_Pairwise_Comparisons, \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}}}\) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\), The Tukey Honestly Significant Difference Test, Computations for Unequal Sample Sizes (optional), status page at https://status.libretexts.org, Describe the problem with doing \(t\) tests among all pairs of means, Explain why the Tukey test should not necessarily be considered a follow-up test.