Monday, April 4, 2011

Shifting the Business School Hierarchy Part II:
The Real MBA Rankings behind the Financial Times and  
US News & World Report Figures

By Philippe RUIZ, PhD and Benjamin GOURMEL, PhDc

The present article is an extension of the one published on January 21st, 2011 and titled: Shifting the Business School Hierarchy: The Real Rankings behind the Financial Times Figures. We strongly recommend our readers to read the previous article before reading this one, in particular the first four parts.
For this second issue, we have decided to scrutinize the rankings of MBA courses. Indeed, the MBA, despite recent criticism regarding its lack of specialization and relevance to the actual conduct of business, remains unquestionably the hallmark of higher business education. Besides, it is also one of the first degrees that went through the process of ranking in higher education. Therefore it seems natural to review the current ranking methodologies and to apply some sound statistics to the existing publications.
To compute the present rankings, we use the Financial Times’ (FT) MBA 2011 figures (collected yearly and published in January) as it provides the most commonly agreed upon criteria on the international scene. Therefore, no new underlying data are provided here, only differences in statistical analysis distinguish our work. We also compare our results with the latest US News & World Report (USN) findings (published in March 2011).

1- Method
The FT and USN rankings are based on the weighted average of different criteria such as salary after graduation or the percentage of international students. A weighted average is simply an average in which each quantity to be averaged is assigned a weight. These weightings determine the relative importance of each quantity on the average.
There are some problems with the weighted average that trained statisticians are aware of. The first is whether it is possible or not to add some quantities to obtain a meaningful sum. An example of a meaningless sum would be the sum obtained from the addition of 1) the number of hair on my neighbor’s head, 2) the number of trains leaving my local train station on a Sunday and, 3) the number of stars visible from my bedroom window on a clear summer night. The obtained sum is meaningless because the three numbers have nothing in common. Only things that are more or less related can be added to obtain a meaningful sum or an average. An example of an average of similar quantities is at school, when the overall average of all the courses followed by the students is obtained. In a typical business school, a Grade Point Average (GPA) is thus calculated. The addition of all the grades from different disciplines such as Marketing, Finance or Organizational Behavior is possible because those subjects somehow measure something in common. If they were not related at all, it would be impossible to compute a meaningful average.
Statisticians use the word “correlated” to indicate that two or more variables have tendency to “move together.” An example of two correlated variables in the human population is height and weight. Taller people have a tendency to be heavier (children are smaller and thus much lighter than adults) even if this relationship is not always true (there are some tall people who are lighter than some shorter people, but the bigger the difference in height, the less likely this is to be true). It is said that height and weight are positively correlated (i.e. people who are taller are usually heavier). When two variables are correlated, it is often possible to add them. However, when two variables are not correlated, it is never possible to add them to obtain a meaningful sum or an average. So the very first thing statisticians do before computing an average is to calculate the “coefficients of correlation” between the variables to find out whether it is possible or not to add them. For example, the Employed at 3 months (%) variable used in the FT ranking is absolutely not correlated with the Weighted salary (US$) variable and therefore the two quantities should never be added, even it appears “intuitively” that the two variables should be related.
Once it has been established that the variables are sufficiently correlated and that their average can be computed, another problem to address is what weights should be attributed to each variable. The FT and USN journalists seem to believe that their educated opinion can do the job, but once again, this is at the expense of sound statistical practice. In order to establish the weights when many variables are available, the correlations with all the other variables in the analysis should be obtained first by using a technique called Factor Analysis (FA). The purpose of FA is to replace a given number of correlated variables by a smaller number of uncorrelated factors. FA is an advanced technique but the basic principles are easy to understand. This technique allows to determine “trends” in variables that are interrelated, and to aggregate them into factors. The strength of the links of each variable with all the others will indicate its weight and importance in the study.
Going back to our school example above, when the GPA is obtained for the students, the GPA can be considered to be a factor because it integrates all the original variables; it gives an overall indication of the scholastic level of the students. This indicator or factor is good only if it has strong links to the original variables (strong correlations). Some courses may be more highly related to it (they may have strong correlations with the GPA) while others may be only vaguely related to it (they may have low correlations with the GPA). If there are many courses highly related to it then the factor is strong, if not, the value of the factor is dubious.
Let’s now suppose that Marketing has a strong relationship with the GPA (we are just making this up) and Finance has a weak relationship with the GPA, then marketing should be considered more important than finance and therefore it should be weighted more in the average than finance.
To sum up, if the original data of the FT or USN can be considered relevant and reliable, there are two main problems with these journals’ number crunching that make their study statistically mistaken and their rankings incorrect: 1) they did not check whether the numbers obtained could be added (we found that most of the variables do not share enough in common to compute an overall mean) and,  2) the weights they have picked are arbitrary and subjective, not the outcome of a mathematically objective optimization procedure.

2- Results
We have thus applied correlation and factor analysis to their data and we have discovered two main dimensions, not just one as believed by the FT or USN. It is impossible to add those dimensions/ factors because they are uncorrelated (they share nothing in common). We are now going to explain them and extract the correct corresponding rankings.
The first factor includes the following variables (in decreasing order of importance): Alumni recommend rank, Weighted salary (US$), Placement success rank, FT research rank, Aims achieved rank, and Value for money rank. This factor reflects what we traditionally mean by business school quality: added value or return on investment (ROI) but also the reputation of the school and the quality of its research.


Statistical Ranking of the Financial Times MBA 2011 (in decreasing order of ROI score) 
School name ROI score ROI rank FT rank ROI-FT rank difference USN rank ROI-USN rank difference*
Stanford University GSB 71,55 1 4 -3 1 0
University of Pennsylvania: Wharton 70,13 2 1 1 3 -1
Northwestern University: Kellogg 68,92 3 21 -18 5 -2
Columbia Business School 68,77 4 7 -3 9 -5
University of Chicago: Booth 67,74 5 12 -7 5 0
Dartmouth College: Tuck 66,53 6 18 -12 7 -1
New York University: Stern 66,25 7 15 -8 10 -3
MIT Sloan School of Management 65,82 8 9 -1 3 5
University of California at Berkeley: Haas 65,63 9 25 -16 7 2
London Business School 65,38 10 1 9 n/a n/a
Duke University: Fuqua 65,35 11 20 -9 12 -2
Harvard Business School 64,06 12 3 9 2 9
Yale School of Management 63,30 13 15 -2 10 2
University of Michigan: Ross 62,91 14 24 -10 14 -1
Insead 62,75 15 4 11 n/a n/a
IMD 62,24 16 14 2 n/a n/a
Cornell University: Johnson 61,63 17 30 -13 16 -2
Iese Business School 60,48 18 9 9 n/a n/a
University of Virginia: Darden 60,37 19 41 -22 13 2
UCLA: Anderson 59,82 20 31 -11 14 2
Carnegie Mellon: Tepper 58,45 21 41 -20 18 -1
Vanderbilt University: Owen 57,93 22 51 -29 28 -10
Georgetown University: McDonough 57,54 23 38 -15 25 -6
University of Texas at Austin: McCombs 57,43 24 49 -25 17 3
Emory University: Goizueta 57,26 25 38 -13 23 -2
HEC Paris 57,26 26 18 8 n/a n/a
Indian Institute of Management, Ahmedabad (IIMA) 56,16 27 11 16 n/a n/a
University of Southern California: Marshall 55,87 28 64 -36 21 1
Rice University: Jones 55,37 29 44 -15 34 -11
Indian School of Business 54,77 30 13 17 n/a n/a
Pennsylvania State University: Smeal 54,44 31 59 -28 40 -16
Indiana University: Kelley 54,18 32 73 -41 23 2
University of Cambridge: Judge 54,11 33 26 7 n/a n/a
University of North Carolina: Kenan-Flagler 53,92 34 62 -28 19 7
Hong Kong UST Business School 53,59 35 6 29 n/a n/a
Ceibs 53,01 36 17 19 n/a n/a
Georgia Institute of Technology 52,81 37 97 -60 28 -1
University of Western Ontario: Ivey 52,57 38 46 -8 n/a n/a
IE Business School 51,53 39 8 31 n/a n/a
University of Oxford: Saïd 50,89 40 27 13 n/a n/a
SMU: Cox 50,67 41 88 -47 57 -29
Rotterdam School of Management, Erasmus University 50,34 42 36 6 n/a n/a
Cranfield School of Management 50,10 43 34 9 n/a n/a
University of Washington Business School: Foster 49,91 44 86 -42 37 -8
Boston College: Carroll 49,57 45 74 -29 34 -4
University of Toronto: Rotman 49,47 46 46 0 n/a n/a
Arizona State University: Carey 49,29 47 64 -17 27 4
Esade Business School 49,19 48 21 27 n/a n/a
Wisconsin School of Business 49,05 49 63 -14 28 4
Australian School of Business: AGSM 48,96 50 35 15 n/a n/a
Manchester Business School 48,69 51 29 22 n/a n/a
Ohio State University: Fisher 48,58 52 72 -20 25 8
University of Maryland: Smith 48,43 53 40 13 45 -11
Brigham Young University: Marriott 48,43 54 91 -37 32 3
Imperial College Business School 48,13 55 37 18 n/a n/a
Texas A & M University: Mays 48,06 56 44 12 32 4
Boston University School of Management 48,03 57 68 -11 34 3
University of British Columbia: Sauder 47,96 58 80 -22 n/a n/a
University of Notre Dame: Mendoza 47,52 59 80 -21 37 1
Babson College: Olin 47,28 60 84 -24 52 -13
University of Illinois at Urbana-Champaign 47,04 61 46 15 37 3
City University: Cass 47,03 62 32 30 n/a n/a
National University of Singapore School of Business 46,95 63 23 40 n/a n/a
York University: Schulich 46,84 64 49 15 n/a n/a
Thunderbird School of Global Management 46,68 65 68 -3 75 -34
Kaist College of Business 46,32 66 99 -33 n/a n/a
University of Rochester: Simon 45,61 67 52 15 45 -3
University of Iowa: Tippie 45,55 68 64 4 40 3
Warwick Business School 45,47 69 58 11 n/a n/a
Ipade 45,02 70 64 6 n/a n/a
McGill University: Desautels 44,68 71 57 14 n/a n/a
Wake Forest University: Babcock 44,35 72 78 -6 47 -3
Melbourne Business School 44,27 73 53 20 n/a n/a
Purdue University: Krannert 44,24 74 74 0 49 -4
Incae Business School 44,03 75 77 -2 n/a n/a
University of South Carolina: Moore 43,77 76 80 -4 54 -8
SDA Bocconi 43,16 77 28 49 n/a n/a
University of California at Irvine: Merage 42,69 78 53 25 40 7
University College Dublin: Smurfit 41,66 79 78 1 n/a n/a
Lancaster University Management School 41,64 80 41 39 n/a n/a
University of Florida: Hough 41,01 81 94 -13 47 1
University of California: Davis 40,39 82 83 -1 28 21
Nanyang Business School 40,09 83 33 50 n/a n/a
University of Georgia: Terry 40,08 84 93 -9 57 -7
Hult International Business School 39,10 85 61 24 n/a n/a
EM Lyon Business School 39,08 86 100 -14 n/a n/a
University of Cape Town GSB 38,63 87 60 27 n/a n/a
College of William and Mary: Mason 38,29 88 86 2 83 -32
Pepperdine University: Graziadio 37,80 89 92 -3 n/a n/a
Durham Business School 37,61 90 55 35 n/a n/a
Leeds University Business School 37,39 91 94 -3 n/a n/a
IAE Business School 36,38 92 98 -6 n/a n/a
University of Edinburgh Business School 36,13 93 88 5 n/a n/a
Vlerick Leuven Gent Management School 34,98 94 55 39 n/a n/a
Birmingham Business School 33,60 95 68 27 n/a n/a
Bradford School of Management/TiasNimbas BS 33,10 96 90 6 n/a n/a
SP Jain Center of Management 32,64 97 68 29 n/a n/a
Eada 31,50 98 84 14 n/a n/a
University of Strathclyde Business School 29,78 99 74 25 n/a n/a
Politecnico di Milano School of Management 29,02 100 96 4 n/a n/a







* ROI-USN rank difference is computed from the rankings of US business schools only (once all non-US business schools have been removed from the FT database).
ROI means Return on Investment. 





ROI scores are T score transformations of the factor scores obtained by Principal Component Analysis with Varimax rotation.
Rank difference is obtained by subtracting the original FT or USN ranks from the ROI ranks. A positive number indicates that a school has been overrated by the FT or USN whereas a negative number indicates underrating.


The second factor includes the following variables (in decreasing order of importance): International mobility rank, International students (%), International board (%), International faculty (%), Career progress rank, and International experience rank. It is clearly an international factor that plays an essential role in our changing global economy.


Statistical Ranking of the Financial Times MBA 2011 (in decreasing order of International score) 
School name International score International rank FT rank International - FT rank difference USN rank
Insead 71,78 1 4 -3 n/a
IMD 71,74 2 14 -12 n/a
Hong Kong UST Business School 70,54 3 6 -3 n/a
London Business School 68,58 4 1 3 n/a
Iese Business School 67,55 5 9 -4 n/a
IE Business School 64,95 6 8 -2 n/a
SP Jain Center of Management 64,15 7 68 -61 n/a
City University: Cass 64,06 8 32 -24 n/a
Esade Business School 64,04 9 21 -12 n/a
HEC Paris 63,68 10 18 -8 n/a
Hult International Business School 63,42 11 61 -50 n/a
University of Oxford: Saïd 62,50 12 27 -15 n/a
University of Cambridge: Judge 62,12 13 26 -13 n/a
Australian School of Business: AGSM 60,61 14 35 -21 n/a
Cranfield School of Management 60,57 15 34 -19 n/a
Durham Business School 60,32 16 55 -39 n/a
Nanyang Business School 60,10 17 33 -16 n/a
Manchester Business School 59,93 18 29 -11 n/a
Imperial College Business School 59,81 19 37 -18 n/a
Eada 59,67 20 84 -64 n/a
Melbourne Business School 59,62 21 53 -32 n/a
Rotterdam School of Management, Erasmus University 59,28 22 36 -14 n/a
Warwick Business School 59,00 23 58 -35 n/a
Birmingham Business School 58,48 24 68 -44 n/a
Lancaster University Management School 58,45 25 41 -16 n/a
National University of Singapore School of Business 58,38 26 23 3 n/a
SDA Bocconi 57,79 27 28 -1 n/a
University of Edinburgh Business School 57,63 28 88 -60 n/a
Vlerick Leuven Gent Management School 57,44 29 55 -26 n/a
York University: Schulich 57,37 30 49 -19 n/a
Ceibs 56,38 31 17 14 n/a
University of Strathclyde Business School 55,70 32 74 -42 n/a
EM Lyon Business School 55,42 33 100 -67 n/a
Incae Business School 55,29 34 77 -43 n/a
University of Cape Town GSB 55,28 35 60 -25 n/a
MIT Sloan School of Management 54,95 36 9 27 3
Bradford School of Management/TiasNimbas Business School 54,48 37 90 -53 n/a
Stanford University GSB 54,23 38 4 34 1
University College Dublin: Smurfit 53,72 39 78 -39 n/a
University of Pennsylvania: Wharton 53,72 40 1 39 3
Columbia Business School 53,70 41 7 34 9
University of British Columbia: Sauder 53,62 42 80 -38 n/a
University of Toronto: Rotman 53,53 43 46 -3 n/a
Harvard Business School 53,14 44 3 41 2
McGill University: Desautels 52,84 45 57 -12 n/a
Politecnico di Milano School of Management 51,78 46 96 -50 n/a
Thunderbird School of Global Management 50,92 47 68 -21 75
Leeds University Business School 50,73 48 94 -46 n/a
IAE Business School 49,74 49 98 -49 n/a
New York University: Stern 48,92 50 15 35 10
Yale School of Management 48,76 51 15 36 10
Indian School of Business 48,44 52 13 39 n/a
Dartmouth College: Tuck 48,22 53 18 35 7
Northwestern University: Kellogg 48,20 54 21 33 5
University of California at Berkeley: Haas 47,97 55 25 30 7
University of Chicago: Booth 47,94 56 12 44 5
University of Western Ontario: Ivey 47,62 57 46 11 n/a
Indian Institute of Management, Ahmedabad (IIMA) 47,11 58 11 47 n/a
University of Michigan: Ross 47,06 59 24 35 14
Cornell University: Johnson 46,82 60 30 30 16
Duke University: Fuqua 46,52 61 20 41 12
Babson College: Olin 45,27 62 84 -22 52
UCLA: Anderson 44,93 63 31 32 14
University of Rochester: Simon 43,91 64 52 12 45
University of North Carolina: Kenan-Flagler 43,89 65 62 3 19
Georgetown University: McDonough 43,21 66 38 28 25
Carnegie Mellon: Tepper 43,10 67 41 26 18
Ipade 42,88 68 64 4 n/a
University of Virginia: Darden 42,58 69 41 28 13
University of South Carolina: Moore 42,46 70 80 -10 54
Boston University School of Management 42,43 71 68 3 34
Pennsylvania State University: Smeal 41,73 72 59 13 40
University of Maryland: Smith 41,67 73 40 33 45
Emory University: Goizueta 41,26 74 38 36 23
Ohio State University: Fisher 41,19 75 72 3 25
University of Texas at Austin: McCombs 41,03 76 49 27 17
Texas A & M University: Mays 40,82 77 44 33 32
Indiana University: Kelley 40,37 78 73 5 23
University of Washington Business School: Foster 40,22 79 86 -7 37
University of California at Irvine: Merage 40,05 80 53 27 40
Vanderbilt University: Owen 39,80 81 51 30 28
Purdue University: Krannert 39,75 82 74 8 49
University of Southern California: Marshall 39,40 83 64 19 21
Wisconsin School of Business 38,77 84 63 21 28
Rice University: Jones 38,21 85 44 41 34
Pepperdine University: Graziadio 37,57 86 92 -6 n/a
University of Illinois at Urbana-Champaign 37,48 87 46 41 37
Georgia Institute of Technology 37,46 88 97 -9 28
Arizona State University: Carey 37,10 89 64 25 27
SMU: Cox 37,08 90 88 2 57
Boston College: Carroll 37,03 91 74 17 34
Kaist College of Business 36,88 92 99 -7 n/a
University of California: Davis 36,88 93 83 10 28
University of Notre Dame: Mendoza 36,28 94 80 14 37
University of Iowa: Tippie 36,26 95 64 31 40
College of William and Mary: Mason 35,69 96 86 10 83
University of Florida: Hough 35,65 97 94 3 47
University of Georgia: Terry 34,91 98 93 5 57
Wake Forest University: Babcock 34,78 99 78 21 47
Brigham Young University: Marriott 33,76 100 91 9 32






International scores are T score transformations of the factor scores obtained by Principal Component Analysis with Varimax rotation.
Rank difference is obtained by subtracting the original FT ranks from the International ranks. A positive number indicates that a school has been overrated by the FT whereas a negative number indicates underrating.


3- Discussion
As it appears quite noticeably, if a sound statistical methodology is applied, neither the FT nor the USN rankings are very accurate. When we compute the average rank differences between the FT ROI ranks and our own we obtain a value of 17. This means that the average ranking mistake made by the FT is 17! The average rank difference between the USN ROI ranks and our own is 12 (this last value was corrected for range restriction because the US sample is twice as small as the world sample).
Nevertheless, our ROI results seem to match better with the results of the USN ranking. It makes sense because USN focuses mainly on the ROI sub dimensions and does not include any international information or schools. This US specificity of the USN rankings has been largely discussed and criticized, and has contributed to the success of the FT method.
The world economy is less and less US centric and international schools have an increasingly important competitive advantage over their US counterparts. This is confirmed by the very low scores of US universities on the International dimension. Indeed, the first US school - MIT Sloan School of Management – is ranked # 36 on this dimension, and this should be a reason for concern. US schools have become increasingly aware of the issue, and are working to catch up with the leading international institutions, but it seems there is still a long way to go.
Lastly, a fatal problem with the FT rankings remains the addition of two factors that have nothing in common. The FT combination of ROI elements with international elements is a Frankenstein’s statistical monster that should not exist: the ROI results are polluted by the international results and vice versa.
In terms of ROI, we have shown that Stanford is number 1 and London Business School number 10. London Business School is not number 1 as believed by the FT. In terms of Internationalism, we have shown that Insead is number 1, London Business School number 4, and Stanford number 38.  The combination of a rank of 10 on ROI and a rank of 4 on Internationalism does not qualify London Business School for a 1 on anything. If it were so, then all journalists could be statisticians, and pigs could fly.
As we already underlined in our first article, the Financial Times’ ranking is biased by a fatal error, i.e. the combination of the ROI and International dimensions, which is statistically incorrect, however good their intentions were. A strong quantitative tool called Factor Analysis has allowed us to compile rigorous rankings and to validate that the two above mentioned dimensions can NOT be added.
This article has focused on the ranking of MBA courses, applying a sound methodology to the existing Financial Times 2011 data. The main outcome of the present study seems to confirm our expectations, both on publishers’ ranking methodologies and the characteristics of business school strategies. Regarding the publishers, the results of our statistical enquiry show that the methodologies used by the journalists to compile their rankings are flawed and unreliable.
As for the results of the schools, our expectations have also been met, with the US-based MBA courses being the most profitable (the first non-US school on the ROI table ranks number 10, and 13 out of the top 15 are based in the US), but seriously lagging behind when it comes to the criteria measuring the international orientation of the teachings. The majority of international schools are located in Europe (9 out of the top 10) and the first US school is ranked 36th. This should be a serious matter of concern for US schools, as we live in an increasingly multipolar world, and adaptation to other cultures and management techniques becomes imperative to thrive in different business environments.

No comments:

Post a Comment