The Rating Game

By Jonathan Cummings and Tom Doak

“Jonathan Cummings has for years been an inspirational mentor to the golf rating community. With The Rating Game, he has now established himself as the authority as well. An excellent, and much-needed, book.”—Josh Lesnik, President, KemperSports, and magazine panelist

“Cummings provides a sound and insightful look inside the arcane world of golf course rating. Golfers love to argue over which is the best and why and The Rating Game will be a welcome addition to many over-heated debates.”—Gary Lisbon, President, GolfSelect; golf course photographer and magazine panelist

“Jonathan Cummings is the Nate Silver of golf course ratings. The Rating Game will open up a lot of eyes about hidden mathematical distortion in the golf course ratings system.”—Bradley S. Klein, Golf Channel/GolfAdvisor.com

“I’ve known and read Jonathan Cummings for over forty years. He always provides fair and distinctive insight into a course’s design and character (including some of my own). I applaud him for advancing the discussion in The Rating Game.”—Tom Clark, Golf course architect and Past President of the American Society of Golf Course Architects

“If there ever was a golfer/writer cut out to analyze the course rating process, it’s Jonathan Cummings. I met him twenty-five years ago, after he sent me a fifty-page missive detailing every technical aspect of every course he had played in the last year. I immediately read The Rating Game, and his thorough, analytical style lays out everything you need to know about the subject.”—Jeff Thoreson, Editor, GolfStyles Media Group

• Language: English
• Publisher: Post Hill Press
• Release date: Jun 2, 2020
• ISBN: 9781642936032

Book preview

A POST HILL PRESS BOOK

ISBN: 978-1-64293-602-5

ISBN (eBook): 978-1-64293-603-2

The Rating Game

© 2020 by Jonathan Cummings

All Rights Reserved

Cover photo by Gary Lisbon

Interior design and layout by Sarah Heneghan, sarah-heneghan.com

No part of this book may be reproduced, stored in a retrieval system, or transmitted by any means without the written permission of the author and publisher.

Post Hill Press

New York • Nashville

posthillpress.com

Published in the United States of America

In memory of Dad, whose love of golf knew no bounds

Contents

Foreword

Chapter 1: Introduction

Chapter 2: The Ranking History

Chapter 3: The Measurement Game

Chapter 4: Categories or Not

Chapter 5: Ingredients of an Evaluation

Chapter 6: The Perfect Rater

Chapter 7: The Perfect Rating System

Chapter 8: The Nineteenth Hole

Acknowledgments

FOREWORD

As golf course designers, we accept that whether our newest project compares favorably with Sand Hills or Pine Valley is an entirely subjective exercise. We have to be confident enough in our beliefs to not worry too much about what course raters think.

At the same time, good rankings lead to more work, better sites, and higher design fees. So, we all have a vested interest in seeing the rankings of courses done impartially.

Part of the argument is, Great for whom? What makes a course great is personal to each golfer, and it can be influenced by all kinds of things: from the beauty of the site, to the routing and the shaping of the bunkers and greens, to the conditioning, to the weather, to even whether you lost a ball in the rough on the third hole. (When I ask a golfer what he thinks of one of my courses, and he tells me what he shot, my internal hearing aid switches to mute.) The most enjoyable courses for amateur golfers are often pushovers for Tour players, and the most challenging courses for Tour players are five hours of torture for the seventeen-handicap player, even from the forward tees. How does a ranking balance those different points of view? The answer depends on the people you ask, and how you ask them.

Jon Cummings and I have an affinity for the subject and for each other because we’ve both seen how the sausage of golf course rankings is made, and we both believe it could be done better—even if we don’t entirely agree on how. I have no idea how many other people are as fascinated by the topic as Jon and I are, but for those who are, you will find here many of the gruesome details of how the rankings are done.

I saw many courses for the first time just before they were ranked, and trying to figure out for myself what made them special—or not—shaped my ideas on design. I hope Jon’s book will cause you to trust your opinion more, and magazine rankings a little less.

Tom Doak

2019

CHAPTER 1

INTRODUCTION

Humans love to compare. We have always done it, and do it all the time—at work, at play, and even in the locker room.

The other thing humans love to do is make lists. Put those two powerful impulses together, and you have the modern mania for rankings, the Yelp-ification of society. We are bombarded by an endless stream of rankings—shopping, entertainment, eating, vacationing, socializing, and even romancing require each of us to assess, weigh, rank, and ultimately choose.

Ranked lists are everywhere because we are forever looking for shortcuts. Lists are a quick way to get insight on what is best without having to do the research, from the US News & World Report college rankings to those Michelin stars for restaurants. Nowhere are rankings more popular than in sports. For team sports and some individual ones, lists are determined by wins and losses. For some individual sports, the best are determined by a stopwatch or a tape measure, while others rely on judges.

Becoming the best golfer was once a mostly subjective opinion of peers and media types. In recent years, however, this assessment has been formalized into an Official World Golf Ranking. The OWGR formula, developed by sports agent Mark McCormack in the 1990s, has a few components and is only slightly more complicated than the instructions for building a nuclear power plant.

That is the official ranking, but there are other player rankings, including FedEx points, mostly based on money won, and Golfweek’s Sagarin ranking, which uses a head-to-head method of determining an individual’s play against every other player in a tournament. Imagine computing your performance in match play over everyone else’s in the field; the winner in a 156-player field has a record of 155–0–0, the runner-up is 154–1–0, etc.

Easy enough, right? But still not enough. Unlike almost all other sports, golf is a game in which the field changes as golfers travel the world playing different courses. And the differences can be monumental. Sure, the outfield dimensions of baseball diamonds vary (Fenway Park’s Green Monster, for example), and there are some variables in NFL stadiums—open-air or domed; grass or artificial turf. And tennis has grass, clay, and surfaced courts. But these variables have only a minor impact on the game being played. First base is ninety feet from home plate in all professional ballparks, football fields are always a hundred yards long, and the dimensions of every tennis court are identical. Those many similarities make ranking these various venues difficult, because it must be almost completely subjective. When sportswriters talk of the best ballpark or best stadium, they are usually making an emotional, nostalgic, or historical argument.

Golf couldn’t be more different. Other than having grass, sand, and eighteen holes with cups cut into each green, no two courses are the same, and even playing the same course on the same day can be a vastly different experience. Simply put, the golf course is an important variable in the outcome of a match or tournament. With around thirty thousand golf courses worldwide, each different from the next, it is natural and inevitable that comparisons are made. Moreover, it is natural that golf publications get into the business, comparing courses and generating ranked lists.

The media loves ranked lists and golf publications are no exception. As you will see in the following chapter, ranking courses has a long history. People like lists. They promote discussion and sell magazines, although some lists can be a little wacky. Peruse past issues of major golf publications and you will find lists of Top 10, 50, 100 or even 200 courses ranked by era, location, or architect. Other lists rank the best residential courses, the best resort courses, the best college ones, the best daily fee courses, the best courses for women, the best for tournaments, the best for your skill level, and on and on and on…. You can even find lists of the best holes in the world, the toughest holes, the best quarry holes, and the best par-3s, 4s or 5s.

Want more? How about the best teachers, the ten greatest rivalries, the best golf tips, the best players nobody knows, and the best (I kid you not) badass golfers.

Still want more? Here are a few odd lists I found: the best halfway houses, top moments in golf history, the best Chicago versus the best New York courses, and even the ten best golf orgies. (No, I’m not providing a link for that one. You’ll have to dig up that dirt on your own.)

And in Golf Magazine’s October, 2010 edition: The Best of Everything!

So yeah, golfers like lists. But there is another component to this. These lists are excellent free advertising for the blessed courses. That’s why some clubs encourage visits by evaluators (called raters or panelists) and do everything possible to garner higher rankings. It’s common for course evaluators to play a tract they are rating for free, and some clubs may throw in a dozen balls or a shirt from the pro shop. Some also comp a meal in the clubhouse, provide free replay, or even offer access to neighboring clubs. A few high-roller clubs may pony up for hotel rooms and even airfare for raters.

How much impact this has on rankings is tough to quantify, but it’s hard to imagine it is zero. But I’m not here to address that question. The following pages are devoted to exploring the ranked listings of the top golf courses. We’ll gain a historical perspective by looking at the development of course rating systems used over the years by the major golf publications. We’ll delve into a little measurement science and simple statistics to assess these various approaches. We’ll assess the merits of a category-based and non-category-based evaluation system. We’ll even evaluate the evaluators: what qualifications should be required of them and how to measure their performance. We’ll explore what makes a golf course great. We’ll propose an ideal rating scenario, and finally, I’ll lay out my ideal ranking system.

My hope is that the reader finds these pages both entertaining and informative. I’m under no delusion that there’s a massive audience for this topic. I know I’m speaking to the architectural geeks who read everything they can about golf courses and course design. They love course rankings because they generate spirited discussions. While reading these pages, they are sure to be both nodding and shaking their heads, sometimes vigorously. I look forward to all of their comments.

My goal in writing The Rating Game is to provoke conversations, and, as a longtime Golfweek rater myself, to learn more about this topic. I have always been curious about the rating process, and writing this book has allowed me to research and gain an education and deeper appreciation of what I do for pleasure. I hope the reader gets even a small portion of the satisfaction I got from writing this book.

CHAPTER 2

THE RANKING HISTORY

Beginnings

Golf traces its roots back some five hundred years, but ranking courses only started about a hundred years ago. In 1907, John Low, a British magazine editor, asked a small number of golfing luminaries to name their favorite British courses. The Old Course at St. Andrews topped his poll.

The first published record of assessing a course’s quality based on its components was probably C.B. Macdonald’s 1907 listing of Essential Characteristics of an ideal course. Macdonald, who built the first eighteen-hole course in America in 1893, specified seven areas, each weighted, to assess a course:

1. Course = 45 percent

2. Putting surfaces = 18 percent

3. Bunkers and other hazards = 13 percent

4. Hole lengths = 13 percent

5. Fairway turf quality = 6 percent

6. Fairway widths = 3 percent

7. Tee ground and proximity = 2 percent

Macdonald always ranked his courses as the best, but to be fair, he didn’t have much competition for many years, because there were so few courses in the US back then.

The Golf Courses of the British Isles, the 1910 book by Bernard Darwin, may have been the first