The Academy Members Project: Methodology Page

Methodology

This page describes the methodology used throughout The Academy Members Project. It answers the questions: “How do we know what we know?”, and “How do we go about our research?”.

There are a few guiding principles that define our approach:

• One name at a time.
• Citation of evidence.
• Evidence-based inferences.
• Using font strength, font color and indentations.
• Gathering a community of researchers.
• Modesty about our limitations.
• A generous correction policy.

One Name At A Time

Because Academy membership is by invitation, the only guaranteed method of finding members is to find a document or interview where they actually *say* that they are members.

That means that every name on our site has to be individually researched, one name at a time.

Sometimes we get lucky and find an article that quotes several academy members by name, or discover a historical document that lists multiple members within or across branches. But even in those cases, we approach each mention as a data point about that particular individual.

We wish there was a short cut. But a quick look at our Non-Members pages shows that there are many Oscar Nominees who have never been invited to join the Academy -- and others who have turned down that invitation or left at some point. Even being an Oscar Winner does not always guarantee you an invite. Nor does being an Oscar host or presenter, or having a certain number of film credits or years in the business or even being a household name. Lists of these people can be a good place to start our individualized research, but they are not conclusive and require us to do additional research to determine their membership status.

Citation Of Evidence

Citations: Those nasty things that your teachers make you put in research papers.

But it turns out that they are really quite useful -- both for proving that you know what you’re talking about and for keeping track of large amounts of information that you’ve already found.

We cite extensively on this site, because we know that you have no other reason to believe us. When possible, we include multiple citations and hyperlinks so that you can go read the evidence yourself. But when that’s not possible we’ll give the old fashioned bibliographic information that you can go hunting for.

The types of resources we use include: Information from the Academy’s official press releases, publications, website and archives; trusted news media and entertainment media that cover the film industry and the Oscar race; as well as social media interactions with members of the film community online.

Ideally, we are looking for an actual statement that someone “is an academy member” or “is an Oscar voter”, or that they have appeared on a members-only committee or event. But we also record less direct and even circumstantial evidence as we find it, with the hope that it will eventually accumulate into a clearer picture.

Evidence-Based Inferences.

The Academy’s invitation-only membership policy makes it dangerous to assume someone’s membership status if you are looking merely at their fame or their body of work. That’s why we ordinarily require a specific citation to show that someone is a member.

But it is sometimes possible to make logical inferences based upon very specific types of evidence that comes directly from the Academy itself.

For example:

• While we have found examples of Oscar nominees who have NOT been invited to join the Academy, we can logically infer that nominees who are seen participating in Academy events over the course of several decades are probably members.
• Similarly, if we know that someone was invited to join the Academy, and is later seen participating in Academy events that don’t involve one of their own films, the most logical assumption is that they have in fact accepted that invitation.

This type of logical inference can also be used to show that someone is NOT a member. For example:

• The Academy has made all of their invitations to membership public since 2004. So if your favorite artist had their first film (or their first big breakthrough) after 2004 and they haven’t shown up on the invite lists, then there’s a strong inference that they’re not members yet.
• The Academy has published partial Memorial Lists on its website since 2003, and those lists seem to have improved in accuracy and completeness since 2011. So depending on when an artist died, we may be able to logically infer that they were not a member at the time of their death.
• It was widely reported that 2015 was the first year any South Korean filmmakers were invited to join the Academy. If those reports are indeed correct, then anyone from South Korea who has not appeared on an invitation list can be assumed to be a non-member.

Note that each of these examples requires that we compare specific pieces of evidence on an individualized basis to show membership or non-membership. Those artists who we simply do not know about either way are better left on our Wish List of people who need further research.

Using Font Strength, Font Color And Indentations

Because we know that most readers are not going to click through all of our sources, we use visual tools as a shorthand to give information about the individuals we’ve researched.

As a general principle, Font strength (Bold, Standard, Italics) signals how strong our evidence is; Font color (Black, Gold, Pink) shows whether the person is living, dead or unknown; and Indentations are used to distinguish special circumstances that differentiate them from others on that list. The specific ways that these visual tools get applied varies slightly depending on whether you are looking at a Members Page, a Non-Members Page, or the Wish List:

On the Members Pages:

• Names in BOLD font represent individuals whose membership has been confirmed publicly by a reliable and documentable source, or where there is overwhelming evidence of membership.
• Names in STANDARD font represent individuals who have been publicly invited to join the Academy. While it is assumed that most people would accept such an invitation, they are not moved to the bold category until documentation of their acceptance can be found.
• Names in ITALIC font represent individuals who are believed to be Academy members, but their status has not yet been verified using our rigorous documentation standards.
• Members who have died are shown using the same three-tiered system, but in GOLD.
• If we do not have enough biographical information to know if a member is living or dead, or to conclusively identify who they are, then the name is shown in PINK.
• The numbered names on the list show those believed to be current members (including those in active, associate, emeritus and retired status).
• Indented names represent individuals who are not currently members, but who may have been in the past. This includes members who have died, but also people who have left the Academy or turned down invitations to join.

On the NON-Members Pages:

• Names in BOLD font mean that we have strong evidence that they are not members. Names in STANDARD font and Names in ITALIC font represent individuals who are believed to be non-members, but where our documentary evidence is weaker.
• Individuals who have died are shown using the same three-tiered system, but in GOLD.
• If we are uncertain whether the person is still living or dead, then the name is shown in PINK.
• Numbered names are those who -- as far as we can tell -- are not members because they were simply never invited to join.
• Indented names represent individuals who are believed to have turned down their invitation or resigned their membership at some point.

On the Wish List:

• Since these names are still being researched, they are all shown in standard font (no bold or italics).
• Black text represents people who are still alive, Gold represents those who have died, and Pink is for those who we aren’t sure about.
• No indentations are used on the Wish List.

Gathering A Community Of Researchers

A key element to our success has been making our discoveries public so that more people can find us and join in the search.

This has helped in the obvious sense that more people looking translates into more names being found, more places that we’re able to look, and more ideas and hunches to follow.

But there is an additional benefit to this approach that we didn’t foresee when we began, and it has to do with morale. Several of our contributors report that they’ve often wondered who made up the Academy, and some had even begun to construct lists of their own in the past. But the size and scope of the project led each of them to give up when they were working solo. Research and archival work can get pretty lonely, after all, especially on those days when you haven’t gotten lucky with any new information.

By sharing our findings, we’re better able to keep the momentum going. There’s someone to share with when you find a new name, and more opportunities to celebrate what other people have found even if your own research is temporarily coming up empty.

For now, at least, we have decided to stick to a model of “curated crowdsourcing” -- where everyone is welcome to send citations that they find to nevertooearlymoviepredictions@gmail.com, but only the curator/moderator can make changes to the site -- instead of a “Wiki” model where people can write on the site directly. That may change in the future, but for now we feel like the extra level of oversight is helpful in maintaining consistency.

Learn more about how you can Join The Search, take on a Special Project, or sign up for our monthly Newsletter.

Modesty About Our Limitations

We promise not to hold back on our readers: We’ll tell you the names of all the Academy members that we know. And it’s equally important that we tell you what we don’t know.

We begin with our tagline: “The largest public list of Oscar voters you’ll find on the internet!”. We don’t claim to have the full list, and don’t pretend that we can compete with the private lists that studios and journalists keep guarded.

The Home Page also keeps a running tally of how many Academy members we’ve identified, listed as both an absolute number and a percentage of the entire club. It allows us to simultaneously brag about how far we’ve come, while making clear that our work is not completed yet.

Our approach to citations, and particularly our three-tiered system of BOLD-STANDARD-ITALIC fonts, keeps us away from speculation and overstatements. While some readers may think that we’re being overly cautious when they see that a high profile artist is only listed in italics, we find that sticking to our documentation rules helps us to safeguard the accuracy and integrity of the list, and encourages reader participation as fans research and submit additional information about their favorite artists. And we use the pink font color when we discover a name that we can’t identify who they are or if they’re still living.

We also have the Wish List, which could just as easily be called the “We Don’t Know List”. It fills a middle ground between those artists who we are confident enough to add to our Members Pages, and those who we have reason to believe are Non-Members. Some have been researched extensively without finding a clear answer either way. Others have received only a cursory look and are placed here for future study. In all cases, we need better evidence before we can authoritatively add them to one camp or another.

And finally, we have a whole page of disclaimers, making clear that our project is a work in progress and is not endorsed by the Academy Of Motion Picture Arts And Sciences (AMPAS).

A Generous Correction Policy

Even with all of these safeguards in place, it is still possible to get things wrong. We’re happy to correct any errors that come to our attention.

Additions, suggestions, corrections and verifications may be submitted by emailing nevertooearlymoviepredictions@gmail.com. You can also contact me on Twitter @NeverTooEarlyMP.

Return to HOME/INDEX Page.

.