A man tries out Sony's Morpheus virtual reality headset at the 2014 Electronic Entertainment Expo in Los Angeles on June 10, 2014. In a new play, the main character uses software to create a fantasy environment where adult clients molest young children and then kill them. The author asks what ethical dilemmas arise from such virtual crimes.KEVORK DJANSEZIAN/REUTERS
Virtual reality is taking off. With the newly announced Oculus Rift, HTC Vive, the Sony PlayStation VR, Google Cardboard, Microsoft HoloLens, One Plus and Jaunt, there are several options for those wishing to join the sector.
Consumers can use the game “Eagle Flight” to fly over Paris, “The Walk” to take a tightrope that runs between skyscrapers or “Sports Challenge” to play quarterback for a football team.
Players report stunning increases in visual resolution that bring the entertainment very close to reality.
Those who have played the tightrope-walking game describe how difficult it is to take that first step onto the rope, while suspended hundreds of feet off the ground, even though they know it is a fantasy experience. Observers report that people’s legs wobble as if they actually are walking the tightrope. Most participants perspire heavily and exhibit evidence of extreme nervousness as they simulate the high-wire activity.
However, what happens when virtual reality crosses into unethical territory? In a new play, “The Nether,”that opened this month at the Woolly Mammoth Theater in Washington, D.C., playwright Jennifer Haley explores the troubling questions that arise when the main character known as Papa uses advanced software to create a fantasy environment where adult clients molest young children and then kill them.
The play shows detective Morris quizzing Papa on the line between fantasy and reality, and the rightful boundaries of human experience.
For those concerned about a Frankenstein future of misguided technology, the production raises a number of thorny questions. Should there be limits on human fantasies involving heinous thoughts? Do fantasies that remain in the private realm of someone’s brain warrant any rules or regulations by society as a whole?
More complex is the relationship between fantasy and reality. Even if the bad behavior rests solely in one individual’s private thoughts, does that thinking pose a danger to other people?
For example, there is some evidence that repeated exposure to pornography is associated with harmful conduct toward women and that it legitimizes violent attitudes and behaviors.
Does that evidence mean we should worry about misogynistic or violent virtual reality experiences? Will these “games” make it more acceptable for people to engage in actual harmful behaviors?
For privacy buffs, there is another interesting angle on this production. In light of the ongoing battles between government agencies and a number of Web-hosting firms and Internet providers, should those suspected of engaging in questionable activities be compelled to reveal the location and contents of their computer file servers?
What kind of evidence constitutes the grounds for search and seizure? Does participation in violent fantasy games cross the line into activities society should limit?
Currently, there are limits on children purchasing violent games but not adults. For the latter, the societal norm remains libertarian in nature.
As the world moves toward a future based on virtual reality, artificial intelligence and machine learning, we have to think about where to draw virtual lines, what kinds of situations are problematic and how to recraft our laws, regulations and policies for the digital world.
Digital technology is not just a new platform, but a new mechanism for people to connect with one another. There are many legal and ethical issues to resolve as we grapple with emerging technologies.
U.S. President Barack Obama speaks at the "town hall" discussion with British youth at the Royal Horticultural Halls in London, April 23. Obama will announce on Monday he plans to send as many as 250 additional U.S. troops to Syria.CHRISTOPHER FURLONG/GETTY IMAGES
President Barack Obama will announce on Monday he plans to send as many as 250 additional U.S. troops to Syria, a sharp increase in the American presence working with local Syrian forces fighting Islamic State militants, U.S. officials said.
The deployment, which will increase U.S. forces in Syria to about 300, aims to accelerate recent gains against Islamic State and appears to reflect growing confidence in the ability of U.S.-backed forces inside Syria and Iraq to claw back territory from the hardline Sunni Islamist group.
Obama will explain his decision in a speech at 11.25 a.m. local time in Hanover, where he discussed the Syria crisis with German Chancellor Angela Merkel on Sunday. They will meet with other major European leaders after his remarks.
Islamic State, also known as ISIS or ISIL, controls the cities of Mosul in Iraq and Raqqa in Syria and is proving a potent threat abroad, claiming credit for major attacks in Paris in November and Brussels in March.
While Obama has resisted putting U.S. troops into Syria, where a five-year civil war has killed at least 250,000 people, he sent 50 U.S. special operations forces to Syria last year in what U.S. officials described as a "counterterrorism" mission rather than an effort to tip the scales in the war.
His decision to boost those numbers was first reported by the Wall Street Journal on Sunday and confirmed an April 1 Reuters report that the Obama administration was considering a significant increase in U.S. forces.
"He (Obama) intends to put in mor ... forces to the tune of 250 in Syria," one U.S. official said on Sunday, adding he was unable to break down how many of those would be special operations forces and how many might be medical or intelligence support personnel.
"The president has authorized a series of steps to increase support for our partners in the region, including Iraqi security forces as well as local Syrian forces who are taking the fight to ISIL," said a second Obama administration official.
Obama pledged to wind down wars in the Middle East when he was first elected in 2008. But in the latter part of his presidency, he has found it necessary to keep or add troops to help with conflicts in Iraq, Syria and Afghanistan.
The president is ending a six-day international trip that began in Riyadh, where he held talks with Gulf Arab monarchs concerned that Washington's commitment to the Middle East had diminished.
After that meeting, Obama sidestepped a question about whether he would add special forces in Syria, saying: "None of the options are good" if political talks fail to end the civil war there.
Obama has said the U.S.-led coalition fighting Islamic State had made key gains in squeeze the group's territory in Iraq and Syria, reducing its numbers and cutting off its finances.
"We have momentum, and we intend to keep that momentum," he said earlier this month.
In Iraq, the group has been pulling back since December when it lost Ramadi, the capital of the western province of Anbar. In Syria, the jihadist fighters have been pushed from the strategic city of Palmyra by Russian-backed Syrian government forces.
The Pentagon also announced last week that about 200 more troops would be deployed to Iraq, mainly to advise Iraqi troops fighting Islamic State.
Since U.S.-backed forces recaptured the strategic Syrian town of al-Shadadi in late February, a growing number of Arab fighters in Syria have offered to join the fight against the group, U.S. officials told Reuters in early April.
Syria dominated the Sunday talks between Obama and Merkel. The German leader had just returned from a trip to Turkey to see refugee camps along the border.
The European Union has grappled with the flood of about a million migrants last year, most fleeing the Syria crisis. Merkel pushed her EU partners to accept refugees, and recently hammered out a deal with Turkey to stop the migrant flow.
After meeting with Merkel for about 90 minutes, Obama told reporters he was "deeply concerned" about a surge in violence in Syria, where government forces have stepped up bombing of rebel-held areas around the strategic city of Aleppo.
On Monday, the two leaders will be joined by British Prime Minister David Cameron, French President Francois Hollande and Italian Prime Minister Matteo Renzi to discuss Syria and other foreign policy issues.
Obama and Merkel made it clear they were not considering safe zones enforced with military support in Syria.
"The issue surrounding a safe zone in Syrian territory is not a matter of an ideological objection on my part," he said. "It's not a matter of me not wishing I could help and protect a whole bunch of people. It's a very practical issue about how do you do it?"
The March 29, 1933, front page of the Chicago Daily Tribune was sent to the U.S. Holocaust Memorial Museum as part of the History Unfolded project. (Chicago Tribune)
Growing up in Lowell, Mass., in the 1950s and ’60s, Andrea Hoffman learned about the Holocaust at Hebrew school and later married into a family that included Holocaust survivors. Along the way, certain questions haunted her.
“I’ve always wondered what people knew. When did they know it? How did they know it?” said Hoffman, 65. Her mother had been a teenager in Boston during the war but had not paid much attention to the persecution of Jews in Europe at the time, and Hoffman was curious to know how aware her mother and others in the United States would have been.
So a couple of months ago, when she saw a newspaper ad about a new project encouraging “citizen historians” to investigate American newspapers’ accounts of Holocaust, she dived in.
The project, History Unfolded, is an initiative of the U.S. Holocaust Memorial Museum, which is using crowdsourcing to scour newspapers across the country for articles that ran between 1933 and 1945 on the plight of Europe’s Jews.
As it turns out, there were a lot of them. Since the project was launched in full in February, the museum has received 1,030 submissions from articles published in 46 states and the District. So far, 610 people have signed up, including 32 teachers working on the project with their students.
A section of the Nov. 11, 1938, front page of the Free Lance-Star in Fredericksburg, Va. (Fredericksburg Free Lance-Star)
Although historians have studied the U.S. media’s take on the Holocaust, much of the investigation was done before the Internet and crowdsourcing widened the range of what was possible.
“Nobody has done this research, looking at so many papers in the 1930s and ’40s and seeing what the average American citizen would have been reading,” said Elissa Frankle, the museum’s digital projects coordinator. “If you live in that town, it’s going to be a lot easier for you to see than for us and to engage with primary sources.”
The museum selected 20 events related to the Holocaust in general or in relation to the United States’ involvement — for example, the opening of the Dachau concentration camp in 1933, the failure of a child refugee bill in the U.S. Congress in 1939, and Charles Lindbergh’s 1941 speech accusing President Franklin D. Roosevelt, the British and Jews of pushing the country toward war.
An exhibit planned for spring 2018 will incorporate the project’s findings, which will remain available online for researchers. By then the museum hopes to have material from 50 percent of the newspapers that were in circulation in 1940 and engage a fifth of the nation’s high school history classes — around 240,000 students.
“It helps teach young people that history is not just memorizing facts and dates,” said Aleisa Fishman, a historian at the museum. “It’s sort of a mystery that you have to solve, and you have to go looking for stuff.”
Jennifer Goss, a high school history teacher in Staunton, Va., said her students immediately took to the quaint format. “They thought it was so neat to go to the library and use microfilm,” she said.
They were also excited to see their community against the backdrop of major historical events. Their local paper, Staunton News Leader, had reported on nearby German POW camps, and the students spoke with people who remembered seeing German officers doing work around town.
Andrea Hoffman has been contributing clippings from the Free Lance-Star in Fredericksburg, Va., for the U.S. Holocaust Museum’s crowdsourcing project History Unfolded. (Michael S. Williamson/The Washington Post)
“They don’t feel like Staunton’s a hub of world affairs, so they thought it was interesting that the government would have chosen to put those camps there,” Goss said.
Perhaps because they live in an age of unremitting information, her students overestimated how much material would be available. “Looking at it from a present-day lens, they’re like, ‘Oh, this is so important,’ ” she said. “Some of them were kind of frustrated that there wasn’t more.”
But others were shocked to see how much news had been printed on the Holocaust.
“My prevailing notion about this period in time was that a lot of what had happened with the Nazis during the ’30s and ’40s was not that well-known,” said Sandi Auerbach, 62, a retired IBM financial manager in Somers, N.Y., who is a member of the museum and has contributed more than two dozen articles to the project.
“I am amazed, quite frankly, at the coverage that there was in a lot of different papers,” Auerbach said. “For example, in 1933 there was a huge rally in Madison Square Garden with 20,000 people in attendance to protest the persecution of Jews in Germany. . . . The sad thing is that, given all that publicity, still the Holocaust happened.”
Tayte Patton, 17, whose English class in Lexington, Ky., is participating, said he was shocked at the United States’ inaction. “I never knew that we didn’t want to let Jews into the country,” he said. “I always thought that we would let anyone in, that we would be a refuge for the Jews.”
The research includes dailies and weeklies, African American newspapers, college papers, and U.S. papers in Yiddish, Spanish and other languages. Holocaust-related news sometimes made the front page, but small publications often printed it inside the paper.
However, a local connection might cause a paper to run a story more prominently. For example, when the SS Quanza, carrying hundreds of Jewish refugees from Portugal, was denied entry to Mexico in 1940 and docked in Norfolk for supplies, the Virginian-Pilot covered it. (The stranded travelers were eventually issued U.S. visas after Eleanor Roosevelt intervened on their behalf.)
Contributors say they have been struck by detailed accounts of the Nazis’ persecution and slaughter of Jews, along with a wide range of American opinions on whether to act on it.
But not all Americans got a chance to read what was in the papers, Frankle said, describing her conversations with people who were alive at the time. “They were saying, ‘Who had the ability to buy a paper? We were just trying to buy bread.’ ”
Even circulation figures do not tell the whole story, as family members and neighbors might have passed a single newspaper around.
Alex Adams, 72, a retired computer software developer in Marlton, N.J., who volunteers once a month at the museum, has focused on small papers from Montana, where he grew up in a town called Big Timber.
“There are dozens that don’t have anything,” he said, noting that front-page stories tended to focus on “wheat prices and fights over right of way on their property and Fourth of July picnics and such.” But in three papers he has found stories on subjects such as boycotts of Jewish businesses and discussions about what to do with the Jews.
Even coming up empty-handed is a contribution, showing that readers of certain papers would have had less exposure to what was happening, said David Klevan, the museum’s educational outreach specialist. “We’re asking folks to do real research, and a big part of real research is finding nothing.”
“What could be better for a high school student than saying, ‘I’m not just doing a research project; what I’m finding could have implications [for] what’s being presented at the Holocaust Museum,” she said. “The question is if they come up with conclusions that are different from what the historians have always believed, that will be a moment of crisis [as to] how we’re going to work that out.”
Several contributors noted a clear connection between the events of the 1930s and ’40s and current affairs.
“These things that we’re hearing, with people against immigration and congressmen standing up and speaking against it, it’s exactly the arguments that we’re hearing now, and that’s been astonishing to read,” said Hoffman, who is also a volunteer at the museum and has been focusing on the Free Lance-Star in Fredericksburg, Va.
Goss said her students echoed that thought.
“Especially since we got to this project right around the time of the Syrian refugee crisis, they drew tremendous connections,” she said. “One student said, today, human rights violations and refugees are in her opinion much more in the news and that today no one has an excuse not to know.”
An ISIS fighter waves the militant group's flag in Raqqa, Syria, June 29, 2014. New analysis has shown that fighters for the group originate from 70 countries around the world.REUTERS
The latest trove of documents leaked from within the Islamic State militant group (ISIS) has shown that its new recruits originate from more than 70 countries, according to researchers.
The documents, leaked by an ISIS defector to a number of news organizations, contained some 11,000 files filled out by ISIS recruits upon joining the group in Syria in 2013 and 2014. Many of them were duplicates but 4,600 were handed to the Combating Terrorism Center (CTC) based at the West Point Military Academy in the U.S.
CTC said that it was able to verify 98 percent of the documents by matching them with those obtained by the U.S. Department of Defense. Some 15,000 jihadis joined ISIS in 2013 and 2014 and the documents analyzed represent approximately 30 percent of those.
The CTC report, entitled The Caliphate’s Global Workforce: An Inside Look At The Islamic State’s Foreign Fighter Paper Trail , says that its findings “reveals an organization that is attempting to vet new members, manage talent effectively within the organization, and deal with a diverse pool of recruits.”
The average age of the recruits is 26 or 27 years of age but the ages range from 12 to almost 70 years. Some 400 were aged under 18. The foremost group of nationals was Saudi (579), then Tunisian (559), Moroccan (240), Turkish (212), Egyptian (151) and Russian (141). From Europe, there were 49 fighters from France, 38 from Germany and 26 from Britain.
The fighter records analyzed show that they come “from a wide range of educational backgrounds, but as a whole, the group appears to be relatively well-educated when compared to educational levels in their home countries,” the report says. More than 1,000 said they had completed high school education and more than 1,000 said they had at least attended a university.
Only 10 percent of the fighters said that they had prior experience in a jihadi group, mostly in Syria, Libya and Afghanistan. Most of those had previously fought for the Al-Qaeda-affiliated Nusra Front in Syria. In addition, only 12 percent of the recruits expressed a “preference for a suicide role” over a regular fighting position within the group.
“While the Islamic State needs some suicide bombers, it also needs personnel to fill roles like conventional soldiers, sharia officials, police and security or administrative positions,” the CTC report said.
It also includes 431 “exit forms” for fighters who wished to leave the group’s self-proclaimed caliphate for a variety of reasons including medical treatment and family reasons. This leak is the biggest regarding ISIS to date, providing a significant paper trail that offers an insight into the motivations of the group’s recruits.
The World Trade Center towers burn after being hit by a plane in New York on September 11, 2001. The author writes that comments from those who have read the classified reports, and previously declassified information, suggest that junior Saudi officials may have played some role in the 9/11 attacks.SARA K. SCHWITTEK/REUTERS
In the run-up to President Obama’s visit to Saudi Arabia later this week, two domestic issues that concern the U.S.-Saudi relationship are also gaining attention.
Yet these developments—a congressional bill that allows Americans to sue foreign governments for supporting terrorist groups, and growing calls to declassify the remaining 28 pages of the 9/11 Commission’s report—are unlikely to substantially impact the U.S.-Saudi relationship, which is already on a downward trend due to other, more substantive factors.
Certainly, the bill would have major legal implications for relatives of victims of the 9/11 attacks, who have previously tried to sue the Saudi government for their possible involvement. However, their hope that the declassified report would yield a better understanding of the scope of that involvement is unlikely to yield any smoking gun revelations.
Some of the purported revelations are, in fact, already known. It has been long known that Saudi Arabia has had a hand in the spread, through schools and philanthropic endeavors, of a certain kind of extremist Islamic philosophy often described as Wahhabism. That this philosophy is shared by various radical groups including ISIS and Al-Qaeda is likewise well known, but there is no evidence that the Saudi government ever provided material support to either group.
Though lesser-known, it is also the case that many private Saudi citizens have provided funding to extremist groups over the years. And while it did not come from the government, as Ben Rhodes, the president’s Deputy National Security Advisor noted this week, the Saudi government often paid“insufficient attention” to such funding, particularly prior to 2001.
The 9/11 Commission report, though likely to be less detailed than many of the studies of this phenomenon conducted over the last decade, may well include data on the extent to which the Saudi government turned a blind eye to terrorist funding.
Comments from those who have read the reports, and previously declassified information, also suggests that junior Saudi officials may even have played some role in the 9/11 attacks themselves. Indeed, perhaps the best known line in the 9/11 report itself is the assertion that the Commission found “No evidence that the Saudi government as an institution or senior Saudi officials individually funded the organization,” an obvious loophole that leaves little to the imagination.
Yet it is worth noting that any such revelations contained in the report would be at best preliminary, based on unvetted and unverified intelligence.
In fact, while the Saudi government has not objected to declassification of the report, it clearly perceives the congressional bill as a larger concern, threatening economic reprisals. The threat to sell-off American assets if the bill passes is likely an empty one, but it certainly underscores the concern Saudi leaders feel about the potential for such lawsuits.
Perhaps the most interesting aspect of this whole episode is that it is happening at all, a development at least partially driven by the deteriorating U.S.-Saudi relationship. President Obama’s trip to Riyadh will not be an entirely pleasant one, given all the tensions in the U.S.-Saudi relationship. Indeed, only a few weeks ago, Obama himself publicly questioned the Saudi alliance in an interview with the Atlantic’s Jeffrey Goldberg.
Ultimately, the Saudi alliance is changing. Once thought unshakeable, common U.S.-Saudi interests such as energy security and anti-communism have diverged or disappeared entirely. Meanwhile, disagreements on regional stability, Saudi involvement in conflicts like Syria and Yemen and their support for various extreme groups have helped to sour the relationship.
Whether or not the 9/11 Commission report is declassified, it is these larger tensions that present the major obstacle to smooth U.S.-Saudi relations in the future.
Emma Ashford is a visiting research fellow with expertise in international security at the Cato Institute.
In the summer of 2006, Todd Carlisle, a Google analyst with a doctorate in organizational psychology, designed a 300-question survey for every Google employee to fill out, The New York Times later reported.
Some questions were straightforward: Have you ever set a world record? Other queries had employees plot themselves on a spectrum: Please indicate your working style preference on a scale of 1 (work alone) to 5 (work in a team). Other questions were frivolous: What kind of pets do you own?
Carlisle crunched the data and compared it to measures of employee performance. He was looking for patterns to understand what attributes made a good Google worker. This was strongly related to another question that interested his boss, Lazlo Block, vice president of People Operations: What attributes could predict the perfect Google hire?
Ten years ago, Google was infamous for its complex application process and brain-teasers—something Block recently admitted were "a complete waste of time.” Google was essentially trying to Google the human-resources process: It wanted a search algorithm that could sift through tens of thousands of people—Google’s acceptance rate is about 0.2 percent, or 1/25th that of Harvard University—and return a list of the top candidates. But after a great deal of question-asking and number-crunching, it turned out that the best performance predictor wasn’t grade-point average, or type of pets, or an answer to the question, “How many times a day does a clock’s hands overlap?” The single best predictor was: absolutely nothing.
***
Hiring is hard. General managers know it. Startup founders know it. School principals and casting directors know it. But for readers who are none of those things, consider America’s most public hiring processes—aside from presidential elections, perhaps—which are sports drafts.
Every year, millions of Americans watch professional talent evaluators try to predict who will be the best future athletes in the NBA and NFL Drafts. Again and again, audiences get valuable lessons in the inability of experts to divine future talent. Scouts aren’t dumb. Overall, the first pick tends to be better than the tenth pick, and he tends to be better than 100th pick. But years after the draft, at least one squad almost always looks foolish. For every top five team that can’t believe it picked a Darko Milicic or Ryan Leaf, there is a top five team that can’t believe it missed a Stephen Curry or Tom Brady.
Hiring is hard for the same reason that dating is hard: Both sides are in the dark. "The fundamental economic problem in hiring is one of matching with costly search and bilateral asymmetric information,” Paul Oyer and Scott Schaefer write in "Personnel Economics.” In English, that means hiring is expensive, time-consuming, and inherently uncertain, because the hirer doesn't know what workers are the right fit, and the worker don’t know what hirers are the right fit.
But hiring is not hopeless. Like any consequential business decision, it has been exhaustively studied. The companies with the most successful hiring practices are the ones who learn which metrics and processes reliably help to predict performance and quickly identify the metrics and processes that are complete wastes of everybody’s time.
Here again, sports are instructive. One of the most important jobs of a scout is to isolate skills (which are inherent) from outcomes (which depend on a zillion variables). In baseball, for example, it’s impressive for any young pitcher to have far more wins than losses. But wins rely on factors like the team’s fielding and runs scored, which don’t have much or anything do with pitching quality. It’s far better for scouts to focus on metrics that the pitcher controls exclusively, like strikeouts and walks. In basketball, scouts have historically been seduced by high scorers on successful college teams. But work by David Berri has shown that raw points and Final Four appearances aren’t good predictors of NBA talent, at all. Instead, subtler statistics like rebounding, assists, and field-goal percentage were much more consistent metrics from year to year.
This is a fundamental challenge in hiring: identifying the metrics that actually predict employee success, rather than relying on the most available pieces of information. The gap between these two groups is quite clear in one of the most popular industries to study the science of hiring: American schools.
***
A great teacher is hard to identify and hard to measure. You can observe a star athlete’s quality in his or her own performance. But a teacher’s quality is reflected in the performance of other people—and (this part doesn’t make the evaluations any easier) those other people are very often children.
Even so, the academic literature suggests that principals seem to value the wrong metrics. They hire for credentials (e.g. teacher certificates) and proximity (e.g. does the applicant live in the same state as the schools?) while discounting the factors that make the best instructors.
A straightforward yet under-appreciated fact about instruction is that smart people make good teachers. College GPA and SAT scores correlate very highly with teaching performances, in a range of studies. But several papers in the last 20 years have shown that teaching applicants with higher GPAs don't receive more offers; instead, applicants in the same state as the school do. Meanwhile, basic credentials like graduate degrees and certifications are valued in applications, but they "have little or no power to explain variation in performance across teachers,” according to one recent study published in March by researchers at the University of Michigan, Columbia University, and Harvard Graduate School of Education.
This study used unique access to applicants, hires, and teacher performance in Washington, D.C., Public Schools. Like many similar papers, it found that undergraduate performance and scores on a teacher screening test both strongly predicted teacher effectiveness. (“Effectiveness,” in this case, was measured using D.C.'s IMPACT system, which evaluates teachers based on their year, subject, classroom observation grades, and reports from principals and assistant principals.)
The researchers found that DC public school principals consistently missed the best teaching prospects. For example, they hired more people who went to college in D.C. (which had nothing to do with better teaching) but ignored SAT scores and GPA (both of which were “significantly positively related to performance”). Some principals might assume that high-achieving workers will feel overqualified teaching in public school and quit after a year. But the study found no correlation between academic credentials and attrition. Instead, their basic conclusion was a stinging indictment of teacher hiring: The attributes of the best teaching prospects were "only weakly, if at all, associated with the likelihood of being hired.”
***
Teaching puts one individual in charge of a classroom. That’s different than being a member of a large product team within a larger company. Hiring team members requires filtering for different hard and soft skills, so that new employees can slip into established patterns of company behavior. In this case, many companies depend on asking their employees to double as HR recruiters by leaning on referrals.
Researchers have long known that referrals surface better job candidates. Referred candidates are more likely to get call backs, more likely to be hired, and more likely to stay at the company. Furthermore, they had a pretty good theory about why referrals work. Most hiring is a blind date, and referrals are an introduction. They give both sides a little bit more certainty and information about fit. But academics couldn’t figure out why referred candidates were actually better. A May 2013 paper suggests an simple answer: Company referrals don’t work because they yield smarter workers. They work because they yield better fits.
The study found that referrals produce "substantially higher profits per worker” who are "less likely to quit," "more innovative,” and "have fewer accidents—all this, even after controlling for factors like college, SAT scores, and IQ. Team-based companies require openness, compatibility, and a willingness to cooperate. Referral programs work because great employees pass along workers who similarly match the company culture.
Although they account for only six percent of total applications, referrals now result in more than a quarter of all hires at large companies, according to a recent paper from the Federal Reserve Bank of New York and MIT. But while referrals are extremely useful, they can create their own problems. Many industries—tech and media, for starters—are infamous for disproportionately hiring white, upper-middle class young men who went to elite colleges. Relying exclusively on referrals could deepen workplace homogeneity.
What’s more, referrals help winnow the applicant pool, but that’s not nearly enough. As the New York Fed study showed, the majority of jobs are still filled without referrals and the majority of referred candidates are still rejected. More important than a strong referral program is a strong interview process. How does a hiring manager distinguish between a merely acceptable candidate and the great one, without devoting thousands of hours learning the secret talents, hobbies, and motivations of every single applicant?
Google, which depends on referrals, once administered up to 25 interviews for each job candidate. Todd Carlisle, the organizational psychology doctorate who administered the company’s surveys in 2006, thought this might be overkill. He tested exactly how many interviews were necessary to be confident about a new hire. The right number of interviews per candidate, he discovered, was four. This new policy, which Google calls the Rule of Four, "shaved median time to hire to 47 days, compared to 90 to 180 days,” Lazlo Block wrote in his book Work Rules.
But Carlisle’s research revealed something deeper about the hiring process, which has resonance for every industry: No one manager at Google was very good, alone, at predicting who would make a good worker.
Four meticulously orchestrated Google interviews could identify successful hires with 86 percent confidence, and nobody at the company—no matter how long they had been at the company or how many candidates they had interviewed—could do any better than the aggregated wisdom of four interviewers. (Okay, technically, one employee could: a data center worker, named Nelson Abrasion, who interviewed exclusively for a "very distinctive skill set.”)
There are several reasons to aggregate interview scores. First, everybody is a little bit biased in one direction or another—toward older or younger, toward extroverts or introverts. Combining scores mitigates that inevitable bias. Second, sometimes people just have bad interviews, and it’s unreasonable to base an entire hiring decision exclusively on one 30 minute performance.
Third Google’s finding suggests there are no magical hirers in the world. There are no performance oracles who just know a good candidate when they see it.
This is perhaps the most interesting an important conclusion. In a November 2015 study, researchers looked at 15 firms that used a job test, which placed applicants into three buckets: green (positive), yellow (tentative), or red (negative). These job tests accurately predicted worker performance and retention. The greens stayed longer than the yellows. The yellows stayed longer than the reds.
But sometimes, the human-resources managers ignored the data and went with their gut. Why? Perhaps the managers thought they knew better than a cold piece of data. But these "gut” picks were busts, according to Mitchell Hoffman at the University of Toronto, Lisa Kahn at Yale University School of Management, and Danielle Li at Harvard Business School. When managers thought they were smarter than the hiring systems they set up, it's the systems that ended up looking smart.
It will always be difficult to predict fit and performance, because humans are complex and humans interacting in human systems are even more complex. The right lesson is more subtle: Hiring is hard, and nobody is very good at doing it alone, whether you’re a Google boss, a high-school principal, or a sports general manager. They need help—sometimes in the form of standardized tests, and sometimes in the form of aggregated interview reports. When it comes to identifying the best future talent, groups are better than individuals, data-plus-groups is better than groups alone, and nearly anything is better than brainteasers.
