A machine-learning system may have predicted the World Series winner

You may remember the Silicon Valley startup Unanimous AI from 2016, when its human-augmented AI platform, UNU, correctly guessed 11 of 15 Academy Award winners. The company followed that feat in 2017 when it successfully predicted the top four winners of that year’s Kentucky Derby — a Superfecta that returned 540-to-1 odds. This year, the company has turned its predictive services toward Major League Baseball.

“We’re really very different than how other companies work,” Unanimous founder and chief scientist Dr. Louis Rosenberg told Engadget. “Most AI companies and researchers are focused on replacing people with algorithms, where they do deep learning on big data sets and find patterns in the data.” Unanimous, conversely, puts humans at the center of the decision making.

“We want people to be part of the process,” he said, “because people have knowledge and wisdom and insight and intuition that does not exist in just the raw data. People also have a really good sense of the context. … They have intuition that’s just not present if you’re just looking at the historical data.”

To generate a prediction, the company gathers a bunch of serious fans — for the MLB postseason, they recruited 50 — and connects them in real-time through the company’s Swarm AI platform, which as you can see above, looks like a cross between a Ouija board and a web poll.

This platform enables the participants to combine their knowledge and intuition, becoming a “super expert” capable of making predictions with a far higher degree of accuracy than any one of them could do alone.

Specifically, the system works through an online portal where group members can log on with a PC or mobile device. When prompted, the group attempts to collectively move a cursor towards a displayed answer to the question while the AI’s algorithms interpret each person’s actions to determine their conviction and assign relative weight to their input.

Using the previous Kentucky Derby results as an example, Rosenberg explains that “as a swarm, [these participants] all have different views and different perspectives … some people feel like the most important factor is the jockey, other people think that the weather conditions and other people think that the pole position is most important.” Rosenberg also points out that none of the 20 horse-racing fans recruited for the project correctly guessed the four winning horses on their own. Only through combining their expertise were they able to correctly predict the Superfecta.

Rosenberg likens this effect to the group decision-making processes found in nature. It’s “the reason why birds flock or fish school or bees swarm,” he said. “They become significantly smarter together. You can almost think of it as this invisible tug of war, where they’re all pulling in different directions and they’re converging on one answer that basically combines the strength of their conviction.”

Interestingly, this effect can be scaled only so far. If you can get improved predictive results from 100 people, then using 1,000 people should be even better, right? Not so, says Rosenberg. Turns out that this system sees quickly diminishing returns as more members move to the group. But even with small numbers, this system can provide powerful insights.

In 2018, researchers from Stanford University School of Medicine teamed up with Unanimous AI to study whether the system could help improve pneumonia diagnosis. The study recruited five radiologists and had them decide whether patients had pneumonia based on a series of chest X-rays.

“When doctors worked together, we reduced errors [by] 33 percent compared to diagnoses done by doctors alone, and [by] 22 percent compared to a Stanford software called CheXNet,” Rosenberg told the Stanford Daily.

Of course, the Swarm AI system is not infallible, especially when predicting the outcome of a game as quirky as baseball. When Engadget interviewed Dr. Rosenberg, only the first game of divisional series (four games in total, two from each league) had been played.

“We see that the most likely winner in the American League is the Astros, the most likely winner in the National League is the Dodgers,” Rosenberg said at the time. “And we see that the most likely winner if they face each other in the World Series, we see the Astros win it all.”

“We’re currently 75 percent accurate,” he continued. “There’s only been four games played, but we’re 75 percent accurate in predicting those four games.”

Unanimous has also since revised its estimates throughout the playoffs, accounting for upsets. While the Nats’ have seen a 24 percent jump (from 6 percent to nearly 30 percent) in their chances of winning the Series since the start of the playoffs, the system still holds a comfortable 33 percent chance of taking home the Commissioner’s Trophy this year.

But nobody could have foreseen the Dodgers’ spectacular first-round upset to a wild card team after dominating the regular season with 104 wins or have seen Clayton Kershaw choke in game 5 of the NLDS coming (even though he keeps doing it), or even predict the impact of the MLB “unjuicing” balls used in the postseason. Each of these factors contributed to an outcome that neither sports fans nor machine-learning systems could have predicted.

Still, Rosenberg remains confident of the system’s potential. “We’ve been predicting the whole baseball season so far,” he said. “We track ourselves against Vegas, which is what everybody cares about, and anyone who’s been following our pics would have a positive ROI for the season against Vegas.”

Rosenberg doesn’t see either Swarm AI or traditional machine-learning systems wiping out the other in a VHS vs BetaMax format fight, but rather each occupying a complementary niche. “There are really good powerful uses of machine-learning,” he said. “The issue that, while the technology of machine-learning works well, the data that you need to train the system doesn’t exist, or it does exist, but gets out of date very quickly.”

Swarm AIs on the other hand, rely on a continuously updated database of the participants’ experience. “That database is really good at solving many problems, especially issues that involve human sentiment or human perspective,” Rosenberg said. “It solves a different class of problems that the traditional systems aren’t good at.”

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.