Read time: 5 mins
Crowdsourcing: Using Disruptive Technology to Solve Challenging Problems
In recent years, technology has shaken up traditional business: from apps to AI, the disruptions are transforming industries. Innovative businesses are harnessing technology to stay ahead of the competition and lead the way into the digital future. Crowdsourcing is one example of how organizations can use technology to solve challenging problems, and find solutions that far exceed their expectations.
Watch this video from Andrew McAfee, Faculty Director in the Digital Business Strategy online short course from the MIT Sloan School of Management to find out more.
Transcript
Utilizing a crowdsourcing strategy
A little while back, a friend of mine named Karim Lakhani, and his colleagues did a really interesting piece of research where they set up a head-to-head competition between the performance of the core of the medical research industry on a tough problem versus the performance of the crowd. And again, the crowd are just all of the people out there connected to the internet somehow. And the problem that they were working on, the really difficult problem, was sequencing the genomes of lots of human white blood cells.
Our white blood cells have a very strange genome to fight off all of our biological enemies out there. And obviously, to understand how this all works, we would like to use genetic technologies to sequence the genomes of lots of our white blood cells, hopefully understand the world of disease better. So, this is a pretty fundamental problem. It is a really interesting, really important, very difficult problem. Like you can imagine, we use computers to do the sequencing now, and when Karim and his colleagues started the research, the US National Institutes of Health had a baseline method for doing this sequencing. It was an algorithmic approach, used a lot of computing, and it worked with about 72 percent accuracy and one run, one computer run, took about four-and-a-half hours. And the question was, “How good is that? Can we expect to do a lot better?” Keeping in mind the NIH, the National Institutes of Health, is absolutely part of the core of the US medical research establishment. It is set up to tackle these kinds of tough problems. So, can we expect to do a lot better or not?
There is a research team at Harvard Medical School, which again, I would say it’s absolutely part of the core of the medical research community, that did a lot better. They did the same work, not in four and a half hours, but in a little bit less than one hour. And they got the accuracy up to about 77 percent. Okay, that’s not a small improvement, that’s a pretty big improvement, but again, it came from a different part of the core of the medical research community.
The impact of crowdsourcing
Where things got really interesting, for me, is when Karim and his colleagues worked to open this problem up to the crowd and see what the crowd could do. And the way they did this was they reformulated it so it was more of a pure algorithmic challenge and they posted it to an online community called Topcoder, which is a place where people who love to work on algorithms, for whatever reason, show up and they come across interesting problems, and they work on them. And they ran a two-week competition, it ran for 14 days, and they said, “Look, if this is interesting to you, download the problem, download the data, and upload your solution, and we’ll see how much better you can do.” The absolute best submissions that came in, there were a handful of these absolute best submissions that came in from the crowd, they could do the same work not in four-and-a-half hours, not in a little bit less than one hour, but they could do the same work in less than a minute. And the accuracy that they came up with was not 72 percent or 77 percent. It was up at about 80 percent, and they really didn’t expect that anyone could be more accurate than that.
The research team interviewed the people who uploaded these solutions. It turns out a lot of them were young, a lot of them were still students, and very few of them said that they had any life sciences or any R&D experience at all. What I what I take away from this example is that if you can tap into the energy and the power and the wisdom of the crowd, if you can get them excited about your opportunity or about your problem or your challenge, you’ll get a solution that is better than anything that the core had been able to come up with.