Chris Wiggins gave an excellent talk at Rice this afternoon on data science at the New York Times. In the Q&A afterward, someone asked how you would set up a machine learning algorithm where you’re trying to optimize for outcomes and for information.
Here’s how I’ve approached this dilemma in the past. Information and outcomes are not directly comparable, so any objective function combining the two is going to add incommensurate things. One way around this is to put a value not on information per se but on what you’re going to do with the information.
In the context of a clinical trial, you want to treat patients in the trial effectively, and you want a high probability of picking the best treatment at the end. You can’t add patients and probabilities meaningfully. But why do you want to know which treatment is best? So you can treat future patients effectively. The value of knowing which treatment is best is the increase in the expected number of future successful treatments. This gives you a meaningful objective: maximize the expected number of effective treatments, of patients in the trial and future patients treated as a result of the trial.
The hard part is guessing what will be done with the information learned in the trial. Of course this isn’t exactly known, but it’s the key to estimating the value of the information. If nobody will be treated any differently in the future no matter how the trial turns out—and unfortunately this may be the case—then the trial should be optimized strictly for the benefit of the people in the trial. On the other hand, if a trial will determine the standard of care for thousands of future patients, then there is more value in picking the best treatment.
Tom Fleming at UW Biostat introduced me to an intriguing idea: many study subjects volunteer in order to advance science and serve future generations. In situations where a treatment starts to show harm and patients are notified, it’s common for them to remain, sacrificing their own safety so that the trial can move forward. In this situation, the ethical choice is to prioritize those patient’s wishes over their health, which doesn’t match your objective function.
That’s a good point. I admire the people willing to sacrifice themselves. I hope they are well-informed. Clinical trials in the US require “informed consent” but I’ve seen patient’s give informed consent. They’ll take the form and sign it without reading. “Whatever you say, Doc.”
Some people argue for the opposite, saying more emphasis should be given to the well being of the patients who volunteered for the study. I suppose that could vary by what disease is being treated, say acne vs pancreatic cancer. Some have said we should geometrically discount future patients: the further someone is in the future, the less they benefit from the trial. That’s plausible since the further out you go the more likely something new will replace the treatment recommended by the trial.
Whether you’re giving preference to patients in the trial or after the trial, at least you’re forming a weighted sum of benefits to patients, not adding patients and probabilities.
It sounds like this should be treated as sequential decision making. The point of maximizing information is exploring and not missing the *good, promising* options. So the concepts of exploration/exploitation, bandits and bounded regret probably apply pretty directly.