Artificial intelligence and machine learning systems are gaining a lot of ground in healthcare, with everyone from tech giants like Google and Amazon to startup companies building systems for healthcare provider organizations. Claims about algorithms beating physicians at their job are made every month. But are all algorithms made equal? Will invest in machine learning result in meaningful gains for an organization? The Algorithm Science team at Partners Connected Health invests a great deal of time thinking about the right questions, working out potential pitfalls and developing best practices in evaluating machine learning based solutions.