Loading…
MinneBOS has ended
Presented by MinneAnalytics. Hosted by Boston University Questrom School of Business.
Thursday, August 23 • 10:30am - 11:00am
Entity Resolution with Machine Learning

Log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Organizations of various sizes and across verticals face challenges with how their data evolves as they scale. One of the most significant challenges as they look to integrate data from additional systems is duplication of entities across systems. These duplicated entities may not have clear criteria to help engineers identify and combine them. However, as organizations are increasingly looking to aggregate data sources—both internal and external—addressing this challenge is critical to realizing value from the data. At medium- to large-scale data, combining the data sets can't be done without the aid of software. Traditional techniques for tackling this challenge involved rule-based systems to look at every pairwise combination and determine whether there was a match or not. More recently, machine learning approaches that block and score probable matches allow engineers to set an acceptable probability threshold, as well as validate and tune the algorithm to help it continuously improve.

In this presentation, we will present entity resolution in the context of a detailed example scenario using physician data matching. We will walk through a machine learning model we created by forking and building on an open source entity resolution library, dedupe.io. We will describe how the model initially performed, how we evaluated it, and how we improved it. Finally, we will walk through high-level workflows for performing entity resolution on your own data.

Speakers
avatar for Wassaf Farooqi

Wassaf Farooqi

Director, Engineering, Manifold.AI
avatar for Jim Clouse, MLA

Jim Clouse, MLA

Senior Software Engineer, Gerson Lehrman Group (GLG)


Thursday August 23, 2018 10:30am - 11:00am
Room 408 Boston University Questrom School of Business, 595 Commonwealth Avenue Boston

Attendees (28)