Data Comparison

Question

We have a SQL Server table containing Company Name, Address, and Contact name (among others). We regularly receive data files from outside sources that require us to match up against this table. Unfortunately, the data is slightly different since it is coming from a completely different system. For example, we have "123 E. Main St." and we receive "123 East Main Street". Another example, we have "Acme, LLC" and the file contains "Acme Inc.". Another is, we have "Ed Smith" and they have "Edward Smith" We have a legacy system that utilizes some rather intricate and CPU intensive methods for handling these matches. Some involve pure SQL and others involve VBA code in an Access database. The current system is good but not perfect and is cumbersome and difficult to maintain The management here wants to expand its use. The developers who will inherit the support of the system want to replace it with a more agile solution that requires less maintenance. Is there a commonly accepted way for dealing with this kind of data matching? JavaScript questions and answers, JavaScript questions pdf, JavaScript question bank, JavaScript questions and answers pdf, mcq on JavaScript pdf, JavaScript questions and solutions, JavaScript mcq Test , Interview JavaScript questions, JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)

Answer 1

Here's something I wrote for a nearly identical stack (we needed to standardize the manufacturer names for hardware and there were all sorts of variations). This is client side though (VB.Net to be exact) -- and use the Levenshtein distance algorithm (modified for better results): Public Shared Function FindMostSimilarString(ByVal toFind As String, ByVal ParamArray stringList() As String) As String Dim bestMatch As String = "" Dim bestDistance As Integer = 1000 'Almost anything should be better than that! For Each matchCandidate As String In stringList Dim candidateDistance As Integer = LevenshteinDistance(toFind, matchCandidate) If candidateDistance < bestDistance Then bestMatch = matchCandidate bestDistance = candidateDistance End If Next Return bestMatch End Function 'This will be used to determine how similar strings are. Modified from the link below... 'Fxn from: http://ca0v.terapad.com/index.cfm?fa=contentNews.newsDetails&newsID=37030&from=list Public Shared Function LevenshteinDistance(ByVal s As String, ByVal t As String) As Integer Dim sLength As Integer = s.Length ' length of s Dim tLength As Integer = t.Length ' length of t Dim lvCost As Integer ' cost Dim lvDistance As Integer = 0 Dim zeroCostCount As Integer = 0 Try ' Step 1 If tLength = 0 Then Return sLength ElseIf sLength = 0 Then Return tLength End If Dim lvMatrixSize As Integer = (1 + sLength) * (1 + tLength) Dim poBuffer() As Integer = New Integer(0 To lvMatrixSize - 1) {} ' fill first row For lvIndex As Integer = 0 To sLength poBuffer(lvIndex) = lvIndex Next 'fill first column For lvIndex As Integer = 1 To tLength poBuffer(lvIndex * (sLength + 1)) = lvIndex Next For lvRowIndex As Integer = 0 To sLength - 1 Dim s_i As Char = s(lvRowIndex) For lvColIndex As Integer = 0 To tLength - 1 If s_i = t(lvColIndex) Then lvCost = 0 zeroCostCount += 1 Else lvCost = 1 End If ' Step 6 Dim lvTopLeftIndex As Integer = lvColIndex * (sLength + 1) + lvRowIndex Dim lvTopLeft As Integer = poBuffer(lvTopLeftIndex) Dim lvTop As Integer = poBuffer(lvTopLeftIndex + 1) Dim lvLeft As Integer = poBuffer(lvTopLeftIndex + (sLength + 1)) lvDistance = Math.Min(lvTopLeft + lvCost, Math.Min(lvLeft, lvTop) + 1) poBuffer(lvTopLeftIndex + sLength + 2) = lvDistance Next Next Catch ex As ThreadAbortException Err.Clear() Catch ex As Exception WriteDebugMessage(Application.StartupPath , [Assembly].GetExecutingAssembly().GetName.Name.ToString, MethodBase.GetCurrentMethod.Name, Err) End Try Return lvDistance - zeroCostCount End Function