Diffrence betwee Lookup, Fuzzy Lookup and Fuzzy Grouping

Look-Up:
           Using Lookup Transformation we can find the exact matched records.

Fuzzy Lookup:
           Using Fuzzy Lookup Transformation, we can find the percentage matches between two strings.

Fuzzy Grouping:


           Using Fuzzy Grouping Transformation, we can find the duplicate records.

Diffrence Between Merge and Merge Join

Merge:

• Using Merge Transformation, We can combine two sorted data sets into single data set.
• The Merge Transformation similar to 'Union all'.
• The two data sets, Number of columns and data types should match.

Merge Join:

• Using Merge Join Transformation, We can Perform 'Join' Operation using two sorted data sets.
• We can perform 'Inner Join, Left Outer and Full Outer Join.
•  The two data sets, number of columns and data types can be differ but the Key columns (Using to join two data sets) data types should match.

MERGE JOIN TRANSFORMATION

            Using Merge Join transformation we can join two sorted data sets based on common column. We can perform “Inner join, Left outer join and Full outer join" using Merge join transformation.

Sample package creation:

Requirement:-

Two source tables are available EMP (is having Employee information) and DEPT (is having department information).

EMP Table:-

CREATE TABLE EMP (ID INT,NAME VARCHAR(20),DEPT_ID INT)

GO

INSERT INTO EMP VALUES (101,'A',10)

INSERT INTO EMP VALUES (102,'B',20)

INSERT INTO EMP VALUES (103,'C',10)

INSERT INTO EMP VALUES (104,'D',30)

DEPT Table:-

CREATE TABLE DEPT (DEPT_ID INT,LOC VARCHAR(20))

GO

INSERT INTO DEPT VALUES (10,'Location1')

INSERT INTO DEPT VALUES (20,'Location2')

INSERT INTO DEPT VALUES (30,'Location3')

Join the source tables based on ‘DEPT_ID’ column and load the source data into target table “EMP_TARGET”.

EMP_TARGET:-

CREATE TABLE EMP_TARGET (ID INT,NAME VARCHAR(20),DEPT_ID INT,LOC VARCHAR(20))

Package creation:

• Open BIDS.
• F
  rom Control flow items, drag and drop “Data flow task” into designer surface.
• From Data flow items, drag and drop tow “OLEDB Sources” into designer surface (Firs one is for “EMP Table” second one for “DEPT Table”) 
• Map the "OLEDB Sources" with source tables.
• We need to sort the source data based on ‘DEPT_ID’ column because “Merge Join” transformation accepts only sorted data sets as input.
• We can sort the source data in two ways.
  o   Using Sort transformation.
  o   Using OLEDB source “Advanced Editor”
• In this example I used ‘OLEDB Source Advanced Editor’ to sort the source data sets.
• From Data flow items, drag and drop the “Merge Join” transformation into designer surface.

• Select ‘OLEDB source’ (EMP) green arrow and connect to the transformation. Then “Input Output Selection” Window will open. In that window, Select “Merge Join Left Input” from the “Input:” drop down list.

• Select second ‘OLEDB Source’ (DEPT) green arrow and connect to the transformation.

• Right click on “Merge Join” and select “Edit” option.

• Select ‘Join type:’ as Inner join and select the required columns and click “OK” button.

• From ‘Data flow’ items, drag and drop “OLEDB Destination” into Designer surface.
• Connect the ‘Merge Join’ transformation green arrow to ‘OLEDB Destination’.
• Map the ‘OLEDB Destination’ to the target table and execute the package.

Target table Output:

MERGE TRANSFORMATION

The “Merge transformation” combines two sorted data sets into single sorted data set. The Merge transformation accepts only two sorted data sets as input. It combines the source data sets into single data set and sends to the transformation output.

            If we don’t want to sort the data sets or having more than 2 source data sets, then we can use “Union all” transformation to combine the data sets.

Sample package:

Requirement:-

I am having two tables (Product1, Product2), those are having product information. I need to create a file with product information from the two product tables.

Source table creation script:-

Product1:-

CREATE TABLE Product1(ID int,Name varchar(20),Price decimal(8,2))

go

insert into Product1 values (101,'Salt',10)

insert into Product1 values (102,'Rice',30)

insert into Product1 values (103,'Oil',60)

Product2:-

CREATE TABLE Product2(ID int,Name varchar(20),Price decimal(8,2))

go

insert into Product2 values (201,'Pen',10)

insert into Product2 values (202,'Book',30)

insert into Product2 values (203,'Bag',60)

Package creation:-

• Open BIDS.
• From Control flow drag and drop “Data flow task” into designer surface.
• Go to “Data flow”, from data flow items drag and drop two ‘OLEDB sources’ into designer surface (one is for Product1 other one is for Product2).
• Map the OLEDB sources with source tables.
• We need to sort the source data because “Merge” transformation accepts only sorted data sets as input. Based on “ID” column I am going to sort the source data sets.
• We can sort the source data in two ways
• Using Sort transformation
• Using OLEDB source “Advanced Editor”
• In this example I used ‘OLEDB source Advanced Editor’ to sort the source data sets.
• From data flow items drag and drop the “Merge” transformation into designer surface. 

• Select one of 'OLEDB sources' green arrow and connect to the transformation. Then “Input Output Selection” window will open. From “Input:” drop down list box select “Merge Input 1” and then click “OK”.

• Select the second OLEDB source green arrow and connect to the “Merge” transformation.

 

• Right cilck on “Merge” transfomation and select “Edit” option, then “Merge Transformation Editor” window will open.Check the column mappings and click “OK” button.

• From data flow items drag and drop the “Flat File Destination” into designer surface. Connect the “Merge transformation” green arrow to the destination.
• Configure the “Flat File Destination” and then execute the package.  

Output File:

SORT TRANSFORMATION

Sort transformation sorts the input data in ascending or descending order and sends to the transformation output.

Example:

            I am having a source file “Sort”, that contain persons information shown below screen shot.

            The source data I want save it into another file “Sort_Target”, the target file data should be in descending order based on ID column.

Package creation:

• Open BIDS.
• From control flow items, drag and drop “Data flow task” into designer surface.
• Go to “Data flow”, from data flow items drag and drop “Flat file source” into designer surface. Configure the flat file source with source file.
• From Data flow items drag and drop the “Sort” transformation into designer surface.
• Connect the “Flat file source” green arrow to the “Sort” transformation.

• Right click on the “Sort” transformation and select “Edit” option. Then “Sort Transformation Editor” window will open.

• Select the required column to sort the data based on that column, and then select the Sort type.

• Click “OK” Button to complete the sort transformation mappings.
• From data flow items drag and drop the “Flat file destination” into designer surface and connect the “Sort” transformation green arrow to the destination.
• Edit the “Flat file destination” and configure the target file.

• Execute the package.

Target file “Sort_Target” data:

Target file data is in descending order.