Hi there!

Photo credit: https://www.flickr.com/photos/kalexanderson/5696097036/

Hi there, visitor #292756!

On this blog, you’ll find articles about TSQL, SSIS, Databases, Administration and SQL Server. Mostly.

Do you like awesome learning opportunities or resources? Me too! That’s why I share stuff on this blog, Twitter or Facebook whenever I come across any that is worth your time. Please subscribe via email or connect on social media so that we can keep in touch.

What topics would you like to read about? Let me know via comments, email or a tweet.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

 

Did I mention that I’m on Twitter too?

Example of SSIS Foreach Loop Item Enumerator and Dynamic OLEDB Connection

One of the looping constructs available in SSIS packages is the Foreach Loop Container. One of its enumerator options is the For Each Item Enumerator. It enumerates through a list of items populated at design time. The list is static at execution time, i.e. the list items can not be dynamically loaded from a configuration file or a another source like an SQL table. Probably due to the static nature of the item list, this enumerator type is not used too often in packages.

On the other hand, there are some scenarios where a static list can prove to be useful. For example –

  • no need to create a database table just to pass on values for the Foreach Loop
  • enforce a specific order to the parameters passed to the loop
  • initialize variables values with the Item Enumerator instead of doing that in a Script task

An another frequent requirement in SSIS packages is to execute a set of operations against different SQL Servers or databases. The package should change the OLE DB connection string dynamically during execution to point towards the appropriate target. A looping construct iterates through the list of servers and databases, and the package Expressions change the OLE DB connection properties.

I am creating this step-by-step scenario to demonstrate the use of a Foreach Item Enumerator and use it to make a dynamic OLEDB connection. A single query is executed in a loop against three different SQL databases and the output is exported to a single text file.

The steps –

Add a Foreach Loop Container to your SSIS package. Double-click on it to open its editor. Click on the Collections tab on the left of the editor. Select the enumerator as Foreach Item Enumerator. Click on the Columns… button. Add two columns of String type. As you add columns, the editor gives them names like Column0, Column1 and so on, and there is no way to customize these names.

ItemEnumeratorDemo1

Populate the column values by simply typing in the rows. Use Column0 for server names and Column1 for corresponding database names on that server. I have used my local server for all three rows but you can use your own server names here. This is the list of items that are iterated by the Foreach Loop.

ItemEnumeratorDemo2

Now you need two variables that will hold the values shredded from the item list as the Foreach Loop iterates through the rows. These variables are used later in the Property Expressions of the OLEDB connection to make it dynamic.

Click on Variable Mappings tab on the left and then on the <New Variable…>

ItemEnumeratorDemo3

Add two variables of String type, namely ServerName and DatabaseName.

ItemEnumeratorDemo4

The Index 0 and 1 of the variables correspond to the Column0 and Column1 of the Foreach Item list respectively.

ItemEnumeratorDemo5

That is all there is to it. Now the loop is ready to iterate through the item rows.

Let us move on to the dynamic OLEDB connection. Add a Data Flow Task to the Foreach Loop Container.

ItemEnumeratorDemo6

Double-click the Data Flow Task to view its design surface. Now add one OLE DB Source.

ItemEnumeratorDemo7

Double-click OLE DB Source to set its properties. First window is of its connection manager.

ItemEnumeratorDemo8

Once the connection information is done, you will view its properties editor. Set the Data Access Mode to SQL Command and put the following query in the SQL Command Text –

select
@@SERVERNAME as ServerName
,DB_NAME() as DatabaseName
,GETDATE() as LogDateTime
,COUNT(*) as NumberOfIndexes
from sys.indexes;

ItemEnumeratorDemo9

Clicking the Columns tab is important so that the SSIS can find the metadata like column names, data types etc. from the underlying tables/views. If you don’t do this then you’ll find later that SSIS does not know of this metadata.

ItemEnumeratorDemo10

Again, it is a good idea to drag the connector from OLEDB source to the Flat File destination before modifying the Flat File destination properties. In this way, the SSIS Designer helps you by matching the destination with the source metadata automatically. Otherwise you have to perform some extra steps to match columns. I hope you like to save time as much as I do!

ItemEnumeratorDemo11

Give a file path and check the box for Column Names in the First Row.

ItemEnumeratorDemo12

Verify the column mappings in the Mappings tab.

ItemEnumeratorDemo13

The trick to make the OLEDB connection string dynamic lies in the next couple of steps where you edit the properties of the OLEDB connection and assign expressions to its ServerName and InitialCatalog properties.

ItemEnumeratorDemo14

The property ServerName gets the variable @[User::ServerName] and the property InitialCatalog gets the variable @[User::DatabaseName]. Remember that these variables will get their values modified during each iteration of the Foreach Loop and as a result the OLEDB connection string will get modified too.

ItemEnumeratorDemo15

The last step is just to make sure that data is appended to the destination text file instead of being overwritten with each loop iteration and new source connection.

ItemEnumeratorDemo16

Spring Time!

Keeping up with the spirit of spring season, I’ve refreshed the look of the blog by using the “Flounder” theme. I was using the “Enterprise” theme earlier.

What I like about Flounder is :

  • the colour scheme
  • the overall font choice
  • a well-defined sidebar
  • and most of all, its mobile device compatibility. It readjusts from 3-column layout to a 2-column or a 1-column layout as the screen size becomes smaller. That really allows very easy browsing on a tablet or a cell phone, for folks who don’t use the WordPress app, by completely avoiding horizontal scrolling.

I have also changed the blog’s tag line from “Learnings as a SQL Server DBA and Developer” to “Erudition as a SQL DBA and Developer”.

er·u·di·tion [er-yoo-dish-uhn, er-oo-]

noun
knowledge acquired by study, research, etc.; learning; scholarship.

And finally, I’ve added my Twitter handle @TsqlREX to the publicize section of the blog so that a new post is announced on Twitter too. I’m not a big Twitter user right now but I plan to learn more about it in this year.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.