Off-Topic: T-SQL: Replace all occurrences in all columns in all tables?

Have you ever been asked to write something where you knew from the start that it could posisbly wreak havok? Well I did today and since I haven’t posted in a while I thought I’d quickly share. Don’t worry I’ll be back blogging Sync, MVC and more soon (and hopefully tell you why I was gone).

Without talking too much about the real scenario that this was needed for, let me just say there is a large database that stores configuration data including a set of connection strings in multiple places spread across numerous tables. And our admins moved the database but didn’t know where exactly to change these config strings. I was the only dev left in the office at the time and they turned to me asking me if I could write a script that “replaced all occurrences of stringA with stringB throughout the whole database”.

Yikes. It sounds like trouble, you know it’s gonna hurt and I don’t want to be anywhere nearby when they press F5 to run the script. Ok, nevertheless I put together a little script that did just that, sent it to them and let’s see what happens next week. They have backups so even if everything goes wrong we can quickly restore things.

Here’s what I came up with. It’s not rocket science, but does the job. Use the script at your own risk (I didn’t find anything comparable on the web so maybe it will come in handy to someone). It shows some interesting usage of the system tables that I use now and again for things like this (when I don’t have my beloved SQLSMO). Have fun and use with caution (I can’t say that enough!)


------------------------------------------------------------
-- Name: STRING REPLACER
-- Author: ADUGGLEBY
-- Version: 08.05.2008
--
-- Description: Runs through all available tables in current
-- databases and replaces strings in text columns.
------------------------------------------------------------

-- PREPARE
SET NOCOUNT ON

-- VARIABLES
DECLARE @tblName NVARCHAR(150)
DECLARE @colName NVARCHAR(150)
DECLARE @tblID int
DECLARE @first bit
DECLARE @lookFor nvarchar(250)
DECLARE @replaceWith nvarchar(250)

-- CHANGE PARAMETERS
SET @lookFor = 'serverOld'
SET @replaceWith = 'serverNew'

-- TEXT VALUE DATA TYPES
DECLARE @supportedTypes TABLE ( xtype NVARCHAR(20) )
INSERT INTO @supportedTypes SELECT XTYPE FROM SYSTYPES
WHERE NAME IN ('varchar','char','nvarchar','nchar','xml' ) --

-- ALL USER TABLES
DECLARE cur_tables CURSOR FOR
SELECT SO.name, SO.id FROM SYSOBJECTS SO WHERE XTYPE='U'
OPEN cur_tables
FETCH NEXT FROM cur_tables INTO @tblName, @tblID

WHILE @@FETCH_STATUS = 0
BEGIN
    ------------------------------------------------------------------
    -- START INNER LOOP - All text columns, generate statement
    ------------------------------------------------------------------
    DECLARE @temp NVARCHAR(4000)
    DECLARE @count INT
    SELECT @count = COUNT(name) FROM SYSCOLUMNS WHERE ID = @tblID AND
        XTYPE IN (SELECT xtype FROM @supportedTypes)

    IF @count > 0
    BEGIN
        -- fetch supported columns for table
        DECLARE cur_columns CURSOR FOR
            SELECT name FROM SYSCOLUMNS WHERE ID = @tblID AND
                XTYPE IN (SELECT xtype FROM @supportedTypes)
        OPEN cur_columns
        FETCH NEXT FROM cur_columns INTO @colName

        -- generate opening UPDATE cmd
        SET @temp = '
    PRINT ''Replacing ' + @tblName + '''

    UPDATE ' + @tblName + ' SET
        '
        SET @first = 1

        -- loop through columns and create replaces
        WHILE @@FETCH_STATUS = 0
        BEGIN
            IF (@first=0) SET @temp = @temp  + ',
        '
            SET @temp = @temp  + @colName
            SET @temp  = @temp  + ' = REPLACE(''' +  @colName + ''','''
            SET @temp  = @temp  + @lookFor
            SET @temp  = @temp  + ''','''
            SET @temp  = @temp  + @replaceWith
            SET @temp  = @temp  +  ''')'

            SET @first = 0
            FETCH NEXT FROM cur_columns INTO @colName
        END

        PRINT @temp

        CLOSE cur_columns
        DEALLOCATE cur_columns
    END
    ------------------------------------------------------------------
    -- END INNER
    ------------------------------------------------------------------

    FETCH NEXT FROM cur_tables INTO @tblName, @tblID
END

CLOSE cur_tables
DEALLOCATE cur_tables

Academic on Tour: Community Launch 2008

I’m quite busy at the moment with the Academic Community Launch, so I’ll leave the talking to Christian, who has posted some information about why I’ve seen most of Austria’s big cities within the last week.

Info and Pictures of the ACL 2008

dot.net Magazin 5.08 Sync Framework Article and File Sync with VB.net

This morning I finally got my authors copy of the dot.net Magazin that I wrote an article on Microsoft Sync Framework. It’s in german, 7 pages long and you can buy it tomorrow.

Anyhow, I got an email this morning asking about one of the code listings and asking if there was anything different to do in VB.net. It’s been a while since I last did VB.net (I actually did more VB6 than VB.net) but I converted the sample from the magazine to VB.net. Just remember to reference Microsoft.Synchronization and Microsoft.Synchronization.Files.

(By the way I still haven’t got round to looking at CTP2 of the Sync Framework. All this stuff (and the article) applies to CTP1. Maybe there was a breaking change in CTP2 with regard to the FileSyncProvider. I must get round to looking at CTP2 a.s.a.p.)

Here is the code:


Imports Microsoft.Synchronization
Imports Microsoft.Synchronization.Files
Imports System.IO

Module Module1

    Sub Main()

        'Define directories
        Dim _sourceDir As String = "c:\temp\1"
        Dim _destDir As String = "c:\temp\2"

        'Define system ids
        Dim _sourceId As SyncId = New SyncId(New Guid("A4715EAF-341E-4ebf-90C3-71E8644CA6E8"))
        Dim _destId As SyncId = New SyncId(New Guid("FF766014-0DA4-401d-BDA1-0679326ECF96"))

        'Exclude some patterns
        Dim _syncFilter As FileSyncScopeFilter = New FileSyncScopeFilter()
        _syncFilter.AttributeExcludeMask = FileAttributes.Hidden Or FileAttributes.System
        _syncFilter.FileNameExcludes.Add("DoNotCopy.txt&quot ;)
        _syncFilter.SubdirectoryExcludes.Add("DoNotCopy&quot ;) 

        'Setup options (makes sure files aren't deleted for ever)
        Dim _syncOptions As FileSyncOptions = FileSyncOptions.RecycleDeletes Or FileSyncOptions.RecycleOverwrites

        'Do the sync
        Using _sourceProvider As FileSyncProvider = New FileSyncProvider(_sourceId, _sourceDir, _syncFilter, _syncOptions)
            Using _destinationProvider As FileSyncProvider = New FileSyncProvider(_destId, _destDir, _syncFilter, _syncOptions)

                Dim _syncAgent As SyncAgent = New SyncAgent()
                _syncAgent.LocalProvider = _sourceProvider
                _syncAgent.RemoteProvider = _destinationProvider
                _syncAgent.Direction = SyncDirection.UploadAndDownload
                _syncAgent.Synchronize()

            End Using
        End Using

    End Sub

End Module

TriggerCollection does not contain an Add method.

This is the kind of the thing you don’t want to see when you check out a project from SVN that should be compiling. It took a moment of thinking and a bit of reflectoring to find that the method was internal in the dll I was using:

SQL SMO TriggerCollection in reflector

The TriggerCollection is part of the table object in SQL Management Objects. I had the identical project compiling on a different PC. The funny thing was that the versions of the dll were identical. Only in reflector did I see a different version Guid:

// Assembly Microsoft.SqlServer.Smo, Version 9.0.242.0
Location: c:\Programme\Microsoft SQL Server\90\SDK\Assemblies\Microsoft.SqlServer.Smo.dll
Name: Microsoft.SqlServer.Smo, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91
Type: Library

// Module Microsoft.SqlServer.Smo.dll
Version: b74f0ed4-1269-420b-b67e-8bbae0c9c035
Location: c:\Programme\Microsoft SQL Server\90\SDK\Assemblies\Microsoft.SqlServer.Smo.dll
Size: 1555232 Bytes

This led me to think about the service pack level installed and I noticed you need Service Pack 2 to be able to use the Add method publically. (At least it wasn’t the other way round, they could had made it internal in the newer service pack *G*. Thankfully not.) Case solved, back to getting through this pile of work…

Jetbrains dotTrace: Profiling VS2008 Web Apps

We are currently running traces against a web applications that we are developing and one of my colleagues was telling me that the JetBrains dotTrace applications didn’t allow running profiles against the ASP.net web development server (it way grayed out). The reason was that only Visual Studio 2008 was installed, which by default puts the WebDev.WebServer.exe (which is the ASP.net web development server) into the c:\<program files>\<shared files>\Microsoft Shares\DevServer directory instead of the old Visual Studio 2005 path which was c:\windows\Microsoft .NET\Framework\v2.x.y.z\.

There is a way to profile against the VS08 web server as described here, but you can also just copy the WebDev.WebServer.exe from the one directory to the Windows\Framework directory. This probably isn’t the best way to do it (because patches will only go to the original install directory), but with it dotTrace can start a profiling session with the ASP.net web server.

Microsoft Sync Framework CTP2 is out…

Tomorrow I’m off to the academic community launch and will have to wait until next week to take a look at it, but it’s available for download:

http://www.microsoft.com/downloads/details.aspx?FamilyId=C88BA2D1-CEF3-4149-B301-9B056E7FB1E6&displaylang=en

Remember to uninstall all previous editions.

Technorati Tags:

A Reusable Secure Login Form

For the impatient, source and binaries and quick explanation available at: http://code.msdn.microsoft.com/UtilsCommonViews/

How many of the last ten projects you implemented had a login form of some kind? Thanks to all those wonderful apis that you coded applications against, probably the majority of them did. Now try to remember how many of those login forms used strings for storing passwords in memory and how many saved the password to a settings file in plain text. I know most of my small utilities did and I’m not proud of it, but it happened and I’m almost sure you have some of those lying around in your src folder.

Anyway, today I had a couple of hours of spare time and decided to implement a reusable secure login form. My goals were to use SecureString for storing the password in memory and encrypt the username and password in a settings file for storing them between the sessions. I started off with a little research about the components I’d need to use.

SecureString is a class that was introduced in .NET Framework 2.0. Why not simply use a string for storing a password? Well let’s have the MSDN do the explaining:

An instance of the System.String class is both immutable and [...] cannot be programmatically [...] deleted from computer memory. Consequently [...] there is a risk the information [stored in it] could be revealed after it is used [...].

A SecureString object is similar to a String object in that it has a text value. However, the value of a SecureString object is automatically encrypted, can be modified until your application marks it as read-only, and can be deleted from computer memory by either your application or the .NET Framework garbage collector.

That’s a great scenario for our password. Next up I found this great secure textbox control that handles the user input into a SecureString object. The original control and source can de downloaded at: http://www.theglavs.com/DownloadItem.aspx?FileID=46. (Thanks to Glav and Dominik Zemp who created or extended the control.)

Then I wrote this simple extension method to get the characters of the contents of the SecureString because it can get quite dirty to do it inline. It uses requires execution of unmanaged code and some marshalling.


public static Char[] GetCharacters(this SecureString secureString)
{
	if (secureString == null)
		throw new ArgumentNullException("secureString");

	lock (secureString)
	{
		char[] _chars = new char[secureString.Length];
		IntPtr _ptrToChars = IntPtr.Zero
		try
		{
			_ptrToChars = Marshal.SecureStringToBSTR(secureString);
			Marshal.Copy(_ptrToChars, _chars, 0, secureString.Length);
		}
		finally
		{
			if (_ptrToChars != IntPtr.Zero)
				Marshal.ZeroFreeBSTR(_ptrToChars);
		}

		return _chars;
	}
}

The second feature was to encrypt the username and password in a file. I looked at encrypting the configuration file but decided against that path and went with encrypting the data using the ProtectedData API (in System.Security.Cryptography). It’s a simple call to the static method:


ProtectedData.Protect(byte[] data, byte[] entropy, DataProtectionScope);

The entropy specifies a kind of salt for your application and the DataProtectionScope can be set so that only the current user can decrypt the data or to local machine. The library will save a file UserData.bin to the local application directory (including the calling assemblies name). Additionally I decided to encrypt the file using the FileInfo.Encrypt method. This works only on NTFS, so the library will currently only work on NTFS systems. Now that I’ve written that sentence I think I’ll make that optional in the next release.Which brings me to the release and the source code. I want to try out MSDN code gallery so I’ve published the project here: http://code.msdn.microsoft.com/UtilsCommonViews/

Using the library is very simple:


LoginController _loginController = new LoginController();
_loginController.GetCredentials();

string _username = _loginController.Username;
char[] _password = _loginController.Password;
// _password needs to be zero'ed a.s.a.p. after usage

That last comment is important. If you don’t zero out that char array, the password will be floating around in memory.

Comments, suggestions are of course welcome! Here or in the MSDN code gallery.

Note: The project uses another utility library (Utils.Extensions) of mine that I’ve built-up over the last weeks that contains a few extension methods for common stuff I was missing in the BCL. The source is not clean enough yet for publishing but it’s in the pipe. For now it’s only available in this project as a release dll.

LINQ to SQL for SQL Compact Edition - Did you know?

You can’t actually drag and drop a SQL Compact Edition 3.5 database to the LINQ to SQL designer. But it is very easy to create the classes and work beautifully. Fire up a VS08 prompt and find the SDF file that your database is stored in, then run the following command:

Prompt: SqlMetal /code:DotNetBooksDB.cs /pluralize /namespace:Books.Client.Library.Store.SqlCe dotNetBooks.sdf
Microsoft (R) Database Mapping Generator 2008 version 1.00.21022
for Microsoft (R) .NET Framework version 3.5
Copyright (C) Microsoft Corporation. All rights reserved.

The parameters are simple.

/code = the path where you would like to save the generated code.

/pluralize = changes the table book into books so you can write code like this: var x = from b in _ctx.Books (just looks a bit nicer)

/namespace = which namespace would you like your classes to have

and finally the path to the sdf file.

There is no output, but if you run the command your csharp should have been generated and just include that file into your project (and switch the project to .net Framework 3.5).

Sync Framework: Where does the metadata come from for SqlCeClientSyncProvider (Sync Services for ADO.net 2.0)?

I was preparing some demos for the upcoming Academic Community Launch Tour that we - a bunch of Microsoft Student Partners in Austria - are organizing (see http://anuga.at for details). I had just finished a wonderful little app using SQL Compact Edition, Sync Framework, ASP.net MVC and all the funky stuff. Just before I wanted to call it a night I noticed the sync portion of my app suddenly wasn’t working anymore. Somehow I had done something bad and the sync framework wasn’t finding any rows that had changed. This morning I got up and took a closer look underneath the covers of SqlCeClientSyncProvider. This is what I found out.

A word of warning: The information here only applies to Sync Services for ADO.net and is based on Version 2.0 with some help of Reflector. Use the information and my interpretation with caution. Especially beware of the fact that Sync Services for ADO.net in the Sync Framework are handled completely differently than Sync Services for File Systems and Custom Sync Providers.

The sync framework uses anchors to determine what rows have changed. These anchors are compared to the metadata in the table. When creating the tables on the server you probably inserted columns such as InsertTimestamp or UpdateTimestamp. That’s the metadata. Fine, but what happens on the client side in a SQLce database? At first I thought the metadata columns are transferred to the client in the schema (remember the SqlCeClientSyncProvider retrieves the schema to create on the table from the server) and then used for the change tracking there too.

Well not quite. In the app I’m working on I created the SQLce database first (and used used SQLMetal to generate my Linq2Sql classes by the way which is really easy. I’ll do another post in a moment). SQLce doesn’t support triggers so I only created a timestamp column called ‘TS’. When it came to sync’ing the client with a server database, that looked similar but had more metadata columns, I noticed I had no idea how the metadata part worked on the client. (It’s actually quite nice too know that for standard use cases it just works out of the book.)

Enough bla, let’s get to the details. Thanks to a couple of hours with reflector and running queries against my SQLce database I found out where the metadata goes.

Preparing my custom tables for the meta data

When you configure your SqlCeClientSyncProvider you add SyncTables with a couple of options:


foreach (SyncTable _syncTable in syncTables)
{
_syncTable.CreationOption = TableCreationOption.UploadExistingOrCreateNewTable;
_syncTable.SyncTableTransferOption = SyncTableTransferOption.Bidirectional;
m_clientProv.Configuration.SyncTables.Add(_syncTable);
//
// This would recreate the schema for each table on each sync, which resets the timestamps to 0.
// m_clientProv.CreateSchema(_syncTable, null);
}

You can see a line at the bottom that I commented out. I figured that since my client database was in place and I specified UploadExistingOrCreateNewTable that my client database needed to be adapted to include the sync metadata. So I called CreateSchema on the provider which does just that. But you don’t want to create the schema every time, so I should actually move the call into my database initialization code, but I didn’t want to mix my sync code with my DAL code and I didn’t need to. CreateSchema is called at the beginning of the sync session only if the metadata is not found in the table.

Once that method has run all sync tables now have two extra columns: __sysChangeTxBsn and __sysInsertTxBsn. These are initialized with 0 and contain a binary timestamp.

CreateSchema also creates some more tables. First of all it creates a sync subscription table called ‘__sysSyncSubscriptions’. This contains the client’s id and it’s pairing with any server sync providers. Next up is an article table called ‘__sysSyncArticles’. An article is just a sync table, so it contains the table name, the last sent and last received anchors for a specific table.

Once it has done that it just switched on the built-in SQLce change tracking which in turn updates the metadata columns when ever a change occurs.

But wait there is one thing missing. What happens to deleted rows? So far we don’t have any tombstone or archive tables. Actually we do, a table called ‘__sysOcsDeletedRows’ was created that is used by the SQLce change tracking to store deleted rows. It has columns ‘__sysTName’ (the table name), ‘__sysRK’ (the reference key = primary key that was deleted) and the timestamp value when it was deleted and inserted.

Windows 2008 - 3 things I wish I had known…

Ok, so I decided to setup a new VPC image with Windows 2008, SQL 2005, WSS 3.0 and Visual Studio 2008. Basically everything is going quite well, but there are a couple of things I encountered while installing, that I thought I would share.

1. Installing SQL Server 2005 with Reporting Services

w08_features From previous experience I knew that Reporting Services requires IIS if you want to install it. The first thing you notice that the server manager has changed a little. I went ahead and installed ASP.net and the Application Server which also installed IIS, but when I started my SQL Server Setup Reporting Services was disabled, so there was something wrong with the IIS install. It took a few minutes, but it seems you actually have to install the IIS role separately. (I had chosen ASP.net, and activated further options manually.) Anyway once the following picture appeared I was set to restart my setup. This time no problem.

2. Internet Explorer Enhanced Security

I don’t like it, so I always go and uninstall it after finishing my base setup. (Usually when I start downloading stuff I need to install.) So I go to my Control Panel, go to Programs & Features and click on Manage Windows Features. Woha, the server manager appears. Ok, this is different to Windows 2003. I think it was David Platt who said something along te lines of “If you want people to use it, make it obvious. If you don’t want people to switch it off, hide it”. Well done, it’s hidden.

It took a bit of googling (sorry, but live’ling just doesn’t sound well as a verb) to find the right option to change. The feature is called IE ESC which can be found in the overview page of server manager. Now I know, now you know.

w08_ieesc

3. Installing Windows Sharepoint Services 3.0

Ok, I live’d for “Windows Sharepoint Services 3.0″ (I really did use live this time.) and downloaded the package. Started Sharepoint.exe and voila “This program is blocked.”. WTF? I read on and it asks you to install the setup package with the latest service pack and won’t accept anything else. Ok, my mistake, don’t blindly take the first link presented (I bet google would have pointed me to the service pack 1 download package - but I didn’t check).

Next Page »


Subscribe / Search

ic_3dlogo_07korea.gif
msplogo_small.jpg
mcprgb.png

 

May 2008
M T W T F S S
« Apr    
 1234
567891011
12131415161718
19202122232425
262728293031  

Blog Stats

  • 6,328 hits