Some facts:

• SQL Server Reporting Services were great in the past. SSRS 2008 are better.
• MSDN is great. MSDN has lots of details. Sometimes it’s too much if you are under pressure.
• Google usually gets me the information I want faster. Especially when I need to get something done quickly.

All in all this post is for those of you that need to learn how to use SQL Server Reporting Services fast or need to solve the above mentioned problem fast, without knowing too much about the details. I read a lot of and like these introductory posts and so now I’m writing another one, here goes.

I assume you have SQL Server Reporting Services already setup (by your grand one-and-only unmissable sys-admins or by yourself, but in the latter case you probably are not the target audience for this) and you have access to the all required tools. Talking about tools, there are three you should know of:

- RSConfigTool - (Start menu > SQL Server 2008 > Configuration > Reporting Services Configuration): What do I need it for: Troubleshooting, Finding Urls, Configuration Report Server. Having problems trying to connect. See Additional Tip #1.

- Report Manager - (http://<yourserver>/reports): What do I need it for: Managing your reports (you won’t actually see anything on a clean install yet)

- Business Intelligence Studio - (Start menu > SQL Server 2008 > SQL Server Business Intelligence Development Studio): What do I need it for: Creating and editing your reports

and of course you will need SQL Server Management Studio for managing the data the report is based upon.

(side note: This tutorial is based on the 2008 version, can’t say how many differences there are between this and the older versions.)

1. Create your views

There are ways to aggregate your data in the report itself, but it’s easier and cleaner to use SQL views to provide the data for the report and use Reporting Services only for the report. So go to SSMS and create your view.

2. Create a new BIDS (Business Intelligence Development Studio) solution based on the “Report Server Project” Template. Add a new report by clicking on the Report node in the Solution Explorer. Next we need to connect to a data source. Give it a name, click edit and connect to your database.

3. Click next and the Query Designer appears. Here you enter the T-SQL statement for your data. Ideally it’s just a “SELECT * FROM yourView” (or select each column distinctly). You can test it using ‘Run Query’. You can also choose to use data from a stored procedure if you like, just select the appropriate option.

4. Design your report. You can add elements to the report surface, by right clicking on the center surface and select them from the insert menu. The most important are Header, Footer and Table. In the Header and Footer you can add Textboxes with text or by right-clicking on them and choosing expressoin you can choose some variable input. For example page number, report name etc. In the table you can drag columns from the “Report Data” (on the left showing the data source you just added). When you are finished, you might have something like this. Click preview to get a glimpse at what your boss might get.

5. Now let’s deploy it to the report server. First we must set the report server url. Right-click on the solution in the Solution Explorer and click on Properties. You will see the following dialog. Change the TargetReportFolder value to something you wish and provide the TargetServerURL as http://yourserver/reportserver (not just /report!). Press save. Now right-click on the solution again and select Deploy.

6. When it finishes open the url http://yourserver/report (not /reportserver!). You will see something like this. Go ahead and click on Test.Reports (or whatever you specified as TargetReportFolder) and then click on your test report. You will be greeted with the web version of your report.

7. Now your Boss will not want to go the web page to get that report, so let’s look at sending it to him per email automatically. First we need to change the security settings for connecting to the data source. In the report page, click on Properties > Data Sources. Change the “Connect using” to “Credentials stored securely in the report server” and specify an account that has access to the data. Also check “Use as Windows credentials…”.

8. Now you can go to Subscriptions. Click on email. Don’t have email as an option? See Additional Tip #2. Enter the recipient information as you wish and select a schedule for the sending. If you press save and encounter an error that ‘SQL Server Agent is not running’ either figure it out yourself what the problem could be or read Additional Tip #3 :). Once you have finished, lean back and let SSRS make your boss happy and in turn make you happy.

So we have seen the absolute basics of report creation. A quick and dirty introduction. This should cover your first encounters with SSRS. Hope it helps someone!

Stop reading here unless you are Troubleshooting…

Additional Tip #1:

I try connecting with SSMS (Management Studio) and it won’t connect. Maybe you renamed the server? Open RSConfigTool, go to Database > Change Database > Choose an existing report server database. Just choose the existing one, and let it run through it’s config. That should repair everything and you should be able to connect using SSMS again.

Additional Tip #2:

You don’t have email in the options for delivery in the subscriptions dialog? Probably it’s not configured. Thankfully that is very easy. You start the RSConfigTool. Connect and go to the Email tab. There you just specify a sender address and the SMTP server you would like to use. Click apply and voila. The server will automatically restart and the now the email option appears.

Additional Tip #3:

Saving the subscription results in an error message saying SQL Server Agent is not running. This is a DB or sysadmin issue, but just fire up SQL Server Configuration Manager (in the Start Menu under SQL Server 2008 > Configuration) then click on SQL Server Services, right-click on the SQL Server Agent > Properties. Go to the service tab, change Start Mode to Automatic and press OK. On the next reboot it will start automatically, but you still need to start it once for now. Right-click on the ‘SQL Server Agent’ and press Start.

So after Paris it’s back to work again and this week I need to prep a VM for a customer project. I already received a clean Windows 2003 Machine and on Monday I installed SQL Server 2005 before I went. I should have seen this coming, but anyhow today I wanted to install a Mini-SAP 4.6D.

MiniSAP (now called MiniWAS) is available for Linux as a download but the Windows version is only distributed in the shop or as part of the ABAP Objects book (I hope the program manager who made that decision never crosses my way). Anyhow, I got the CDs somehow (don’t ask) and looked at the Documentation.doc file. (Actually I of course immediatelly clicked on setup.bat and bang it crashed, but doesn’t it sound calmer when I say I looked at the documentation first?).

Seems that I have versions of the CD that contain a production bug, so after doing some manually editing in the install files (as described in the documentation) I was ready to go. It tries to install MSDE on the machine, but that fails with an (’Instance name not valid’) but I have a big SQL Server 2005 running on the machine, so I just commented out the MSDE install line in the setup.bat and it continued.

The first error I got was a collation problem. Having installed SQL Server 2005 with the default collation SAP expected it to be SQL_Latin1_General_Cp850_BIN.

Thankfully it was a clean sql server so I only needed to mount my SQL Server DVD again and run:

setup.exe /qb INSTANCENAME=MSSQLSERVER REINSTALL=SQL_Engine REBUILDDATABASE=1 SQLCOLLATION=SQL_Latin1_General_Cp850_BIN

(you can specify SAPWD if you are using mixed and not Windows Authentication). You will be prompted:
You have chosen to re-install, and therefore overwrite the system databases. Are you sure you want to proceed? Yes! (Don’t do this on a live production server. Ever.)

Started setup again, and next I was presented with an error for a registry key it couldn’t find in ‘Software\Microsoft\MSSQLServer\MSSQLServer\Parameters’. I found the MSSQLServer node but had to manually create the Parameters node. And restarted setup.bat and this time it finished without problem! Hurray.

Next up was the GUI which the original setup tried installing but couldn’t locate the setup file. It’s in the MINIGUI folder. Once that is installed, just double click the ‘Start R3 MBS’ on the desktop, then fire up the GUI and follow the rest of the instructions in the documentation. Don’t forget to install the latest Kernel Patches from the SAP Site. Voila. Hope it helps.

So a couple of hours ago the top 12 finalists were announced that rise to round 2 of the world finals. Since you are reading this - you might guess - we are not really under much pressure because we didn’t rise up to the next round. Interestingly enough though I honestly thought we were quite good this year (remember I do have quite a bit of grounds for comparison) but it seems the judges really liked us, but they liked others even more. Anyway you win a few, you lose a few and you can’t really talk about losing if you are sitting in a four star hotel near the Eiffel tour relaxing and having fun. There are 49 other teams experiencing the same and the excitement is at a bit of a low, but there is one thing I know from the last years: Cry for about half an hour and then forget it and move on. Otherwise you lose the beauty of the remaining finals.

Anyhow, I promised some pictures but I though I’d just pop them up on Flickr for everybody to see. If you are interested which interesting projects did advance try the official blog. Will I participate again next year? Who knows. It depends. I could finally get the record for most imagine cup world finals, I could finally advance to a TOP12, but on the other hand I’m getting old, it’s a lot of work (but always worth it). But I have to start balancing work life, student life and all other activities.

So you’ll probably hear more about Paris now, I’ll blog some more in the next few days. I’ll leave you with some screenshots of BioMatch.

	This is the newest module we developed for the world finals. An environmental simulation that can forecast which species will become endangered and why.
	The two standard modules we developed for the first round, which were the Bayes prediction of where the species could most likely survive and the self organizing net showing clusters of species.

Just for completeness in case you are interested. The application is built using WPF (my first WPF application actually) and uses a couple of data mining algorithms to analyse data aggregated from different sources (such as the US PLANTS database) to simplify it, group it and show similarities. The visualisation helps the scientists to use and understand the simplified data.

As far as press is concerned we had interviews with a french magazine, with some netherland media of course with the Austrian press OE3 and derStandard (I’ll post them when they come out). The judges generally were impressed and expressed it at the end of the sessions. In the second presentation they even thanked us personally (something I hadn’t experience before) but I’m looking forward to seeing the presentations of the others on Monday to gain some insight into why they were better. To finish off with a Soma’esque reference to a past Imagine Cup world finals.

Namaste!

It’s been quiet for a while due to my participation in the Imagine Cup (again). Not only have I participated in each Imagine Cup since the start:

- 2008: BioMatch - www.biomatch.at (I’ll report some more on that of course later…)
- 2007: Kalie - Decision making system to support students during studying.
- 2006: MyCaps - e-Sports Framework with real-life movement integration for a healthier life
- 2005: Gecco - Global Emergency Clinic Coordination - Fast automatic help for people requiring constant medical monitoring to break their boundaries.
- 2004: NoName - This was the worst year by far because we had not realized there was a theme, but we still took part but did not get very far.
- 2003: SAMANTA - System for Astute Management of Time-aware Archictectures - The beginning of it all and we had an intelligent organisation system for trade fares. (2nd place locals)

but it is also my third time at the WORLD FINALS (2005: Yokohama, Japan; 2006: Delhi/Agra, India; 2008: Paris, France). I have heard that there is a polish short film team, that I met the years before (but not this time) that have also been here three times. So I wonder if I have to participate again next year to make the record official (on the otherhand I think there may actually be some guys in the IT challenge that been here more often…).

Anyway if you are interested in why I take on this challenge every year take a look at an interview I gave to the Danish Academic Relations Manager Martin Esmann over on his blog: http://blogs.msdn.com/mesmann/archive/2008/07/04/imagine-cup-2008-torsdag-dag-1.aspx

That’s all for now, pictures will follow this evening. We have to rehearse the presentation once more. Just under 3 hours to go until round 1. Wish us luck!

Well we all know companies usually set their release dates way too early and they get corrected on the way to that date (not picking on any particular big software company). But the other extreme seems to be brutal honesty. Not quite sure which one I prefer? Will we be using agile software methodologies in 2020 at all? Will there be software at all? Just too much for me to cope.

Ok, I’m starting to get fond of these “a slice of reality” or “what am I going to show you” title blog posts…

So once again this piece of code is a bit drastic. It’s neatly written if you ask me and should show you how you can replace the file security settings (ACLs) with some other files security settings (same for directories). The only meaningfull place to do this (off the top of my head) is when you have two identical files and for some reason the one file has the correct permissions and the other doesn’t. But daily developer business is always full of these little request, and it’s like a personal search for the holy grail - you just have to give it a try.

So if you ever want to copy the exact permission structure from one directory tree to another (for example you used some copy tool that doesn’t copy the permissions) then you can use this little tool to do just that.

A word of warning (as always). Have a backup of your permissions (most probably you will, because you need to copy them from somewhere) and of course run this tool at your own risk. The binary is available here and is compiled from the source you see below. Have fun and tell me if you find bugs or if it’s useful.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Security.AccessControl;
using System.Diagnostics;
using System.Security.Principal;

namespace AlexDuggleby.Tools.MirrorNTFSPermissions
{
    class Program
    {
        #region Properties

        /// <summary>
        /// The root we are copying from
        /// </summary>
        private static DirectoryInfo m_diFromRoot;

        /// <summary>
        /// The root we are copying to
        /// </summary>
        private static DirectoryInfo m_diToRoot;

        /// <summary>
        /// The log file to write to
        /// </summary>
        private static FileInfo m_fiLog;

        #endregion

        #region Main Operation

        static void Main(string[] args)
        {
            // Check args and output usage if required
            if (args.Length != 3)
            {
                OutputUsage();
                System.Environment.Exit(-1);
                return;
            }

            if (PopulateAndCheckDirectoriesExist(args) && ConfirmUserAction())
            {
                try
                {
                    Trace.Listeners.Add(new ConsoleTraceListener());
                    Trace.Listeners.Add(new TextWriterTraceListener(m_fiLog.FullName));

                    WriteTraceHeader();

                    Console.Clear();
                    CopyPermissions(m_diFromRoot, m_diToRoot, true);
                    WriteTraceFooter();
                }
                catch (ApplicationException aex)
                {
                    OutputError("Application", aex);
                }
                catch (Exception ex)
                {
                    OutputError("Critical", ex);
                }
            }

        }

        private static void CopyDirectorySecurity(DirectoryInfo diFrom, DirectoryInfo diTo, bool skipRootNameCheck, out bool continueInThisDirectory)
        {
            continueInThisDirectory = true;

            try
            {
                // Name comparison
                if (diFrom.Name != diTo.Name && !skipRootNameCheck)
                    throw new ApplicationException(string.Format("During processing the directory names changed! '{0}'-'{1}'",
                        diFrom.Name, diTo.Name));

                // Set my own permissions
                DirectorySecurity _dsFrom = Directory.GetAccessControl(diFrom.FullName, AccessControlSections.All);
                DirectorySecurity _dsTo = new DirectorySecurity();
                _dsTo.SetSecurityDescriptorSddlForm(_dsFrom.GetSecurityDescriptorSddlForm(AccessControlSections.All));
                Directory.SetAccessControl(diTo.FullName, _dsTo);

                // Tell the user what we are doing
                Trace.WriteLine(string.Format("Copying directory security from {0} to {1}",
                        diFrom.FullName, diTo.FullName));
            }
            catch (PathTooLongException _plex)
            {
                Trace.WriteLine(string.Format("[WARN] Path too long do continue with this directory '{0}'",
                        diFrom.FullName));
                continueInThisDirectory = false; // Abort because all files and subdirs will be affected
            }
            catch (Exception _ex) // General catch here, because we would like the batch to continue
            {
                Trace.WriteLine(string.Format("[ERROR] Error while doing current directory '{0}': {1}",
                        diFrom.FullName, _ex.ToString()));
                // We will continue here, since error type is undefined.
                continueInThisDirectory = true;
            }
        }

        #endregion

        #region File/Directory I/O and Security

        private static void CopyPermissions(DirectoryInfo diFrom, DirectoryInfo diTo, bool skipRootNameCheck)
        {
            bool _continueInThisDirectory;

            CopyDirectorySecurity(diFrom, diTo, skipRootNameCheck, out _continueInThisDirectory);

            if (_continueInThisDirectory)
            {
                // Now set files
                foreach (FileInfo _fiTo in diTo.GetFiles())
                {
                    CopyFileSecurity(diFrom, _fiTo);
                }

                // Get all the -to- subdirs ...
                foreach (DirectoryInfo _toSubDir in diTo.GetDirectories())
                {
                    // ... and check if they also exist in the from directory
                    DirectoryInfo _fromSubDir = diFrom.GetDirectories(_toSubDir.Name, SearchOption.TopDirectoryOnly).FirstOrDefault(d => d.Name == _toSubDir.Name);
                    if (_fromSubDir != null)
                    {
                        // Recursive
                        CopyPermissions(_fromSubDir, _toSubDir, false);
                    }
                }
            }
        }

        private static void CopyFileSecurity(DirectoryInfo diFrom, FileInfo fiTo)
        {
            // Turn the paths into relative paths based on the root dirs we started from
            string _relativeFromPath = fiTo.Directory.FullName.ToLower()
                .Replace(m_diToRoot.FullName.ToLower(), "");
            string _relativeToPath = diFrom.FullName.ToLower()
                .Replace(m_diFromRoot.FullName.ToLower(), "");

            // now compare those relative paths, just to be sure
            if (_relativeFromPath != _relativeToPath)
                throw new ApplicationException(string.Format(
                    "Subdirectories went out of sync during processing! '{0}'-'{1}'", _relativeFromPath, _relativeToPath));

            FileInfo _fiFrom = new FileInfo(Path.Combine(diFrom.FullName, fiTo.Name));

            if (_fiFrom.Exists) // if it doesn't exist in the from we will ignore...
            {
                // Copy the file permissions
                FileSecurity _fsFrom = File.GetAccessControl(_fiFrom.FullName);

                // Sometimes there are now access rules on the item, so only the owner can access it
                if (File.GetAccessControl(fiTo.FullName).GetAccessRules(true, true, typeof(NTAccount)).Count == 0)
                {
                    // Tell the user what we are doing
                    Trace.WriteLine(string.Format("File can only be accessed by owner: {0}", fiTo.FullName));
                }
                else
                {
                    try
                    {
                        // Copy the file permissions
                        FileSecurity _fsTo = new FileSecurity();
                        _fsTo.SetSecurityDescriptorSddlForm(_fsFrom.GetSecurityDescriptorSddlForm(AccessControlSections.All));
                        File.SetAccessControl(fiTo.FullName, _fsTo);

                        // Tell the user what we are doing
                        Trace.WriteLine(string.Format("Copying file security from {0} to {1}",
                            _fiFrom.FullName, fiTo.FullName));
                    }
                    catch (Exception _ex)
                    {
                        // Tell the user what went wrong
                        Trace.WriteLine(string.Format("[ERROR] Error while copying file security from {0} to {1}: {2}",
                            _fiFrom.FullName, fiTo.FullName, _ex.ToString()));
                    }
                }
            }
            else
            {
                Trace.WriteLine(string.Format("File did not exist in original tree: {0}",
                 fiTo.FullName));
            }
        }

        #endregion

        #region User I/O and Tracing

        private static void WriteTraceHeader()
        {
            Trace.WriteLine("*****************************************");
            Trace.WriteLine("Mirror NTFS Permissions");
            Trace.WriteLine("Version 1.0 - 27.05.2008");
            Trace.WriteLine("Alex Duggleby - http://alexduggleby.com");
            Trace.WriteLine("*****************************************");
            Trace.WriteLine("== New run @ " + DateTime.Now.ToString() + "");
            Trace.WriteLine("== From : " + m_diFromRoot.FullName + "");
            Trace.WriteLine("== To   : " + m_diToRoot.FullName + "");
            Trace.WriteLine("=========================================");
        }

        private static void WriteTraceFooter()
        {
            Console.ForegroundColor = ConsoleColor.Green;
            Trace.WriteLine("=========================================");
            Trace.WriteLine("Finished!");
            Trace.WriteLine("=========================================");
            Console.ForegroundColor = ConsoleColor.White;
            Trace.Flush();
        }

        private static void OutputError(string errType, Exception ex)
        {
            Console.ForegroundColor = ConsoleColor.Red;
            string _s = string.Format("***********************{1}{2} error: {0}", ex.ToString(), Environment.NewLine, errType);
            Trace.WriteLine(_s);
            Console.ForegroundColor = ConsoleColor.White;
            OutputPressKeyToExit();
            Trace.Flush();
        }

        private static void OutputPressKeyToExit()
        {
            Console.WriteLine();
            Console.ForegroundColor = ConsoleColor.White;
            Console.WriteLine("{0}Press any key to exit", Environment.NewLine);
            Console.ReadKey();
        }

        private static void OutputUsage()
        {
            Console.WriteLine("Mirror NTFS Permissions");
            Console.WriteLine("by Alex Duggleby - http://alexduggleby.com");
            Console.WriteLine();
            Console.WriteLine("Usage: application.exe <path1> <path2> <log-file>");
            Console.WriteLine("Replaces all permissions on all subdirectory and files in path2 with those from path1 if they exist there.");
            Console.WriteLine("The <log-file> is used to write the appplications trace output.");
            Console.WriteLine();
        }

        private static bool ConfirmUserAction()
        {
            Console.WriteLine("Do you wish to copy all permissions form '{0}' to '{1}'? [y/n]",
                m_diFromRoot.FullName,
                m_diToRoot.FullName);

            if (Console.ReadKey().KeyChar.ToString().ToLower() != "y")
            {
                Console.WriteLine("User aborted!");
                OutputPressKeyToExit();
                System.Environment.Exit(-1);
                return false;
            }

            return true;
        }

        #endregion

        #region Prechecks

        private static bool PopulateAndCheckDirectoriesExist(string[] args)
        {
            string _dirFrom = args[0];
            string _dirTo = args[1];
            string _logFile = args[2];

            if (_dirFrom.EndsWith("\\")) _dirFrom = _dirFrom.Substring(0, _dirFrom.Length - 1);
            if (_dirTo.EndsWith("\\")) _dirTo = _dirTo.Substring(0, _dirTo.Length - 1);

            m_diFromRoot = new DirectoryInfo(_dirFrom);
            m_diToRoot = new DirectoryInfo(_dirTo);
            m_fiLog = new FileInfo(_logFile);

            if (!m_fiLog.Exists)
            {
                try
                {
                    m_fiLog.Create().Close();
                }
                catch (Exception _ex)
                {
                    Console.WriteLine("Could not create log file: " + _ex.ToString());
                }

                m_fiLog.Refresh();
            }

            if (!m_diFromRoot.Exists)
                Console.WriteLine("Directory '{0}' does not exist", m_diFromRoot.FullName);

            if (!m_diToRoot.Exists)
                Console.WriteLine("Directory '{0}' does not exist", m_diToRoot.FullName);

            if (!m_fiLog.Exists)
                Console.WriteLine("File '{0}' does not exist", m_fiLog.FullName);

            return m_diFromRoot.Exists && m_diToRoot.Exists && m_fiLog.Exists;
        }

        #endregion
    }
}

Ok, SkyDrive is still a baby, and personally I’ve used other services providing file space in the cloud and enjoyed them a little better. But this post isn’t about if SkyDrive is good or bad, it’s just about a missing feature that is very painful. Someone wanted to share some fotos, uploaded them to SkyDrive and all I wanted was to download them all to my PC. Tough look, you can click on each and every image to get to the preview page, where you click on the preview picture to then finally get at the actual picture. Multiply that by about 100. I have better things to do than waste my time on that.

So a Dev does what he does best, fires up Visual Studio 2008 and hacks away (did I just say I had something better to do - well I lied partially, but before I go off to do that, there is always time for some good ol’ C#).

I’ve posted it here not as a finished utility (there are no binaries) but as a small sample. Using WebClients, RegEx and some other stuff it downloads the list page of the SkyDrive folder, fetches the preview page and then downloads the actual image to a folder on the hard disk. Not really rocket science and of course there are a few quirks (no real error handling for example), but it’s just a sample. Feel free to extend as you wish, don’t blame me if it starts downloading Gigabytes of files overnight, because you accidentally crawled a HoneyPot. (And yes, it only downloads jpgs at the moment. I didn’t need any other types.)

May those SkyDrive bytes be with you…

/**********************************************************************************
 *
 * Example Application for crawling web pages and downloading images.
 *
 * This code works if you pass in a SkyDrive Folder Url (http://.... /browse.aspx/...)
 * and will download any jpg images it finds in there.
 *
 * Permission to use, copy, modify, distribute and sell this software and its
 * documentation for any purpose is hereby granted without fee.
 * I make no representations about the suitability of this software for any purpose.
 * It is provided "as is" without express or implied warranty.
 *
 * Alex Duggleby - 24.05.08 - V0.9 - http://alexduggleby.com
 *
 **********************************************************************************/
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Text.RegularExpressions;
using System.IO;
using System.Web;
using System.ComponentModel;

namespace Tools.SkyDrive.DownloadAll
{
    class Program
    {
        // Used for tracking how many items we have left
        private static int _wcInnerCount = 0;
        private static int _wcInnerCompleted = 0;

        // We have to start somewhere
        private static Uri _uriStart;

        // Work we have already done
        private readonly static List<string> _urisCrawled = new List<string>();
        private readonly static List<string> _imagesDownloaded = new List<string>();

        // Download images to?
        private readonly static DirectoryInfo _diDownloadTo = new DirectoryInfo(Path.Combine(Path.Combine(System.Environment.GetFolderPath(Environment.SpecialFolder.Personal), "Downloads"),"Images"));

        // This finds urls in the page
        private readonly static Regex _regexUrl = new Regex("href\\s*=\\s*(?:(?:\\\"(?<url>[^\\\"]*)\\\")|(?<url>[^\\s]* ))");

        // This finds the open url in the image page
        private readonly static Regex _regexUrlOpen = new Regex("href\\s*=\\s*(?:(?:\\\"(?<url>[^\\\"]*)\\\")|(?<url>[^\\s]*)) title=\\\"Open\\\""); 

        /// <summary>
        /// Takes the url to a skydrive folder page and downloads all jpg images.
        /// </summary>
        static void Main(string[] args)
        {
            // Usage check
            if (args.Length != 1)
            {
                Console.WriteLine("Usage: App.exe http://theUrlToThe/SkyDrive/FolderPage");
                return;
            }

            try
            {
                // First parameter is url
                _uriStart = new Uri(args[0]);
            }
            catch (Exception _ex)
            {
                Console.WriteLine("Invalid Url. " + _ex.Message);
                return;
            }

            // Make sure download directory exists
            if (!_diDownloadTo.Exists) _diDownloadTo.Create();

            using (WebClient _wc = new WebClient())
            {
                // This is the index with all the images
                string _pageContents = _wc.DownloadString(_uriStart);

                // Each image has a preview page, so we get the url to that, before we get the url to the actual image
                foreach (Match _matchUrlToImagePage
                    in _regexUrl.Matches(_pageContents))
                {
                    Uri _uriToImagePage =
                        new Uri(_uriStart, HttpUtility.HtmlDecode(_matchUrlToImagePage.Groups["url"].Value));

                    CrawlPreviewPage(_uriToImagePage);
                }
            }

            // Wait for the async web clients to complete...
            while (_wcInnerCompleted < _wcInnerCount)
            {
                Console.WriteLine("Wait for images to complete...");
                Console.ReadLine();
            }

            Console.WriteLine("Should be finished!");
            Console.ReadLine();
        }

        /// <summary>
        /// Parses the preview page and finds the actual image link
        /// </summary>
        /// <param name="uriToImagePage">The url to the preview page</param>
        /// <returns></returns>
        private static void CrawlPreviewPage(Uri uriToImagePage)
        {
            using (WebClient _wc = new WebClient())
            {
                if (!_urisCrawled.Contains(uriToImagePage.ToString()))
                {
                    _urisCrawled.Add(uriToImagePage.ToString());

                    if (uriToImagePage.ToString().ToLower().EndsWith(".jpg"))
                    {
                        string _pageContents = _wc.DownloadString(uriToImagePage);

                        // Find the image we want to download... There should be
                        // only one link with title="Open" in it.
                        foreach (Match _matchImage in _regexUrlOpen.Matches(_pageContents))
                        {
                            Uri _uriToImage = new Uri(_matchImage.Groups["url"].Value);

                            DownloadImage(_uriToImage);
                        }
                    }
                }
            }
        }

        /// <summary>
        /// Downloads async'ly an image from a Uri
        /// </summary>
        /// <param name="uriToImage">The uri to download</param>
        private static void DownloadImage(Uri uriToImage)
        {
            // Output the url
            Console.WriteLine("{0}{1}", uriToImage.ToString(), Environment.NewLine);

            if (!_imagesDownloaded.Contains(uriToImage.ToString()))
            {
                _imagesDownloaded.Add(uriToImage.ToString());
                string _lowerUrl = uriToImage.ToString().ToLower();

                // Simple checking
                if (_lowerUrl.EndsWith(".jpg") &&
                   (!_lowerUrl.Contains("browse")) &&
                   (!_lowerUrl.Contains("self")))
                {
                    // HtmlDecode here because some urls have encoded characters
                    string _localFilename = HttpUtility.HtmlDecode(
                        uriToImage.Segments[uriToImage.Segments.Length - 1]);

                    // Create a valid local filename
                    Path.GetInvalidPathChars().ToList().ForEach(
                        c => _localFilename = _localFilename.Replace(c, '_'));

                    Console.Write("Downloading {0}...{1}", _localFilename, Environment.NewLine);

                    // Create a seperate web client for each image (uses async, and you can't
                    // issue two downloads at the same time for the same client). Of course
                    // here we should be using some kind of pooling but this is the quickest
                    // way to do it.
                    using (WebClient _wcInner = new WebClient())
                    {
                        _wcInnerCount++;
                        _wcInner.DownloadFileAsync(uriToImage, Path.Combine(_diDownloadTo.ToString(), _localFilename));
                        _wcInner.DownloadFileCompleted += new AsyncCompletedEventHandler(_wcInner_DownloadFileCompleted);
                    }
                }
            }
        }

        // Is fired when a download complete. We output status and check if we are finished!
        private static void _wcInner_DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
        {
            // Increase the completed counter
            _wcInnerCompleted++;

            // Ok, we could do some more extensive checking, this could trigger
            // even if there are still items to download... but hey, it's just a
            // quick utility!
            if (_wcInnerCompleted == _wcInnerCount)
            {
                Console.WriteLine("{0}{1}{2}", Environment.NewLine, "Finished all files!", Environment.NewLine);
                Console.ReadLine();
            }
            else
            {
                Console.WriteLine("File {0} of {1} completed!", _wcInnerCompleted, _wcInnerCount);
            }
        }
    }
}

Update: You can download the sql file here: http://in.solit.us/archives/download/149837 (saves you cleaning up the copy-pasted code from below… oh and hello experts-exchange-users)

Have you ever been asked to write something where you knew from the start that it could possibly wreak havok? Well I did today and since I haven’t posted in a while I thought I’d quickly share. Don’t worry I’ll be back blogging Sync, MVC and more soon (and hopefully tell you why I was gone).

Without talking too much about the real scenario that this was needed for, let me just say there is a large database that stores configuration data including a set of connection strings in multiple places spread across numerous tables. And our admins moved the database but didn’t know where exactly to change these config strings. I was the only dev left in the office at the time and they turned to me asking me if I could write a script that “replaced all occurrences of stringA with stringB throughout the whole database”.

Yikes. It sounds like trouble, you know it’s gonna hurt and I don’t want to be anywhere nearby when they press F5 to run the script. Ok, nevertheless I put together a little script that did just that, sent it to them and let’s see what happens next week. They have backups so even if everything goes wrong we can quickly restore things.

Here’s what I came up with. It’s not rocket science, but does the job. Use the script at your own risk (I didn’t find anything comparable on the web so maybe it will come in handy to someone). It shows some interesting usage of the system tables that I use now and again for things like this (when I don’t have my beloved SQLSMO). Have fun and use with caution (I can’t say that enough!)

------------------------------------------------------------
-- Name: STRING REPLACER
-- Author: ADUGGLEBY
-- Version: 20.05.2008 (1.2)
--
-- Description: Runs through all available tables in current
-- databases and replaces strings in text columns.
------------------------------------------------------------

-- PREPARE
SET NOCOUNT ON

-- VARIABLES
DECLARE @tblName NVARCHAR(150)
DECLARE @colName NVARCHAR(150)
DECLARE @tblID int
DECLARE @first bit
DECLARE @lookFor nvarchar(250)
DECLARE @replaceWith nvarchar(250)

-- CHANGE PARAMETERS
SET @lookFor = 'virtual2'
SET @replaceWith = 'virtual3'

-- TEXT VALUE DATA TYPES
DECLARE @supportedTypes TABLE ( xtype NVARCHAR(20) )
INSERT INTO @supportedTypes SELECT XTYPE FROM SYSTYPES WHERE NAME IN ('varchar','char','nvarchar','nchar','xml')

-- ALL USER TABLES
DECLARE cur_tables CURSOR FOR
SELECT SO.name, SO.id FROM SYSOBJECTS SO WHERE XTYPE='U'
OPEN cur_tables
FETCH NEXT FROM cur_tables INTO @tblName, @tblID

WHILE @@FETCH_STATUS = 0
BEGIN
	-------------------------------------------------------------------------------------------
	-- START INNER LOOP - All text columns, generate statement
	-------------------------------------------------------------------------------------------
	DECLARE @temp NVARCHAR(4000)
	DECLARE @count INT
	SELECT @count = COUNT(name) FROM SYSCOLUMNS WHERE ID = @tblID AND
		XTYPE IN (SELECT xtype FROM @supportedTypes)

	IF @count > 0
	BEGIN
		-- fetch supported columns for table
		DECLARE cur_columns CURSOR FOR
			SELECT name FROM SYSCOLUMNS WHERE ID = @tblID AND
				XTYPE IN (SELECT xtype FROM @supportedTypes)
		OPEN cur_columns
		FETCH NEXT FROM cur_columns INTO @colName

		-- generate opening UPDATE cmd
		SET @temp = '
	PRINT ''Replacing ' + @tblName + '''

	UPDATE ' + @tblName + ' SET
		'
		SET @first = 1

		-- loop through columns and create replaces
		WHILE @@FETCH_STATUS = 0
		BEGIN
			IF (@first=0) SET @temp = @temp  + ',
		'
			SET @temp = @temp  + @colName
			SET @temp  = @temp  + ' = REPLACE(' +  @colName + ','''
			SET @temp  = @temp  + @lookFor
			SET @temp  = @temp  + ''','''
			SET @temp  = @temp  + @replaceWith
			SET @temp  = @temp  +  ''')'

			SET @first = 0
			FETCH NEXT FROM cur_columns INTO @colName
		END

		PRINT @temp

		CLOSE cur_columns
		DEALLOCATE cur_columns
	END
	-------------------------------------------------------------------------------------------
	-- END INNER
	-------------------------------------------------------------------------------------------

	FETCH NEXT FROM cur_tables INTO @tblName, @tblID
END

CLOSE cur_tables
DEALLOCATE cur_tables

I’m quite busy at the moment with the Academic Community Launch, so I’ll leave the talking to Christian, who has posted some information about why I’ve seen most of Austria’s big cities within the last week.

Info and Pictures of the ACL 2008

This morning I finally got my authors copy of the dot.net Magazin that I wrote an article on Microsoft Sync Framework. It’s in german, 7 pages long and you can buy it tomorrow.

Anyhow, I got an email this morning asking about one of the code listings and asking if there was anything different to do in VB.net. It’s been a while since I last did VB.net (I actually did more VB6 than VB.net) but I converted the sample from the magazine to VB.net. Just remember to reference Microsoft.Synchronization and Microsoft.Synchronization.Files.

(By the way I still haven’t got round to looking at CTP2 of the Sync Framework. All this stuff (and the article) applies to CTP1. Maybe there was a breaking change in CTP2 with regard to the FileSyncProvider. I must get round to looking at CTP2 a.s.a.p.)

Here is the code:

Imports Microsoft.Synchronization
Imports Microsoft.Synchronization.Files
Imports System.IO

Module Module1

    Sub Main()

        'Define directories
        Dim _sourceDir As String = "c:\temp\1"
        Dim _destDir As String = "c:\temp\2"

        'Define system ids
        Dim _sourceId As SyncId = New SyncId(New Guid("A4715EAF-341E-4ebf-90C3-71E8644CA6E8"))
        Dim _destId As SyncId = New SyncId(New Guid("FF766014-0DA4-401d-BDA1-0679326ECF96"))

        'Exclude some patterns
        Dim _syncFilter As FileSyncScopeFilter = New FileSyncScopeFilter()
        _syncFilter.AttributeExcludeMask = FileAttributes.Hidden Or FileAttributes.System
        _syncFilter.FileNameExcludes.Add("DoNotCopy.txt")
        _syncFilter.SubdirectoryExcludes.Add("DoNotCopy")

        'Setup options (makes sure files aren't deleted for ever)
        Dim _syncOptions As FileSyncOptions = FileSyncOptions.RecycleDeletes Or FileSyncOptions.RecycleOverwrites

        'Do the sync
        Using _sourceProvider As FileSyncProvider = New FileSyncProvider(_sourceId, _sourceDir, _syncFilter, _syncOptions)
            Using _destinationProvider As FileSyncProvider = New FileSyncProvider(_destId, _destDir, _syncFilter, _syncOptions)

                Dim _syncAgent As SyncAgent = New SyncAgent()
                _syncAgent.LocalProvider = _sourceProvider
                _syncAgent.RemoteProvider = _destinationProvider
                _syncAgent.Direction = SyncDirection.UploadAndDownload
                _syncAgent.Synchronize()

            End Using
        End Using

    End Sub

End Module