Mark Cohen is a CIO at Australia's largest online retailer and is a hands-on, sleeves-rolled-up, code-cutting geek. He lives in Sydney, Australia with his wife and boys and can sometimes be spotted puffing and panting as he runs at Maroubra Beach

Archive for September, 2006

Lookups powered by Enums

Output of Titles EnumerationI want to be able to maintain lookup tables in databases purely for reporting and similar applications.  I like enums in code instead of retrieving lookup data as they’re integer comparisons, they eliminate the database hit, and they are far more legible.  Here’s how to get the data you need to populate lists, dropdowns etc based on enums.

I declared a sample Titles enum just to test with:

    public enum Titles
    {
        Mr = 1,
        Mrs = 2,
        Dr = 3,
        Hon = 4,
        Miss = 5,
        Master = 6
    }

I then get the type of the enum, and then do a foreach loop to get all the values available.  Enum.GetValues retrieves all the values in the enum.  The behaviour of ToString and ToInt mean that it is possible to populate combos straight off of enums. 

     class Program
    {
       static void Main(string[] args)
        {
            Enumerate();
        }�br />         private static void Enumerate()
        {
            Type title = typeof(Titles);
            foreach (Titles item in Enum.GetValues(title))
            {
                Console.WriteLine(String.Format (”Title number {0} is \”{1}\”",
                    Convert.ToInt16(item), item.ToString()));
            }
            Console.ReadKey();
        }
    }

 

Benchmarking DLINQ vs Direct Stored Procedure

I did some preliminary benchmarking to assess the cost of using LINQ vs using direct SQL today.  I was not convinced that the benchmarking was accurate as I was running against a dev SQL server that other developers were also using, and the SQL box is a virtual server on the same hardware that hosts other virtual servers and other systems.  Too much white noise causing interference.

So tonight I sat down and started doing some isolated, consistent benchmarking.  I’m running SQL Express 2005 and LINQ May CTP.  The hardware is a standalone Core Duo running XP.  Did I mention that it’s a Macbook Pro? ;)   1 GB RAM, some other apps running but nothing resource hungry.

Things were running like this:

I used a dev database with a few thousand rows of text data to test. The table in question (you’ll see the table name in the LINQ code below): ArticleModuleAttribute has 4060 Rows. I did not inspect or modify any indexing or anything else at the table level as the table in question will be used for all tests.

first-up I wanted to emulate the sort of basic behavior we have in a piece of code that does a search, retrieves a page of results, and shows it on screen.  So I created a simple LINQ query as follows:

private static void LinqTest()
{
  DataContext db = new DataContext(”Data Source=macbookpro\\sqlexpress;Initial 
               Catalog=CMDotNet;Integrated Security=True”);
  Table<ArticleModuleAttribute> ArticleModuleAttributes = db.GetTable<ArticleModuleAttribute>();
  var blocks =
            (from ama in ArticleModuleAttributes
             where ama.strAttributeName == “Content” && ama.strAttributeValue.Contains(”unselectable”)
             select ama).Skip(10).Take(10);

  foreach (var block in blocks)
  {
    Console.WriteLine(block.numAttributeID.ToString());
  }
blocks = null;
}

 This code is just a simple single-table search, skipping the first page, returning the second page of ten results.  The console.writeline is just to slow things down a bit, and this will be applied to LINQ and direct tests.

Next up I wrote the simplest equivalent of this LINQ query.  I went into LinqTest in debug and retrieved the SQL.  I made it into a stored procedure as follows:

ALTER procedure [dbo].[testSearchProc]
    @p0 as nvarchar(20),
    @p1 as nvarchar(20),
    @p2 as nvarchar(20),
    @p3 as nvarchar(20)

as

SELECT TOP 10 [t0].[numAttributeID], [t0].[numModuleID], [t0].[strAttributeName], [t0].[strAttributeValue]
FROM [ArticleModuleAttribute] AS [t0]
WHERE (NOT (EXISTS(
SELECT NULL AS [EMPTY]
FROM (
SELECT TOP 20 [t1].[numAttributeID]
FROM [ArticleModuleAttribute] AS [t1]
WHERE ([t1].[strAttributeName] = @p0) AND ([t1].[strAttributeValue] LIKE @p1)
) AS [t2]
WHERE [t0].[numAttributeID] = [t2].[numAttributeID]
))) AND ([t0].[strAttributeName] = @p2) AND ([t0].[strAttributeValue] LIKE @p3)

This is literally a cut-and-paste from LINQ with the only modification being the parameter declarations being changed to stored procedure parameters. Nothing in the select is modified (which in itself blew me away as it is now THAT easy to do server-side paging, no more excuses for sloppy searches :) )

I wrote a simple method to call this stored procedure  as follows:

private static void DirectTest(int i)
{
  SqlConnection conn = new SqlConnection(”Data Source=macbookpro\\sqlexpress;Initial 
     Catalog=CMDotNet;Integrated Security=True”);
  conn.Open();
  SqlCommand cmd = new SqlCommand();
  cmd.Connection = conn;
  cmd.CommandText = “testSearchProc”
  cmd.CommandType = CommandType.StoredProcedure;
  cmd.Parameters.Add(new SqlParameter(”p0″, “Content”));
  cmd.Parameters.Add(new SqlParameter(”p1″, “%unselectable%”));
  cmd.Parameters.Add(new SqlParameter(”p2″, “Content”));
  cmd.Parameters.Add(new SqlParameter(”p3″, “%unselectable%”));

  SqlDataReader reader = cmd.ExecuteReader(CommandBehavior.CloseConnection);
  while (reader.Read())
  {
    Console.WriteLine(reader.GetSqlDecimal(0).ToString());
  }
  reader.Close();
  reader = null;
}

No Rocket Science there either.  the parameter values are exactly as I extracted from LINQ in debug

I then ran these two tests, one hundred times each.  The first runs were always slower as (I assume) connections were opened and then pooling sped it up.  grabbing times between every loop I got the following results:

This was grabbing the ten records, writing a value to the console, looping 100 times.  LINQ appears to be slower than a direct call on a stored procedure.  Applying a ten point moving average to get something a bit easier to look at gives:

Showing DLINQ sitting at almost double the execution time.  to eliminate the possibility that pooling was somehow skewing results, I added pooling=false to the connection strings for both tests, swapped the execution order and reran.  A similar result emerged.  To eliminate the console.writeline from skewing things, I changed the code to set and reset a single decimal variable.  This also eliminated the .ToString (although it was consistent in both tests.

Based on this trivial example I would suggest detailed analysis including to-scale load testing before rolling LINQ out as a replacement for traditional SQL in situations where high-scalability is required. 

Further investigation: Dynamic SQL vs LINQ

Headhunter Day today

The headhunters must’ve all just finished a sales and marketing seminar or something.  I have been phoned by four agents looking to place people today, two in the last five minutes. Pity they don’t read my blog :P

Happy Birthday Google

Google turned 8 yesterday - more or less, seems the exact date is a bit fuzzy.  Happy Birthday Big G :) Talk about eight years that changed the world….

Free Culture

Copyright and free culture came up in a discussion at work today. We were talking about Lew Wasserman. Wasserman purchased Universal Studios and merged it with MCA, making it what it is today. As an agent he represented Ronald Reagan among many others. He made Hollywood run on the studio culture that it still runs on today. He was also instrumental in getting Jack Valenti to head the Motion Picture Association of America up until 2004. During this time Valenti is regarded as having been one of the most effective and influential pro-copyright lobbyists, having a close personal relationships with people such as the once President of the USA, Lyndon Johnson. Johnson took over when Kennedy was asassinated and Valenti is visible in the background of pictures from his swearing in aboard Air Force One (See Valenti on Wikipedia).

Long interview transcript with Valenti on darknet here.

Professor Lessig released his book Free Culture under a creative commons license and has a cool site at http://www.free-culture.cc

The book is reviewed here and is available from Amazon along with all the usual reader feedback.

The quality of the audio book in’t stellar but it’s good enough. I recommend reading the book (or listening to it) to get a good insight into the current copyright frenzy on the web, from a viewpoint other than that propagated by the mainstream media. The connections between the media mogul, the copyright lobbyist, two presidents, and other key industry players is fascinating.

Lessig also touches on the history of hollywood and how the film industry originally moved out West to escape the stringent policing of copyright laws in the East coast cities in the USA.

The book raises more questions than it answers, and once we as a society – globally, not just the US and EU – work out the answers, they will define the shape of the media industry in the years to come.

Indexing Services “Snap-in failed to initialize”

I’m a sucker for beta software.  I know I’m a twit, but I like all the shiny and new bits. Especially things like Office 2007.

Today I have been trying to implement a search based on Indexing Services.  It’s been refusing to play nicely with SQL, and I get the following error messgae when I try connect to the snap-in from Computer Management:

—————————
Microsoft Management Console
—————————
Snap-in failed to initialize.
    Name:Indexing Service
    CLSID:{95AD72F0-44CE-11D0-AE29-00AA004B9986}
—————————

After scratching around a bit I found that uninstalling Windows Search (that Outlook 2007 insists on having) would fix it.  I did, and Indexing Services got better :)  

Un-re-moving on

When I first came to live in Oz I took a job at a software consultancy.  I spent three years there and then moved on to a major player in the media space at a time when I was worried for the future of the company.  After just under two years at the media house I decided to leave, and took the same role with the consultancy again, who were then positioning themselves primarily as providers of content and knowledge management solutions.  After a year back here, I came to the harsh realisation that my days in this business are over.  This epiphany coincided with my previous employer (the major media player, keep up ;) ) needing a technology manager to work on one of the biggest websites in the country.  So after way too much deliberation and back-and-forth discussions I’m now making the move.  Back.  As one of my mates put it, my career is like watching a tennis match :S

Happy New Year :) שנה טובה לכל העולמ

Happy new year and well over the fast to all my Jewish mates and readers, wishing you all a year full of health, happiness and success.  I hope this year brings nothing but good things.

Mark

Skype founder goes .net for his next venture

Om Malik has a story on GigaOm about a new site backed by Morten Lund, one of the Skype founders.  This time he is focusing his attention and penchant for disruptive technologies on stock trading.  His venture, Zecco will be launching soon and aims to recoup the $2 per trade primarily through advertising, as I understand it.  This man is turning into the Walmart of the Internet :)

I also get a kick out of the fact that he sees enough business value in using .net.  Somebody needs to show these guys urlrewriter.net in a rush though – with ad driven revenue and with the volume of content they will have at their disposal, I would suggest some good seo would come in very handy.

Max Automator :)

One of these little guys is the Apple Automator dude, the other is the Microsoft Max dude. Automator has been around since OS/X.

Imitation is the sincerest form of flattery ;)

Next Page »