Friday, 15 March 2013

EntityFramework Reverse POCO Code First Generator v1.4.1 released

Reverse engineers an existing database and generates EntityFramework Code First POCO classes, Configuration mappings and DbContext.

v1.4.1 includes these new features:

  • If a field has a default contrainst of GetDate(), add DateTime.Now() in the ctor for the field.
  • Add support for the datetimeoffset type, including support for the default constraint sysdatetimeoffset().
  • Add Resharper naming comments.
  • Mapping classes are marked internal.
  • Make sure Foreign keys map only to Primary keys.
To see a video, head over to http://visualstudiogallery.msdn.microsoft.com/ee4fcff9-0c4c-4179-afd9-7a2fb90f5838

Monday, 10 December 2012

“Entity Framework Reverse POCO” OSS project made top link today (Dec 7)

I saw a spike on the Entity Framework open-source project I created, and had a look around to see what it could be…
 
Found it listed as the top link on http://www.alvinashcraft.com
As developers we take so much (understatement) from the open-source community ourselves, it’s nice to give back from time to time.

I've also noticed that it is now listed in the "most popular" category in the VisualStudio add new item.

Thursday, 22 November 2012

EntityFramework Reverse POCO Code First Generator

Reverse engineers an existing database and generates EntityFramework Code First POCO classes, Configuration mappings and DbContext.

To install and use this project:

  • Use Nuget and install EntityFramework.
  • Add a connect string to your app.config. Somethine like:
  • 
       
    
  • In Visual Studio, right click project and select "add - new item".
  • Select Online, and search for "reverse poco". Or you can download it from this page.
  • Select "EntityFramework Reverse POCO Code First Generator".
  • Give the file a name, such as Database.tt and click Add.
  • Edit the Database.tt file and specify the connection string as MyDbContext which matches your name in app.config.
  • Save the Database.tt file, which will now generate the Database.cs file.

To see a video, head over to http://visualstudiogallery.msdn.microsoft.com/ee4fcff9-0c4c-4179-afd9-7a2fb90f5838

Friday, 26 October 2012

Obtaining the database schema, tables, columns, and primary keys in a single SQL call

I am currently writing an EntityFramework reverse engineer code-first generator. It will generate POCO classes, DbContext and Code First mapping for an existing database.

There is one already in Entity Framework Power Tools Beta 2 However this one is going to do the job right, and include table filtering!

Watch this space...
SELECT  [Extent1].[SchemaName],
        [Extent1].[Name] AS TableName,
        [Extent1].[TABLE_TYPE] AS TableType,
        [UnionAll1].[Ordinal],
        [UnionAll1].[Name] AS ColumnName,
        [UnionAll1].[IsNullable],
        [UnionAll1].[TypeName],
        ISNULL([UnionAll1].[MaxLength],0) AS MaxLength,
        ISNULL([UnionAll1].[Precision], 0) AS Precision,
        ISNULL([UnionAll1].[Default], '') AS [Default],
        ISNULL([UnionAll1].[DateTimePrecision], '') AS [DateTimePrecision],
        ISNULL([UnionAll1].[Scale], 0) AS Scale,
        [UnionAll1].[IsIdentity],
        [UnionAll1].[IsStoreGenerated],
        CASE WHEN ([Project5].[C2] IS NULL) THEN CAST(0 AS BIT)
             ELSE [Project5].[C2]
        END AS PrimaryKey
FROM    (
         SELECT QUOTENAME(TABLE_SCHEMA) + QUOTENAME(TABLE_NAME) [Id],
                TABLE_SCHEMA [SchemaName],
                TABLE_NAME [Name],
                TABLE_TYPE
         FROM   INFORMATION_SCHEMA.TABLES
         WHERE  TABLE_TYPE IN ('BASE TABLE', 'VIEW')
        ) AS [Extent1]
        INNER JOIN (
                    SELECT  [Extent2].[Id] AS [Id],
                            [Extent2].[Name] AS [Name],
                            [Extent2].[Ordinal] AS [Ordinal],
                            [Extent2].[IsNullable] AS [IsNullable],
                            [Extent2].[TypeName] AS [TypeName],
                            [Extent2].[MaxLength] AS [MaxLength],
                            [Extent2].[Precision] AS [Precision],
                            [Extent2].[Default],
                            [Extent2].[DateTimePrecision] AS [DateTimePrecision],
                            [Extent2].[Scale] AS [Scale],
                            [Extent2].[IsIdentity] AS [IsIdentity],
                            [Extent2].[IsStoreGenerated] AS [IsStoreGenerated],
                            0 AS [C1],
                            [Extent2].[ParentId] AS [ParentId]
                    FROM    (
                             SELECT QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) + QUOTENAME(c.COLUMN_NAME) [Id],
                                    QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) [ParentId],
                                    c.COLUMN_NAME [Name],
                                    c.ORDINAL_POSITION [Ordinal],
                                    CAST(CASE c.IS_NULLABLE
                                           WHEN 'YES' THEN 1
                                           WHEN 'NO' THEN 0
                                           ELSE 0
                                         END AS BIT) [IsNullable],
                                    CASE WHEN c.DATA_TYPE IN ('varchar', 'nvarchar', 'varbinary')
                                              AND c.CHARACTER_MAXIMUM_LENGTH = -1 THEN c.DATA_TYPE + '(max)'
                                         ELSE c.DATA_TYPE
                                    END AS [TypeName],
                                    c.CHARACTER_MAXIMUM_LENGTH [MaxLength],
                                    CAST(c.NUMERIC_PRECISION AS INTEGER) [Precision],
                                    CAST(c.DATETIME_PRECISION AS INTEGER) [DateTimePrecision],
                                    CAST(c.NUMERIC_SCALE AS INTEGER) [Scale],
                                    c.COLLATION_CATALOG [CollationCatalog],
                                    c.COLLATION_SCHEMA [CollationSchema],
                                    c.COLLATION_NAME [CollationName],
                                    c.CHARACTER_SET_CATALOG [CharacterSetCatalog],
                                    c.CHARACTER_SET_SCHEMA [CharacterSetSchema],
                                    c.CHARACTER_SET_NAME [CharacterSetName],
                                    CAST(0 AS BIT) AS [IsMultiSet],
                                    CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)), c.COLUMN_NAME, 'IsIdentity') AS BIT) AS [IsIdentity],
                                    CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)), c.COLUMN_NAME, 'IsComputed')
                                    | CASE WHEN c.DATA_TYPE = 'timestamp' THEN 1
                                           ELSE 0
                                      END AS BIT) AS [IsStoreGenerated],
                                    c.COLUMN_DEFAULT AS [Default]
                             FROM   INFORMATION_SCHEMA.COLUMNS c
                                    INNER JOIN INFORMATION_SCHEMA.TABLES t
                                        ON c.TABLE_CATALOG = t.TABLE_CATALOG
                                           AND c.TABLE_SCHEMA = t.TABLE_SCHEMA
                                           AND c.TABLE_NAME = t.TABLE_NAME
                                           AND t.TABLE_TYPE IN ('BASE TABLE', 'VIEW')
                            ) AS [Extent2]
                    UNION ALL
                    SELECT  [Extent3].[Id] AS [Id],
                            [Extent3].[Name] AS [Name],
                            [Extent3].[Ordinal] AS [Ordinal],
                            [Extent3].[IsNullable] AS [IsNullable],
                            [Extent3].[TypeName] AS [TypeName],
                            [Extent3].[MaxLength] AS [MaxLength],
                            [Extent3].[Precision] AS [Precision],
                            [Extent3].[Default],
                            [Extent3].[DateTimePrecision] AS [DateTimePrecision],
                            [Extent3].[Scale] AS [Scale],
                            [Extent3].[IsIdentity] AS [IsIdentity],
                            [Extent3].[IsStoreGenerated] AS [IsStoreGenerated],
                            6 AS [C1],
                            [Extent3].[ParentId] AS [ParentId]
                    FROM    (
                             SELECT QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) + QUOTENAME(c.COLUMN_NAME) [Id],
                                    QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) [ParentId],
                                    c.COLUMN_NAME [Name],
                                    c.ORDINAL_POSITION [Ordinal],
                                    CAST(CASE c.IS_NULLABLE
                                           WHEN 'YES' THEN 1
                                           WHEN 'NO' THEN 0
                                           ELSE 0
                                         END AS BIT) [IsNullable],
                                    CASE WHEN c.DATA_TYPE IN ('varchar', 'nvarchar', 'varbinary')
                                              AND c.CHARACTER_MAXIMUM_LENGTH = -1 THEN c.DATA_TYPE + '(max)'
                                         ELSE c.DATA_TYPE
                                    END AS [TypeName],
                                    c.CHARACTER_MAXIMUM_LENGTH [MaxLength],
                                    CAST(c.NUMERIC_PRECISION AS INTEGER) [Precision],
                                    CAST(c.DATETIME_PRECISION AS INTEGER) AS [DateTimePrecision],
                                    CAST(c.NUMERIC_SCALE AS INTEGER) [Scale],
                                    c.COLLATION_CATALOG [CollationCatalog],
                                    c.COLLATION_SCHEMA [CollationSchema],
                                    c.COLLATION_NAME [CollationName],
                                    c.CHARACTER_SET_CATALOG [CharacterSetCatalog],
                                    c.CHARACTER_SET_SCHEMA [CharacterSetSchema],
                                    c.CHARACTER_SET_NAME [CharacterSetName],
                                    CAST(0 AS BIT) AS [IsMultiSet],
                                    CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)), c.COLUMN_NAME, 'IsIdentity') AS BIT) AS [IsIdentity],
                                    CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)), c.COLUMN_NAME, 'IsComputed')
                                    | CASE WHEN c.DATA_TYPE = 'timestamp' THEN 1
                                           ELSE 0
                                      END AS BIT) AS [IsStoreGenerated],
                                    c.COLUMN_DEFAULT [Default]
                             FROM   INFORMATION_SCHEMA.COLUMNS c
                                    INNER JOIN INFORMATION_SCHEMA.VIEWS v
                                        ON c.TABLE_CATALOG = v.TABLE_CATALOG
                                           AND c.TABLE_SCHEMA = v.TABLE_SCHEMA
                                           AND c.TABLE_NAME = v.TABLE_NAME
                             WHERE  NOT (
                                         v.TABLE_SCHEMA = 'dbo'
                                         AND v.TABLE_NAME IN ('syssegments', 'sysconstraints')
                                         AND SUBSTRING(CAST(SERVERPROPERTY('productversion') AS VARCHAR(20)), 1, 1) = 8
                                        )
                            ) AS [Extent3]
                   ) AS [UnionAll1]
            ON (0 = [UnionAll1].[C1])
               AND ([Extent1].[Id] = [UnionAll1].[ParentId])
        LEFT OUTER JOIN (
                         SELECT [UnionAll2].[Id] AS [C1],
                                CAST(1 AS BIT) AS [C2]
                         FROM   (
                                 SELECT QUOTENAME(tc.CONSTRAINT_SCHEMA) + QUOTENAME(tc.CONSTRAINT_NAME) [Id],
                                        QUOTENAME(tc.TABLE_SCHEMA) + QUOTENAME(tc.TABLE_NAME) [ParentId],
                                        tc.CONSTRAINT_NAME [Name],
                                        tc.CONSTRAINT_TYPE [ConstraintType],
                                        CAST(CASE tc.IS_DEFERRABLE
                                               WHEN 'NO' THEN 0
                                               ELSE 1
                                             END AS BIT) [IsDeferrable],
                                        CAST(CASE tc.INITIALLY_DEFERRED
                                               WHEN 'NO' THEN 0
                                               ELSE 1
                                             END AS BIT) [IsInitiallyDeferred]
                                 FROM   INFORMATION_SCHEMA.TABLE_CONSTRAINTS tc
                                 WHERE  tc.TABLE_NAME IS NOT NULL
                                ) AS [Extent4]
                                INNER JOIN (
                                            SELECT  7 AS [C1],
                                                    [Extent5].[ConstraintId] AS [ConstraintId],
                                                    [Extent6].[Id] AS [Id]
                                            FROM    (
                                                     SELECT QUOTENAME(CONSTRAINT_SCHEMA) + QUOTENAME(CONSTRAINT_NAME) [ConstraintId],
                                                            QUOTENAME(TABLE_SCHEMA) + QUOTENAME(TABLE_NAME) + QUOTENAME(COLUMN_NAME) [ColumnId]
                                                     FROM   INFORMATION_SCHEMA.KEY_COLUMN_USAGE
                                                    ) AS [Extent5]
                                                    INNER JOIN (
                                                                SELECT  QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) + QUOTENAME(c.COLUMN_NAME) [Id],
                                                                        QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) [ParentId],
                                                                        c.COLUMN_NAME [Name],
                                                                        c.ORDINAL_POSITION [Ordinal],
                                                                        CAST(CASE c.IS_NULLABLE
                                                                               WHEN 'YES' THEN 1
                                                                               WHEN 'NO' THEN 0
                                                                               ELSE 0
                                                                             END AS BIT) [IsNullable],
                                                                        CASE WHEN c.DATA_TYPE IN ('varchar', 'nvarchar', 'varbinary')
                                                                                  AND c.CHARACTER_MAXIMUM_LENGTH = -1 THEN c.DATA_TYPE + '(max)'
                                                                             ELSE c.DATA_TYPE
                                                                        END AS [TypeName],
                                                                        c.CHARACTER_MAXIMUM_LENGTH [MaxLength],
                                                                        CAST(c.NUMERIC_PRECISION AS INTEGER) [Precision],
                                                                        CAST(c.DATETIME_PRECISION AS INTEGER) [DateTimePrecision],
                                                                        CAST(c.NUMERIC_SCALE AS INTEGER) [Scale],
                                                                        c.COLLATION_CATALOG [CollationCatalog],
                                                                        c.COLLATION_SCHEMA [CollationSchema],
                                                                        c.COLLATION_NAME [CollationName],
                                                                        c.CHARACTER_SET_CATALOG [CharacterSetCatalog],
                                                                        c.CHARACTER_SET_SCHEMA [CharacterSetSchema],
                                                                        c.CHARACTER_SET_NAME [CharacterSetName],
                                                                        CAST(0 AS BIT) AS [IsMultiSet],
                                                                        CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)),
                                                                                            c.COLUMN_NAME, 'IsIdentity') AS BIT) AS [IsIdentity],
                                                                        CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)),
                                                                                            c.COLUMN_NAME, 'IsComputed')
                                                                        | CASE WHEN c.DATA_TYPE = 'timestamp' THEN 1
                                                                               ELSE 0
                                                                          END AS BIT) AS [IsStoreGenerated],
                                                                        c.COLUMN_DEFAULT AS [Default]
                                                                FROM    INFORMATION_SCHEMA.COLUMNS c
                                                                        INNER JOIN INFORMATION_SCHEMA.TABLES t
                                                                            ON c.TABLE_CATALOG = t.TABLE_CATALOG
                                                                               AND c.TABLE_SCHEMA = t.TABLE_SCHEMA
                                                                               AND c.TABLE_NAME = t.TABLE_NAME
                                                                               AND t.TABLE_TYPE IN ('BASE TABLE', 'VIEW')
                                                               ) AS [Extent6]
                                                        ON [Extent6].[Id] = [Extent5].[ColumnId]
                                            UNION ALL
                                            SELECT  11 AS [C1],
                                                    [Extent7].[ConstraintId] AS [ConstraintId],
                                                    [Extent8].[Id] AS [Id]
                                            FROM    (
                                                     SELECT CAST( NULL AS NVARCHAR (1)) [ConstraintId], CAST( NULL AS NVARCHAR (MAX)) [ColumnId] WHERE 1= 2
                                                    ) AS [Extent7]
                                                    INNER JOIN (
                                                                SELECT  QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) + QUOTENAME(c.COLUMN_NAME) [Id],
                                                                        QUOTENAME(c.TABLE_SCHEMA) + QUOTENAME(c.TABLE_NAME) [ParentId],
                                                                        c.COLUMN_NAME [Name],
                                                                        c.ORDINAL_POSITION [Ordinal],
                                                                        CAST(CASE c.IS_NULLABLE
                                                                               WHEN 'YES' THEN 1
                                                                               WHEN 'NO' THEN 0
                                                                               ELSE 0
                                                                             END AS BIT) [IsNullable],
                                                                        CASE WHEN c.DATA_TYPE IN ('varchar', 'nvarchar', 'varbinary')
                                                                                  AND c.CHARACTER_MAXIMUM_LENGTH = -1 THEN c.DATA_TYPE + '(max)'
                                                                             ELSE c.DATA_TYPE
                                                                        END AS [TypeName],
                                                                        c.CHARACTER_MAXIMUM_LENGTH [MaxLength],
                                                                        CAST(c.NUMERIC_PRECISION AS INTEGER) [Precision],
                                                                        CAST(c.DATETIME_PRECISION AS INTEGER) AS [DateTimePrecision],
                                                                        CAST(c.NUMERIC_SCALE AS INTEGER) [Scale],
                                                                        c.COLLATION_CATALOG [CollationCatalog],
                                                                        c.COLLATION_SCHEMA [CollationSchema],
                                                                        c.COLLATION_NAME [CollationName],
                                                                        c.CHARACTER_SET_CATALOG [CharacterSetCatalog],
                                                                        c.CHARACTER_SET_SCHEMA [CharacterSetSchema],
                                                                        c.CHARACTER_SET_NAME [CharacterSetName],
                                                                        CAST(0 AS BIT) AS [IsMultiSet],
                                                                        CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)),
                                                                                            c.COLUMN_NAME, 'IsIdentity') AS BIT) AS [IsIdentity],
                                                                        CAST(COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.TABLE_SCHEMA) + '.' + QUOTENAME(c.TABLE_NAME)),
                                                                                            c.COLUMN_NAME, 'IsComputed')
                                                                        | CASE WHEN c.DATA_TYPE = 'timestamp' THEN 1
                                                                               ELSE 0
                                                                          END AS BIT) AS [IsStoreGenerated],
                                                                        c.COLUMN_DEFAULT [Default]
                                                                FROM    INFORMATION_SCHEMA.COLUMNS c
                                                                        INNER JOIN INFORMATION_SCHEMA.VIEWS v
                                                                            ON c.TABLE_CATALOG = v.TABLE_CATALOG
                                                                               AND c.TABLE_SCHEMA = v.TABLE_SCHEMA
                                                                               AND c.TABLE_NAME = v.TABLE_NAME
                                                                WHERE   NOT (
                                                                             v.TABLE_SCHEMA = 'dbo'
                                                                             AND v.TABLE_NAME IN ('syssegments', 'sysconstraints')
                                                                             AND SUBSTRING(CAST(SERVERPROPERTY('productversion') AS VARCHAR(20)), 1, 1) = 8
                                                                            )
                                                               ) AS [Extent8]
                                                        ON [Extent8].[Id] = [Extent7].[ColumnId]
                                           ) AS [UnionAll2]
                                    ON (7 = [UnionAll2].[C1])
                                       AND ([Extent4].[Id] = [UnionAll2].[ConstraintId])
                         WHERE  [Extent4].[ConstraintType] = N'PRIMARY KEY'
                        ) AS [Project5]
            ON [UnionAll1].[Id] = [Project5].[C1]
WHERE   NOT ([Extent1].[Name] IN ('EdmMetadata', '__MigrationHistory'))
Do not add the following as it slows down the query by 97%. Do the sorting within your code instead.
ORDER BY [Extent1].[CatalogName],
         [Extent1].[SchemaName],
         [Extent1].[Name],
         [UnionAll1].[Ordinal]
Foreign Keys for all tables in database:
SELECT  FK_Table = FK.TABLE_NAME,
        FK_Column = CU.COLUMN_NAME,
        PK_Table = PK.TABLE_NAME,
        PK_Column = PT.COLUMN_NAME,
        Constraint_Name = C.CONSTRAINT_NAME,
        FK.TABLE_SCHEMA AS fkSchema,
        PK.TABLE_SCHEMA AS pkSchema
FROM    INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS C
        INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS FK
            ON C.CONSTRAINT_NAME = FK.CONSTRAINT_NAME
        INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS PK
            ON C.UNIQUE_CONSTRAINT_NAME = PK.CONSTRAINT_NAME
        INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE CU
            ON C.CONSTRAINT_NAME = CU.CONSTRAINT_NAME
        INNER JOIN (
                    SELECT  i1.TABLE_NAME,
                            i2.COLUMN_NAME
                    FROM    INFORMATION_SCHEMA.TABLE_CONSTRAINTS i1
                            INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE i2
                                ON i1.CONSTRAINT_NAME = i2.CONSTRAINT_NAME
                    WHERE   i1.CONSTRAINT_TYPE = 'PRIMARY KEY'
                   ) PT
            ON PT.TABLE_NAME = PK.TABLE_NAME
ORDER BY FK.TABLE_NAME, CU.COLUMN_NAME

Tuesday, 23 October 2012

FizzBuzz generators

SQL

WITH mil AS (
 SELECT TOP 1000000 ROW_NUMBER() OVER ( ORDER BY c.column_id ) [n]
 FROM master.sys.all_columns as c
 CROSS JOIN master.sys.all_columns as c2
)                
 SELECT CASE WHEN n  % 3 = 0 THEN
             CASE WHEN n  % 5 = 0 THEN 'FizzBuzz' ELSE 'Fizz' END
        WHEN n % 5 = 0 THEN 'Buzz'
        ELSE CAST(n AS char(6))
     END + CHAR(13)
 FROM mil

C#

foreach (int number in Enumerable.Range(1, 100))
{
    bool isDivisibleBy3 = (number % 3) == 0;
    bool isDivisibleBy5 = (number % 5) == 0;

    if (isDivisibleBy3)
         Console.Write("Fizz");

    if (isDivisibleBy5)
         Console.Write("Buzz");

    if (!isDivisibleBy3 && !isDivisibleBy5)
         Console.Write(number);

    Console.WriteLine();
}

C# linq

 Enumerable
  .Range(1, 100)
  .Select(i =>
    i % 15 == 0 ? "FizzBuzz" :
    i % 5 == 0 ? "Buzz" :
    i % 3 == 0 ? "Fizz" :
    i.ToString())
  .ToList()
  .ForEach(s => Console.WriteLine(s));

SQL Lotter number selector

Just a bit of fun with SQL

WITH
    L0 AS (SELECT 0 AS C UNION ALL SELECT 0),
    L1 AS (SELECT 0 AS C FROM L0 AS A CROSS JOIN L0 AS B),
    L2 AS (SELECT 0 AS C FROM L1 AS A CROSS JOIN L1 AS B),
    L3 AS (SELECT 0 AS C FROM L2 AS A CROSS JOIN L2 AS B),
    Nums AS (SELECT TOP(49) ROW_NUMBER()
       OVER(ORDER BY (SELECT 0)) AS n FROM L3 ORDER BY n),
    Choice AS (SELECT TOP(6) n FROM Nums ORDER BY CHECKSUM(NEWID()))
SELECT STUFF( (SELECT ',' + CAST(n AS VARCHAR(10)) AS [text()]
FROM Choice ORDER BY n
FOR XML PATH('')), 1, 1, '')

Wednesday, 3 October 2012

Olympus OM-D vs Nikon D7000 & back focus issue

I've had my Nikon D7000 for 9 months now, and put down the image softness to a poor lens as I mainly use a Sigma 28-300mm lens. I recently purchased a 50mm f1.8 prime lens, and although better, its not tack sharp as you would expect with a good prime lens. I just ignored it, and carried on with my photography.
That was until I went to a Damian McGillycuddy http://www.damianmcgillicuddy.com/ shoot, where he was demonstrating the Olympus OM-D camera. The olympus has the same number of pixels as my D7000, 16 million. However the Olympus has a micro 4/3 sensor, meaning the laws of physics are on the D7000 side as that has a larger sensor. Therefore the Nikon should capture better pictures. Or so you'd think. I shot both the Nikon and Olympus carefully, in a studio, with studio lighting, in RAW, and focused carefully on the right eye. I was stunned at the difference in picture quality. The Olympus beat the Nikon hands down.

Here is proof. I took the following pictures, both in RAW, no post corrections at all. Both cameras had their white balance set to flash. Two rear flashes either side about 6 feet away, and a large octobox above and infront of her head. I am basically underneath the octobox. There is also a white reflector just under neath the model out of shot, to bounce any light up.

Olympus OM-D

Nikon D7000


A closeup of the eye, both taken from the RAW images:
Olympus OM-D

Nikon D7000


So what do you think looks better? Thought so, Olympus hand down. It even has better colour balance.

Nikon back focus issue


To test if I had a back / front focus issue with the Nikon, I downloaded a test chart and printed it off. There are loads about, just Google for one.
  1. Stuck the test chart on the wall in the conservatory, during a bright sunny day.
  2. Camera on tripod about 1.5m away with a 50mm f1.8 prime lens.
  3. Camera set to RAW, speed = 1/500, aperture = f1.8
  4. Set the camera to use a single focus sensor in the middle.
  5. Put the camera into live view mode and took a reference shot.
  6. Put the camera into manual focus, I twist the focus ring on the lens, and put the camera back into automatic focus mode.
  7. Take a normal shot.
  8. Repeat last 2 steps three times.
Here are the results:
Reference shot

Shot 1

Shot 2

Shot 3


The shots 1-3 should have the same sharpness as the reference shot, but it doesn't This is definately a front / back focus issue. It isn't that bad however and the model shots should have been a lot better on the Nikon, including colour. The Olympus OM-D is a fantastic bit of kit. It's light too. Remind me again why I bought a Nikon?

In Summary


  1. The Olympus OM-D is a fantastic bit of kit. Really! Go buy one.
  2. My Nikon D7000 has been sent back to Nikon under warranty.

Friday, 17 August 2012

Effortless .Net Encryption

I've just released a new open source project called Effortless .Net Encryption. It can be found here: https://effortlessencryption.codeplex.com

Effortless .Net Encryption is a library that is written in C# 4.0, contains 68 unit tests and 190 Pex unit tests, and provides:
  • Rijndael encryption/decryption.
  • Hashing and Digest creation/validation.
  • Password and salt creation.

Available on Nuget

https://nuget.org/packages/Effortless.Net.Encryption/
To install Effortless.Net.Encryption, run the following command in the Package Manager Console
PM> Install-Package Effortless.Net.Encryption

Wednesday, 25 April 2012

A generic singleton using Lazy< T >

Carring on from the generic singleton post here.

Since we have .NET 4, we can now make use of Lazy< T >

public class Singleton< T > where T : class, new()
{
    private Singleton() {}

    private static readonly Lazy< T > instance = new Lazy< T >(() => new T());

    public static T Instance { get { return instance.Value; } } 
}

Wednesday, 28 March 2012

Please turn off hyperlinks in PowerPoint

When giving presentations, you should really turn off PowerPoints AutoCorrect feature of it changing a hyperlink into an underlined link.

Unless you actually are going to click on the link during the presentation, then turn it off. It's much easer to read without it being in a different font and underlined.

To turn it off in PowerPoint, goto Tools -> Options -> Proofing -> AutoCorrect options. Untick the box highlighted below.

Friday, 23 March 2012

Recursive CTE (Common Table Expression)

There is sometimes a problem of wanting to remove data (email addresses in this example) from within a string which are delimited.

For example, if you want to remove all non bybox email addresses from "some.name@bybox.com; simon@hicrest.net; fred.bloggs@bybox.com"
and do this for every table, without having to create functions to break apart the string first, how are you going to do it?

Here's how:

Our example table looks like this
CREATE TABLE data_export
(
 data_export_id INT NOT NULL IDENTITY(1, 1) PRIMARY KEY,
 email_address VARCHAR(255) NOT NULL
 -- ... Other fields left out for brevity
)

Insert some test data
INSERT INTO data_export (email_address)
VALUES  ('some.name@bybox.com; simon@hicrest.net; fred.bloggs@bybox.com'),
        ('neo@matrix.com; me@bybox.com'),
        ('fred@b.com; xxx@bybox.com'),
        ('fred@bbc.com'),
        ('an.other@bybox.com')

Next is the recursive CTE SQL is in several sections:
  • split_by_delimeter - Breaking apart the string by delimeters.
  • just_bybox - Keep the @bybox emaill addresses.
  • distrinct_set - The ID's we want to update.
  • STUFF - The rebuild section is the final select/update statement which containst the STUFF keyword.

;WITH   split_by_delimeter
          AS (
              SELECT    data_export_id,
                        email_address,
                        CHARINDEX(';', email_address + ';') AS n,
                        CAST('' AS VARCHAR(255)) AS result
              FROM      data_export
              UNION ALL
              SELECT    data_export_id,
                        SUBSTRING(email_address,CHARINDEX(';',email_address)+1, 255),
                        CHARINDEX(';', email_address),
                        LTRIM(RTRIM(SUBSTRING(email_address, 0,
                            CASE WHEN CHARINDEX(';', email_address) = 0
                            THEN 255
                            ELSE CHARINDEX(';', email_address)
                            END)))
              FROM      split_by_delimeter
              WHERE     n > 0
             ),
        just_bybox
          AS (
              SELECT    data_export_id,
                        result
              FROM      split_by_delimeter
              WHERE     result <> ''
                        AND result LIKE '%@bybox%'
             ),
        distinct_set
          AS (
              SELECT DISTINCT data_export_id
              FROM just_bybox
             )
    -- For checking
    SELECT  data_export_id,
            STUFF((
                   SELECT ';' + result FROM just_bybox y
                   WHERE y.data_export_id= x.data_export_id
                  FOR XML PATH('')
                  ), 1, 1, '') AS email_address
    FROM    distinct_set x
    /*
    -- For actual update
    UPDATE  data_export
    SET     email_address = STUFF((
                                   SELECT ';' + result FROM just_bybox y
                                   WHERE y.data_export_id= x.data_export_id
                                   FOR XML PATH('')
                                  ), 1, 1, '')
    FROM    distinct_set x
            JOIN data_export
                ON data_export.data_export_id = x.data_export_id
 */

This returns the following results:
data_export_idemail_address
1some.name@bybox.com;fred.bloggs@bybox.com
2me@bybox.com
3xxx@bybox.com
5an.other@bybox.com

Perfect!

Tuesday, 20 March 2012

C# 5 attributes on optional parameters

With C# 5, you can put a special attribute on an optional parameter and the compiler will fill in the value not with a constant but with information about the calling method. This means we can implement the Logger.Trace to automagically pick up where it’s being called from:
public static void Trace(string message,
                         [CallerFilePath] string sourceFile = "",
                         [CallerMemberName] string memberName = "")
{
    string msg = String.Format("{0}: {1}.{2}: {3}",
       DateTime.Now.ToString("yyyy-mm-dd HH:MM:ss"),
       Path.GetFileNameWithoutExtension(sourceFile), memberName, message);

    LoggingInfrastructure.Log(msg);
}

Now, if the caller calls Log.Trace("some message") the compiler will fill in the missing arguments not with the empty string, but with the file and member where the call happens:

// In file called Fred.cs
public void SomeFunc()
{
  Log.Trace("Hello");
  // Compiles to Log.Trace("Hello", "Fred.cs", "SomeFunc")
}

Another example of how you can use this is in implementing INotifyPropertyChanged without needing either literal strings, expression magic or mystic weavers:
public class ViewModelBase : INotifyPropertyChanged 
{
  protected void Set(ref T field,
                        T value,
                        [CallerMemberName] string propertyName = "") 
  {
    if (!Object.Equals(field, value)) 
    {
      field = value;
      OnPropertyChanged(propertyName);
    }
  }
  // usual INPC boilerplate
}
 
public class Widget : ViewModelBase 
{
  private int _thingy;
  public int Thingy
  {
    get { return _thingy; }
    set { Set(ref _thingy, value); } // Compiler fills in "Thingy" as propertyName
  }
}
I grabbed this info from http://www.mindscapehq.com/blog/index.php/2012/03/18/what-else-is-new-in-c-5/. More can be found there, but I thought this feature important/cool enough to blog about it on its own right. And mainly so I don't forget about it...

Friday, 2 March 2012

www.stilettos-sos.com

I've created yet another WordPress CMS system for a friend of my wifes this time.
Home page is a blog, the others are static pages.
www.stilettos-sos.com

Thursday, 1 March 2012

The real doomsday date is Tue Jan 19 2038 at 03:14:07

Some said it was the Y2K bug we had to worry about, but it's actually Tue Jan 19 2038 03:14:07 you've really to worry about.

There are many, many systems built with time saved as a 32bit long integer value.
The max value of a signed long integer is 2147483647. This value is the number of seconds elapsed since midnight (00:00:00), January 1, 1970, coordinated universal time (UTC)

Here is an example C program, built with Visual Studio 2010.

__time32_t t;

t = 0;
printf( "The min is %s\n", _ctime32( &t ) ); 

t = 2147483647;
printf( "The max is %s\n", _ctime32( &t) ); 

t = 2147483648;
printf( "The date is %s\n", _ctime32( &t) ); 

Output:
The min is Thu Jan 01 00:00:00 1970
The max is Tue Jan 19 03:14:07 2038
The date is (null)

So after Tue Jan 19 03:14:07 2038, the date goes to null which will cause a crash. Let's just hope all the critical (including embedded) systems get fixed before then.

Wednesday, 29 February 2012

PowerShell script to automatically clone/update all repos in Kiln

PowerShell Script files

Download the files here.

Change and replace the following url

Change https://your fogbugz root url to be your root url for fogbugz. For example, if your companies fogbugz url is something like https://secure.bbc.com/FogBugz/default.asp, then your root url will be https://secure.bbc.com

Obtaining a Kiln Token

The first thing you need to do before you can use this script is to obtain a Kiln token. Copy and alter the following link in your browsers URL https://your fogbugz root url/FogBugz/api.asp?cmd=logon&email=[yourEmail]&password=[yourpassword]
Changing the text in square brackets (and also removing the brackets).
You should receive something like this:
ymjt123f8882a6s7td0j8eefa6u2g8
Take the token text: ymjt123f8882a6s7td0j8eefa6u2g8 and paste it into the powerShell script (edit your local copy instead of the SVN one)
token placement

Running the script

Start PowerShell and run the script passing in the root path of your Kiln repos.
example

Will it clone everything?

It could, including branches, but by default I've set it not to clone QA/PROD/UAT repos, or branches.

How to clone branches

Remove this to clone branches

How to clone QA/PROD/UAT repos

Modify the function CanCloneRepo() which is used to filter out repos you may not want.
If you want all repos, simply return 1; from this function.
Cloning QA/UAT/PROD

This script will also remove repository folders from your drive where the repo no longer exists in Kiln!

This prevents orphaned code on your drive. However, if you don't want this functionality, simply remove the following code situated at right the bottom of the script:
remove folders
By leaving the above code in, your repository folder structures will be standardised to how its organised in Kiln. I've tested it and it works. Have faith ;-)

Thanks to

Robin Minto for providing the template of the powershell script.

Monday, 27 February 2012

Ecommerce site

I forgot to blog about an ecommerce site I created for an artist:
Shop

I used Zen Cart and make my own custom template to match the main home site: www.lookonthebrightside.co.uk

Website


E-Commerce site

Friday, 27 January 2012

www.lizigns.com

Just finished a new website for a friend called lizigns.com


I played with three CMS systems: Joomla, concrete5 and finally settled on WordPress CMS system.

I'm pleased with the result. It looks fresh and has all the meta tags Liz wanted. Plus is quick and snappy to use.

My views on the CMS systems I tried:
  • Jooma has good customisation of templates, and a very good admin side of things, but its actually quite slow in use.
  • Concrete5 is excellent in everyway, apart from one major stumble. You can't modify any of the template colours in the admin panel. I couldn't find the right template I wanted, so I ditched it.
  • WordPress. Good admin panel, so I was confident Liz could use it to edit her pages after I handed it over. And it has lots and lots of free template and plugins (like a gallery) to choose from.

Wednesday, 20 July 2011

Microsoft's 'Roslyn' compiler as a service project

Microsoft plans to share more information on its "Roslyn" compiler-as-a-service (CaaS) project during its annual Microsoft Research Faculty Summit this week. 

Roslyn, at a high level, is Microsoft's vehicle for "taking .Net to the cloud." Microsoft's stated goal with Roslyn is to "build a compiler architecture that is amenable to use as a foundation for modern tools." 

http://www.zdnet.com/blog/microsoft/… 

You can do all sorts of fancy tricks with the compiler pipeline, like automatic parallelization for example. 

I forsee compiler add-in companies sprouting up all over the place now. Development teams will have to agree on what add-ins to use, possibly even the ordering of the add-ins, so that the exe's match. Build-servers will have to include this technology, unless you can config the compiler+addins outside of FinalBuilder. 
Overall I see this is as a good thing, a bit like Firefox+add-ins. But I fear there will be only a few real gems, and a ton of dross out there

Tuesday, 7 June 2011

How to stop Skype crashing upon startup

Open a command prompt and run the following:

c:
cd\users
del shared.xml /s

Now you can run skype without it crashing.

Tuesday, 17 May 2011

Setting the PowerShell execution policy - the trap

I was recently pulling my hair out trying to run a powerShell script in VisualStudio. I kept getting the error: "File SomeScript.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details."

So I ran the script:
powershell set-executionpolicy remotesigned

Tried a gain, and still it fails! Only until I spoke to my developer collegue Damian Powell did he say you actually have to run that command for both x86 and x64 powershells.

So I ran:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell set-executionpolicy remotesigned
C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell set-executionpolicy remotesigned


Hey presto! All works. Thanks Damian!

Monday, 14 March 2011

Service Broker sys.transmission_queue clean up

If you’re developing a Service Broker application and you misspell your services, forget to create a master key, or do any of the many things that result in your messages not being delivered, then your transmission queue will fill up.
Due to the nature of service broker, errors don't occur at the client but rather your messages go in to Service Brokers sys.transmission_queue and are then subsequently processed. If they fail, they stay in the queue. Some errors can be resolved, i.e. missing services. However, others can't, or are just plain difficult. I.e. You've misspelt something, or not used the correct security.
To clear your transmission queue you need to end the conversations, this is a handy little script for doing this. Be aware this ends ALL conversations in the queue and so should NEVER be used on a production system.

To see how many messages are in your transmission queue, use this
SELECT COUNT(1) FROM sys.transmission_queue
 
On my system i7, DB on Raid10, Log on Raid1, the following clears messages at approximately 20000/second.

Tuesday, 18 January 2011

How to dump your Skype database to a text file for easy searching

I had a problem recently trying to search the Skype instant messeage history. Searching for a historical chat can be done in Skype by pressing CTRL-F, but only if you know the person(s) that were involved on that chat. You cannot search the chat history globally.

The solution is to dump out the skype database into a TXT file for easy searching with notepad.

  1. Download SQLite command shell at http://www.sqlite.org/download.html. (I downloaded http://www.sqlite.org/sqlite-shell-win32-x86-3070400.zip)
  2. Extract sqllite3.exe to a place on your path, or somewhere you can run it easily from.
  3. Open a DOS command prompt.
  4. Change the folder to C:\Users\[pc user name]\AppData\Roaming\Skype\[skype user name]
  5. There you should find main.db database.
  6. Type the following (in bold):
    • sqlite3 main.db
    • sqlite> .output c:\skype_chat.txt
    • sqlite> .dump
    • sqlite> .quit

You can now start Notepad and open file c:\skype_chat.txt to search through.

Wednesday, 5 January 2011

How to create a Gource graphical repository movie


Download Gource from https://code.google.com/p/gource/ extract to C:\gource-0.28.win32
Download FFMPEG from http://www.videohelp.com/tools/ffmpeg extract to c:\ffmpeg

If using SVN also download the following:
  1. svn-gource.py from https://code.google.com/p/gource/wiki/SVN extract to c:\python31
  2. Python from http://www.python.org/download/ I used v3.1.3. extract to c:\python31

In a DOS command prompt type:
  1. c:
  2. cd C:\Data Files\Projects (Use your own SVN repository location)
  3. svn log -r {2011-01-01}:{2010-01-01} --verbose --xml > c:\code-trunk.log
  4. cd C:\gource-0.28.win32
  5. The follow is only required if using SVN: c:\python31\python svn-gource.py --filter-dirs c:\code-trunk.log > c:\code-trunk-gource.log
  6. gource --log-format custom c:\code-trunk-gource.log -1600x1040 --date-format "%d %B %y" --seconds-per-day 0.5 -a 0.1 --highlight-all-users --stop-at-end --disable-progress --output-ppm-stream c:\code-trunk.ppm --camera-mode overview --bloom-intensity 0.1 --output-framerate 25 --max-files 2000 --hide filenames,mouse -i 20
  7. cd c:\ffmpeg\bin
  8. ffmpeg -y -b 9000K -r 25 -f image2pipe -vcodec ppm -i c:\code-trunk.ppm -vcodec mpeg1video -s 1600x1040 c:\code-trunk.mpg
  9. ffmpeg -vcodec copy -acodec copy -i C:\some-cool-music.mp3 -i c:\code-trunk.mpg c:\code-trunk-audio.mpg
The last step is optional, it adds an audio music to your gource video.
In step 3 above, if you remove the -r {2011-01-01}:{2010-01-01} it will export your whole repository. I just wanted last years.

Enjoy.

Monday, 20 December 2010

SQL Server MAXDOP and parallelism

At our company, we have a 4 core blade running SQL Server 2008 Enterprise Edition.
This is a very, very busy SQL Server and is running on HP EVA hardware, so the I/O is not an issue.
We have over 50 warehouses all running SQL Server 2008 and replicating data to/from this primary database.
This database is also used by our website, webservices and import/export applications. The only thing this database does not serve is our SSRS reports, which are hosted on another blade.

What is an issue, however, are the CPU's. They were very busy. To help with this we can play around with two settings on SQL Server:
  • MAXDOP (Max degree of parallelism)
  • Cost threshold for parallelism



Our DBA and I had a few discussions around this recently. Read plenty of articles on it, such as Adam Machanic

On a normal system, leaving these settings alone are fine, but on a very very busy system you need to change them.

  1. MAXDOP=1. What this does is to always use 1 core for the query.
  2. MAXDOP=0, Cost = 20. This will use 1 core for simple queries, and for complex queries (cost > 20)  will be allowed to use parallel queries.
What we found is setting MAXDOP = 1 is optimal.

This is because running complex queries faster made the replication suffer. By using MAXDOP=1 we save CPU time by not having to join up parallel queries at the end. Save time by stopping the "thread thrashing" that can occur on many queries all running in parallel.

Because the SQL box is very busy fielding many queries at once, using MAXDOP=1still utilises all cores due to running 4 or more queries at the same time.
Also replication is a lot happier now.

Monday, 11 October 2010