# Thursday, September 06, 2007

PSExpect - PowerShell Unit Testing Library

What next - COBOLUnit?  Haven't tried this yet, but this could be a good tool for 'self-diagnostic' tests on Servers...

http://www.codeplex.com/psexpect/Wiki/View.aspx?title=User%20Guide&referringTitle=Home

posted on Thursday, September 06, 2007 7:53:06 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Monday, September 03, 2007

Parse and TryParse

Like many people I'd mistakenly thought that xxx.Parse was the only way to validate types like ints and DateTimes.  This kindof 'was' the case in .NET 1.1, but numeric data could be validated with Double.TryParse then converted to the correct type.

In .NET 2.0 you can happily use the TryParse method on pretty much all the types you'd want to validate - without the nasty overhead of catching an exception.  This post goes into a bit more detail, and links to a tool to benchmark the results for yourself.

I'll be on the lookout to convert whereever I find int.Parse, as it's pretty rare you 'actually' want an exception to be thrown...

posted on Monday, September 03, 2007 9:58:53 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Tuesday, August 14, 2007

Agile Processes at work

This excellent post from Jake Lawlor's a great practical guide to applying agile processes in the real world. 

This isn't new thinking by any means, but when trying to coach others and improve your own processes you find surprisingly few practical reference points for applying agile processes into actual organisations.

posted on Tuesday, August 14, 2007 8:03:57 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Monday, August 13, 2007

CRM Mapping Custom Attributes

Just found a useful post that helped in resolving why Custom Attributes weren't being copied from Leads to Contacts when 'Converting' a lead.  It's all in the mappings...

http://blogs.inetium.com/blogs/rtoenies/archive/2006/07/31/272.aspx

posted on Monday, August 13, 2007 2:54:30 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Wednesday, August 08, 2007

Windows Powershell - Batch files with knobs on

Just downloading a few .NET 3.0 bits and pieces from MS and happened across some 'you might like to download this' links.  Once of which was Windows PowerShell.  Haven't tried it yet, but will certainly give it a go in my next 'scripting' escapade.

Microsoft have obviously developed this quite a lot based on the amount of documentation I've not had chance to look at yet!

posted on Wednesday, August 08, 2007 10:37:05 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]

ReSharper - Getting more out of it

ReSharper's got plenty of functionality (more of which I find and use as time goes on), but I found this on Joe White's blog that is certainly a useful reference and taught me a couple of new things too..

posted on Wednesday, August 08, 2007 8:02:25 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Tuesday, August 07, 2007

Breaking Changes between .NET 1.1 and 2.0

I spotted an old post from Brad Abrams on breaking changes for .NET 1.1 apps recompiled with 2.0.  The links had moved but are still relevant...

http://msdn2.microsoft.com/en-au/netframework/Aa570326.aspx

My own observations from converting projects are a little higher level in that I'm also interested in what effort you're liable for when converting, and also things to watch out for if (like me) you're having to persist a 'shared' code base of 1.1 and 2.0 core libraries (temporarily) while you migrate all your client apps.

ASP.NET Projects.  This includes Web Service projects.  You'll find that you'll probably have to reconstruct these projects based on the content files and you'll also need to make a decision on the model you're going to use for debugging - i.e. whether to use the new debugging host (removing IIS from the debugging equation) and whether to use a dynamic or static port.  Steer clear of the 'Web Site' project too as this could leave you confused for hours wondering why you can't debug anything - look for the ASP.NET projects.  More info on the ASP.NET side can be found from Peter Laudati's post.  Microsoft also have an article covering conversion of ASP.NET projects

DataSets.  These have changed in that a 'designer' file is now the main source file (apart from the .xsd).  If you're sharing DataSets across 1.1 and 2.0 projects just share the .xsd file otherwise you'll be asking for trouble as the other generated files are incompatible.  Any structural changes to a DataSet in one project (say .NET 1.1) will not be reflected in the corresponding 2.0 project.  You'll need to check out the XSD in both places (assuming you're using VSS) and build both in order to avoid breaking one of the projects.

Windows Forms.  I've had a recent experience where the InitializeComponent() method in a form (edited with VS 2005) places calls that aren't compatible with .NET 1.1.  The code compiles (as below) but 'exception thrown by target of an invocation' gets thrown on the following line...

((System.ComponentModel.ISupportInitialize)(this.validatingProgress)).BeginInit();

This is because the control (a PictureBox in this case) doesn't implement ISupportInitialize in .NET 1.1 but does in 2.0.  The equivalent in 1.1 is SuspendLayout()

This would just be one example of something that's syntactically correct, but could fail at runtime.

posted on Tuesday, August 07, 2007 4:38:59 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Tuesday, July 31, 2007

Efficient XPath Queries

This is something I get asked about quite a bit as I had a misspent youth with XSLT...

One of my pet hates is people always using the search identifier '//' in XPath queries.  It's the SELECT * of XML, and shouldn't be used unless you actually want to 'search' your document.

If you're performing SQL you'd SELECT fields explicitly rather than SELECT * wouldn't you? :-)

because:

  1. If the schema changes (new fields inserted) then your existing code has less chance of breaking
  2. It performs better less server pre-processing and catalog lookups
  3. More declarative and the code is easy to read and maintain

With XML (and the standard DOM-style parsers) you're working on a document tree, and accessing nodes loaded into that tree. 

Consider the following XML fragment as an example:

Say your car dealership sells new cars and current prices are serialised in an xml document:

<xml>
 <Sales>
  <Cars>
   <Car Make="Ford" Model="Territory" />
   <Car Make="Ford" Model="Focus" />
  </Cars>
 </Sales>
</xml>

In order to get all cars you can easily use the following XPath: '//Car'.  This searches from the root to find all Car elements (and finds 2).

A more efficient way would be '/*/*/Car' as we know Cars only exist at the 3rd level in the document

A yet more efficient way would be '/Sales/Cars/Car' as this makes a direct hit on the depth and parent elements in the document.

You can also mix and match with '/*//Car' to directly access the parts of the DOM you're certain of and search on the parts you're not.

Now lets say you go into the used car business and refactor your XML format as follows:

<xml>
 <Sales Type="New">
  <Cars>
   <Car Make="Ford" Model="Territory" />
   <Car Make="Ford" Model="Focus" />
  </Cars>
 </Sales>
 <Sales Type="Used">
  <Cars>
   <Car Make="Honda" Model="Civic" />
   <Car Make="Mazda" Model="3" />
  </Cars>
 </Sales>
</xml>

If you want to get all Cars (new and used) you could still use any of the XPaths above.  If you want to isolate the New from the used, then you're going to have to make some changes in any case.

'//Car' is obviously going to pick up 4 elements

'/Sales[@Type='New']/Cars/Car' is probably the most efficient in this case but it will vary based on the complexity of the condition (in []) and the complexity and size of the document.

It's important to note that the effects of optimising your XPath queries won't really be felt until you're operating with:

  • Large documents (n mb+)
  • Deep documents (n levels deep) - n is variable based on the document size 
  • Heavy load and high demand for throughput of requests

This means don't expect effecient XPaths to solve all your problems, but they shouldn't be a limiting factor in a business application.  The other thing to say is that if your XPath queries are getting really complicated then your schema is probably in need of some attention as well.

posted on Tuesday, July 31, 2007 12:12:25 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Wednesday, July 25, 2007

SQL Server - Save all DTS Packages

Here's another useful proc I found to back up all your DTS packages (from I can't remember where).  Put this in your master database: 

Create procedure usp_SavePackages
@Path    
varchar(128
)
as
/*

*/

    
set nocount on

declare 
@objPackage 
int
declare 
@PackageName varchar(128
)
declare @rc 
int
declare 
@ServerName varchar(128
)
declare @FileName varchar(128
)
declare    @FilePath varchar(128
)
declare    @cmd varchar(2000
)
    
    
select     @ServerName 
@@ServerName ,
        @FilePath 
@Path
    
    
if right(@Path,1) <> 
'\'
    begin
        select @Path = @Path + '
\
'
    end
    
    -- create output directory - will fail if already exists but ...
    select    @cmd = '
mkdir 
' + @FilePath
    exec master..xp_cmdshell @cmd
    
    
create table #packages (PackageName varchar(128))
    insert     #packages
        (PackageName)
    select     distinct name
    from    msdb..sysdtspackages
    
    select    @PackageName = ''
    while @PackageName < (select max(PackageName) from #packages)
    begin
        select    @PackageName = min(PackageName) from #packages where PackageName > @PackageName

        select    @FileName = @FilePath + @PackageName + '
.dts
'

        exec @rc = sp_OACreate '
DTS.Package
', @objPackage output
        if @rc <> 0
        begin
            raiserror('
failed to create package rc %d
', 16, -1, @rc)
            return
        end

        exec @rc = sp_OAMethod @objPackage, '
LoadFromSQLServer
' , null,
            @ServerName = @ServerName, @Flags = 256, @PackageName = @PackageName
        if @rc <> 0
        begin
            raiserror('
failed to load package rc %d, package %s
', 16, -1, @rc, @PackageName)
            return
        end
        
        -- delete old file
        select @cmd = '
del 
' + @FileName
        exec master..xp_cmdshell @cmd, no_output
        
        exec @rc = sp_OAMethod @objPackage, '
SaveToStorageFile
', null, @FileName
        if @rc <> 0
        begin
            raiserror('
failed to save package rc %d, package %s
', 16, -1, @rc, @PackageName)
            return
        end
        
        exec @rc = sp_OADestroy @objPackage
    end

GO

Colorized by: CarlosAg.CodeColorizer

posted on Wednesday, July 25, 2007 9:05:39 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Monday, July 23, 2007

Microsoft CRM - Data Migration Framework (DMF) Bugs / Fixes

I've been migrating data for a client recently and part of this has required the use of the MS Data Migration Framework (DMF).  I thought I'd post a couple of bugs I found and(thankfully) how I managed to get past them with the use of SQL Profiler!

The Migration Framework works - albeit slowly as it uses the CRM Web Service to stuff the records into the CRM database.  The CRM structure I've been trying to migrate to contains a large number of custom fields.  The DMF calls many stored procs to get its data (from the CDF database); these are structured by entity e.g

p_dm_GetContact
p_dm_GetContract

I was having a problem with Contacts, and after tracing a migration attempt with cut-down data (for speed), I found that the issue was that the lid was being blown off a VARCHAR(8000) in p_dm_GetContact giving the following SQL Error:

Error: 207, Severity: 16, State: 3

The error is 'Invalid Column Name'.  Further investigation revealed that a slightly dodgy piece of code (replicated in all the other similar procs) effectively truncates the dynamically created field list in the 'SELECT' statement...

-- if there are extended fields, then attached them to the end 
--   of the base field list

if len(@metadatafields) > 0
    set @entityfields = @entityfields + ', ' + rtrim(@metadatafields)

@entityfields contains a comma delimited list of fields from the 'cdf_Contact' table, and @metadatafields contains fields from 'cdf_Contact_Ext' (custom fields).  If you concatenate the two you're obviously asking for trouble.

The following shows the 'fix' to enable a bit more room (it's still not very good - but as the code's not available we can't make a nicer fix right now)

CREATE procedure p_dm_GetContact
as
begin
    set nocount on
    
    declare 
@entity            varchar(100
)
    
declare @entityfields     varchar(8000
)
    
declare @metadatafields varchar(8000
)
    
-- set target entity name
    
set @entity 
'Contact'

    
-- query base field list for targeted entity
    
exec p_dm_GetFields @entity, 'c', @entityfields 
output

    
-- query extended field list for targeted entity
    
exec p_dm_GetMetaDataFields @entity, 'ext', @metadatafields 
output

    
-- if there are fields, then attached them to the end 
        --   of the base field list    
    
if len(@entityfields) > 
0
       
set @entityfields ', ' rtrim
(@entityfields)

    
-- if there are extended fields, then attached them to the end 
        --   of the base field list
------    if len(@metadatafields) > 0
------        set @entityfields = @entityfields + ', ' + rtrim(@metadatafields)

    
if len(@metadatafields) > 0
        
set @metadatafields ', ' rtrim
(@metadatafields)

    
-- build/execute query statement
    -- the status code must be null
    
exec (
'    select top 1 '''' As CRMUpdateStatus, 
            c.ContactId As entityid,
            c.StateCode As statecode,
            dbo.p_fn_GetID(l.DestinationId) As originatingleadid, 
            dbo.p_fn_GetID(p.DestinationId) As defaultpriceLevelid, 
            dbo.p_fn_GetID(a.DestinationId) As accountid, 
            dbo.p_fn_GetID(su.DestinationId) As owninguser 
            ' 
+ @entityfields + @metadatafields + 
'

                 from cdf_Contact c

       inner join cdf_Contact_info info 
                   on c.ContactId = info.ContactId

       left outer join cdf_Contact_ext ext 
                   on c.ContactId = ext.ContactId

      left outer join cdf_Lead_info l 
                   on c.OriginatingLeadId = l.LeadId

      left outer join cdf_PriceLevel_info p 
                   on c.DefaultPriceLevelId = p.PriceLevelId

      left outer join cdf_Account_info a 
                   on c.AccountId = a.AccountId

           inner join cdf_SystemUser_info su 
                   on c.OwningUser = su.SystemUserId

           where (info.StatusCode is null)
         and (info.DestinationId is null)'
)
end
GO


Colorized by: CarlosAg.CodeColorizer

 

posted on Monday, July 23, 2007 2:48:07 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]