# Wednesday, June 23, 2010

Generate SQL Server Inserts from existing data - with Identity Inserts

A good while ago I posted a stored proc that would generate insert statements for table data, along with simple filtering capability.

I broke this out again today, as I needed to recreate part of a database on a local machine.  I didn't have knowledge of the schema so I just went about fixing each constraint error and adding the reference tables as required to my script.  After manually adding 'SET IDENTITY_INSERT xx' about a dozen times I added the functionality to the Stored Proc - so if your table has an identity column it will now wrap the results in IDENTITY_INSERT statements, saving you a bit more time, and headaches.

An interesing side effect if you're explicitly inserting identity values is that the order of your inserts may become pertinent - especially if you've got foreign keys referencing the same table.  I then just added the capability to 'order by'.
 
There's no automatic way to switch off column headers, so you'll need to configure that in the Resutls Output options in Query Analyzer (or management studio).

If you run the proc into 'Northwind' and run:

usp_insertgenerator Employees, @order='EmployeeID'

You'll get...

SET IDENTITY_INSERT Employees ON

INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('1','Davolio','Nancy','Sales Representative','Ms.',convert(dateti
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('2','Fuller','Andrew','Vice President, Sales','Dr.',convert(datet
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('3','Leverling','Janet','Sales Representative','Ms.',convert(date
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('4','Peacock','Margaret','Sales Representative','Mrs.',convert(da
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('5','Buchanan','Steven','Sales Manager','Mr.',convert(datetime,'1
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('6','Suyama','Michael','Sales Representative','Mr.',convert(datet
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('7','King','Robert','Sales Representative','Mr.',convert(datetime
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('8','Callahan','Laura','Inside Sales Coordinator','Ms.',convert(d
INSERT Employees(EmployeeID,LastName,FirstName,Title,TitleOfCourtesy,BirthDate,HireDate,
Address,City,Region,PostalCode,Country,HomePhone,Extension,Photo,Notes,ReportsTo,PhotoPath)
  VALUES('9','Dodsworth','Anne','Sales Representative','Ms.',convert(datet

SET IDENTITY_INSERT Employees OFF


Here's the script.

if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[usp_InsertGenerator]') and 
OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[usp_InsertGenerator]
GO


CREATE PROC dbo.usp_InsertGenerator
(@tableName varchar(100), @where varchar(1000) = NULL, @order varchar(1000) = NULL) as

SET NOCOUNT ON

--Check whether the table has an identity column
DECLARE @TableHasIdentityColumn BIT
SELECT @TableHasIdentityColumn = 1
from INFORMATION_SCHEMA.COLUMNS
where TABLE_SCHEMA = 'dbo'
and COLUMNPROPERTY(object_id(TABLE_NAME), COLUMN_NAME, 'IsIdentity') = 1
AND TABLE_NAME = @tableName

IF ISNULL(@TableHasIdentityColumn, 0) = 1
    PRINT 'SET IDENTITY_INSERT ' + @tableName + ' ON'

--Declare a cursor to retrieve column specific information for the specified table
DECLARE cursCol CURSOR FAST_FORWARD FOR 
SELECT column_name,data_type FROM information_schema.columns WHERE table_name = @tableName
OPEN cursCol
DECLARE @string nvarchar(3000) --for storing the first half of INSERT statement
DECLARE @stringData nvarchar(3000) --for storing the data (VALUES) related statement
DECLARE @dataType nvarchar(1000) --data types returned for respective columns
SET @string='INSERT '+@tableName+'('
SET @stringData=''
 
DECLARE @colName nvarchar(50)
 
FETCH NEXT FROM cursCol INTO @colName,@dataType
 
IF @@fetch_status<>0
 begin
 print 'Table '+@tableName+' not found, processing skipped.'
 close curscol
 deallocate curscol
 return
END
 
WHILE @@FETCH_STATUS=0
BEGIN
IF @dataType in ('varchar','char','nchar','nvarchar')
BEGIN
 SET @stringData=@stringData+''''+'''+isnull('''''+'''''+'+ @colName+'+'''''+''''',''NULL'')+'',''+'
END
ELSE
if @dataType in ('text','ntext') --if the datatype is text or something else 
BEGIN
 SET @stringData=@stringData+'''''''''+isnull(cast('+ @colName+' as varchar(2000)),'''')+'''''',''+'
END
ELSE
IF @dataType = 'money' --because money doesn't get converted from varchar implicitly
BEGIN
 SET @stringData=@stringData+'''convert(money,''''''+isnull(cast('+ @colName+' as varchar(200)),''0.0000'')+''''''),''+'
END
ELSE 
IF @dataType='datetime'
BEGIN
 SET @stringData=@stringData+'''convert(datetime,'+'''+isnull('''''+'''''+convert(varchar(200),'+ @colName+',121)+'''''+''''',''NULL'')+'',121),''+'
END
ELSE 
IF @dataType='image' 
BEGIN
 SET @stringData=@stringData+'''''''''+isnull(cast(convert(varbinary,'+ @colName+') as varchar(6)),''0'')+'''''',''+'
END
ELSE --presuming the data type is int,bit,numeric,decimal 
BEGIN
 SET @stringData=@stringData+''''+'''+isnull('''''+'''''+convert(varchar(200),'+ @colName+')+'''''+''''',''NULL'')+'',''+'
END
 
SET @string=@string+@colName+','
 
FETCH NEXT FROM cursCol INTO @colName,@dataType
END
DECLARE @Query nvarchar(4000)
 
SET @query ='SELECT '''+substring(@string,0,len(@string)) + ') VALUES(''+ ' + substring(@stringData,0,len(@stringData)-2)+'''+'')'' FROM '+@tableName
IF @where IS NOT NULL
 SET @query = @query + ' where ' + @where
IF @order IS NOT NULL
 SET @query = @query + ' order by ' + @order
 
exec sp_executesql @query
 
CLOSE cursCol
DEALLOCATE cursCol


IF ISNULL(@TableHasIdentityColumn, 0) = 1
    PRINT 'SET IDENTITY_INSERT ' + @tableName + ' OFF'
GO 

posted on Wednesday, June 23, 2010 10:03:29 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [2]
# Thursday, June 17, 2010

Use SQL Server Trusted Connections with ASP.NET on Windows 2003 without impersonation

Access control and troubleshooting 401 errors must be one of the most annoying and recurring issues with configuring IIS.  One of the problems is that it's often quite a long time between issues, and you simply forget how you solved something last time.  This time I decided to write it all down as I set things up.

Target Scenario
My target scenario is a local intranet, where you want to use a 'service account' to access SQL Server, directly from your 'trusted' web application, removing the need for impersonation. 

The benefits of this are of course that you can take advantage of connection pooling, but also removing the need to configure passwords in web.config for SQL users (or specific, impersonated domain users).  This also removed the overhead of configuring specific domain users and their SQL Server permissions.  It may also be that you just want to simplify your security model to work only on Windows authentication across the stack.  

SQL Server
  1. Create a new role in the database you're accessing, for the purposes of your application
  2. Add your service domain user account to the role in SQL Server
  3. Assign permissions to objects, stored procedures etc to the role (not directly to the user)
IIS/Web Site
  1. Set up your web site/application as you would normally - one way to do things....
    1. Create your web application root folder on the web server
    2. Copy your files (or use your deployment tools/scripts to do this)
    3. Create a new application pool to house your new web application (probably model this from the default web site).  This is important as this is where the credentials will be set
    4. Create the new IIS web application against the root folder (if not already done as part of step 2)
    5. Associate the new IIS application with the new application pool
    6. Set the ASP.NET version of your IIS application appropriate (may need to restart IIS here)
  2. Ensure 'Integrated Security' is set ON in the Directory Security tab, and 'Anonymous' access is switched OFF
  3. Set the application pool's 'identity' to the domain user you want to run the application (and connect to SQL Server) as
  4. Open a command window and Go to the windows\Microsoft.NET\Framework\vXXXXX folder
    1. run aspnet_regiis -ga <domain>\<user> to grant necessary access to the metabase etc for the service account (as per http://msdn.microsoft.com/en-us/library/ff647396.aspx#paght000008_step3 )
    2. In the command window go to the inetpub\adminscripts folder, and set NTAuthenticationHeaders metabase property as per instructions at http://support.microsoft.com/kb/326985.  you can also use MetaEdit from the IIS Resource Kit to change this.  If you're fully configured to use Kerberos then you can potentially skip this step, as it's all about making IIS use NTLM authentication.
  5. Navigate to 'Web Service Extensions' in IIS Manager, and ensure that the ASP.NET version you're targeting is 'allowed'.  e.g. ASP.NET 4.0 is 'prohibited' by default.
Summary
So here we've circumvented the need to use impersonation by running the ASP.NET application as a specific domain user that is configured as a SQL Server Login, and granted the right access by means of a SQL Server role.  The main work is the plumbing to get IIS to work happily with that user in standard NTLM authentication (you may be able to use Kerberos depending on your network configuration).

Other background on creating service accounts can be found at http://msdn.microsoft.com/en-us/library/ms998297.aspx

posted on Thursday, June 17, 2010 11:15:53 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Friday, May 21, 2010

Run SharePoint Designer 2007 Workflows as System User

There's quite a bit of documentation and confusion around SharePoint Designer workflows and the context under which they run.  I'd naively assumed that Workflows would run in the context of the user that initiated the workflow, but with system user permissions.

The following offers some insight and an approach for certain people to follow in the case they have email-enabled libraries:
http://blogs.technet.com/victorbutuza/archive/2009/03/14/workflows-do-not-start.aspx

This didn't help me, as I'd got some 'working' workflows that, once tested with a non-privileged user simply didn't complete, as they hit permissions issues.

The specific issue was that these users have the ability to add into a list, but not edit, delete or approve, as the entry into the list is via a complex InfoPath form.  Workflow actions occur on addition to the list to route the request to an appropriate approver, and kick of an escalation process.

It appears that SharePoint 2010 has a new Impersonate feature in Workflow, but hey - I'm still using 2007, and am 'specifically' using SharePoint Designer (for business compatibility purposes).  I've used a lot of third-party SPD Workflow actions from Nick Grattan, I Love SharePoint, and wasn't about to be beaten, as I can currently do everything I need from SPD Workflows. 

I finally found a solution that did exactly what I needed).

Though vigorous googling, I found in the comments of the Useful SharePoint Workflow Activities, that people were complaining that the 'Start a Workflow' action was running the new workflow as the system user.  Not me!  - that's exactly what I need.

I downloaded and installed the activities, but had to apply another update to get them to work properly in SharePoint Designer.

I now have a new 'wrapper' workflow, which is automatically called on 'new item' to the list, and this calls the 2 existing workflows (now manually started), that were previously set to be automatic. 

This has had an added benefit, in that the ordering of these workflows is now consistent as one is called after the other.  The only thing to bear in mind of course is whether this behaviour is what you want, based on your situation.

posted on Friday, May 21, 2010 9:40:54 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Thursday, May 06, 2010

Fixing SharePoint error: No item exists at [url]?ID=n. It may have been deleted or renamed by another user

Yesterday I was creating a page in SharePoint Designer and on testing got a server error:

No item exists at http://sharepointsite/WebPages/My-Page?ID=56.  It may have been deleted or renamed by another user.

I'd got code in the page that examined the 'ID' QueryString parameter and tried to load that item from a particular list.  This same code was working on another page.

This MSDN article described the problem,

This problem happens because Sharepoint has its own variable named ID 
which it uses to identify documents/pages on the server. Our solution 
should not be using a variable named ID.
I changed my QueryString parameter to RequestID instead and all was good, apart from the thought of why the other page still worked fine. 

I then realised that if it didn't know what list it was trying to target, then the 'List' querystring parameter needs to be there too - as per other 'application page' requests in the _layouts folder, like approve/reject.

It then became obvious that I was doing too much work anyway:

Passing in the List as well as the ID allows you to then just get straight to the list item with the following code...

        SPListItem item = (SPListItem)SPContext.Current.Item;

as the List and ID are already taken care of.  A little neater than what I 'was' doing....

        SPListItem currentItem = null;

        //Get ID and List Item information
        requestList = SPContext.Current.Web.Lists["My List"];        
        if (requestList != null && Request.QueryString["ID"] != null)
        {
            if (int.TryParse(Request.QueryString["ID"].ToString(), out requestID))
            {
                //Get Item
                currentItem = requestList.GetItemById(requestID);


            }
            
        }

This code's still fine (apart from the querystring 'ID' name) if you need to load an item from a list that you don't know the guid for at the time.

posted on Thursday, May 06, 2010 9:10:51 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Friday, April 30, 2010

Pass in any parameter to an InfoPath form with one piece of code

One thing that's lacking in Microsoft InfoPath 2007 is the ability to simply map input parameters to the main data source (which is most likely where you want them to go). 

Unfortunately there's no getting around the need to write code to 'receive' your input parameters, but with a thought you'll be able to pass in parameters with the same name as the fields in your form and have one block of code to paste into the code behind all forms - that will work for all. 

This includes a couple of utility functions that make life a bit easier when coding around fields in the form.
The 'DeleteSelf' line around the nil attribute is something that apparently gets around data type errors if you've got fields that aren't just 'string' - e.g. 'number' etc.  I found this worked in the InfoPath client, but not in a browser (and had to change my field data type back to string, and add some regexp validation).

        public void FormEvents_Loading(object sender, LoadingEventArgs e)
        {

            //Try and identify any input parameters and put them into their requisite places in the main data source
            foreach (string parameter in e.InputParameters.Keys)
            {

                //Try and find in the main data source, then set the value
                XPathNavigator formNode = SelectSingleNode(String.Format("//my:{0}", parameter));
                if (formNode != null)
                {
                    if (formNode.MoveToAttribute("nil", "http://www.w3.org/2001/XMLSchema-instance"))
                        formNode.DeleteSelf();

                    formNode = SelectSingleNode(String.Format("//my:{0}", parameter));
                    formNode.SetValue(e.InputParameters[parameter]);
                }
            }
        }
        
        /// <summary>
        /// Select a single node from the Main data Source
        /// </summary>
        /// <param name="xpath"></param>
        /// <returns></returns>
        private XPathNavigator SelectSingleNode(string xpath)
        {
            string ns = LookupNamespace("my");
            XPathNavigator navigator = MainDataSource.CreateNavigator();
            return navigator.SelectSingleNode(xpath, NamespaceManager);
        }
 
 
        private string LookupNamespace(string ns)
        {
            return NamespaceManager.LookupNamespace(ns);
        }



posted on Friday, April 30, 2010 10:18:22 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]

Embedding InfoPath forms in SharePoint WebPart pages using the XmlFormView control

I recently needed to take a simple InfoPath form and surface that through a 'themed' SharePoint page.  It's possible to load the form in a browser without needing a specific page, but this effectively eliminates your master page from the equation.

Nick Grattan wrote an excellent paper on using the XmlFormView control to contain InfoPath forms on SharePoint webpart pages.

This is all great until you then need to send in input parameters to the form.  I looked around for quite a while before I found the (rather obvious) Initialize event allows you to set the Input parameters of the form.  All other methods of loading the form (InfoPath client, FormServer.aspx) use a querystring-style format.

The following code should do the trick in the case where you're passing information from a current 'list item' to the form:
 
    private SPList requestList = null;
    private int requestID;
    private SPListItem currentItem = null;

    protected void Page_Load(object sender, EventArgs e)
    {

        XmlformView1.Initialize    += XmlformView1_Initialize;
        //Get ID and List Item information (passed into this page)
        requestList = SPContext.Current.Web.Lists["My List"];        
        if (requestList != null && Request.QueryString["ID"] != null)
        {
            if (int.TryParse(Request.QueryString["ID"].ToString(), out requestID))
            {
                //Get Item
                currentItem = requestList.GetItemById(requestID);

                if (currentItem == null)
                    throw new ArgumentException("The specified ID does not exist");

            }
            
        }
        
    }

    protected void XmlformView1_Initialize(object sender, InitializeEventArgs e)
    {
        //Set input parameters for embedded Form
        e.InputParameters["foo"] = currentItem["foo"].ToString();
        e.InputParameters["bar"] = currentItem["bar"].ToString();

    }
I discovered that in order to write code in a WebPart page (using SharePoint Designer) you'll need to add a PageParserPath to the web.config as follows - otherwise you'll get an error saying something like 'code not allowed in this page'.  I've got a folder called 'WebPages' that houses the page.  The standard web.config already has the PageParserPaths element:

    <SafeMode MaxControls="200" CallStack="true" DirectFileDependencies="10" 
TotalFileDependencies="50" AllowPageLevelTrace="false">
<PageParserPaths>
    <PageParserPath VirtualPath="/WebPages/*" CompilationMode="Always"
AllowServerSideScript="true" IncludeSubFolders="true" />
</PageParserPaths>
</SafeMode>


NOTE:
If you're showing/hiding the form based on a flag/checkbox etc in the page (I'm doing this), then make sure you wrap the XmlFormView in a container Panel/div that you show/hide server-side, rather than showing/hiding the XmlFormView itself.  For some reason this doesn't seem to work properly (at least in FireFox).

posted on Friday, April 30, 2010 9:57:12 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Friday, April 09, 2010

LINQ to SQL Connection Strings with Class Library and Web.Config

Most Microsoft technologies that you can operate with a GUI come with some tradeoffs.  Things have certainly improved over the years and now something like the LINQ to SQL designer is pretty trouble free - unless of course you have something like this fairly common scenario:

I had a class library (Data Access), and decided to add LINQ to SQL classes for a new database that was being introduced.
This class library is also ultimately being consumed by WCF web services.  I have dev, test, prod environments, so I use ASP.NET Web Deployment projects to change configuration per environment for things like appSettings and connectionStrings.

It therefore followed that I wanted to configure the LINQ DataContext connection properties in web.config.  Out of the box you'll find your connection properties go into your Settings properties class, which gets a little bit in the way.

If you start playing around with the generated classes to change where you're getting the connection info from then any changes in the designer will wipe them out, so a (relatively) pain free approach to setting your connection safely is the following:


Go to your LINQ to SQL designer and remove the Connection String, and set Application Settings to False

Create a new partial class to mirror your DataContext, and set the constructor to retrieve from your alternative source...

using System;
using System.Linq;
using System.Configuration;
using System.Data.Linq;

namespace CodeBureau.Services.DataAccess
{
    public partial class MyDataContext : DataContext
    {
        public MyDataContext()
            : base(ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString, 
mappingSource) { OnCreated(); } } }

This will leave all your generated code intact, but will sort out your configuration woes.
posted on Friday, April 09, 2010 8:27:54 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]
# Thursday, April 01, 2010

Returning New Autonumber ID from Microsoft Access using ADO.NET and @@IDENTITY

Some things you hope you never have to know.  Well today I needed to call a query in a legacy Access database (insert) and return the ID of the new record.

I had some normal frustrations with parameters, data types and the like.  One piece of advice here - always make sure you explicitly set data types for parameters to Access queries, otherwise you'll likely get issues when you make the call from ADO.NET, and the error message probably won't tell you much at all.

When all that was resolved, I discovered that it's actually possible to use the same connection you used for your insert, to get the new ID (for an Autonumber field), by simply executing the following SQL

SELECT @@IDENTITY

Surprising and simple


posted on Thursday, April 01, 2010 9:55:49 AM (AUS Eastern Daylight Time, UTC+11:00)  #    Comments [0]