Infopath Performance tips #2

In my last post, I mentioned that I'd recently gone through one of my forms and identified several causes of slow performance.  I was able to reduce the form's load time from 40 to 2 seconds.  Most of that performance gain came from moving file attachments outside the form for storage.  I'll be talking about how I did that in my next post. 
Today I'm going to talk about Web Service calls.  When I was testing my load times, I found that even the quickest web service call took at least 0.5 seconds.  Since my form originally had 7 such calls, that time was really stacking up.  I went through my form and eliminated all my Web Service calls that were being called during the On Load event. :

First, I replaced Web Service calls with native APIs whenever possible.  For example, I was pulling in information about the current user using the Sharepoint User Profile web service.  I replaced this with a call to the User Profile Manager.  Here's a snippet where I get the user's Department:
ServerContext context = ServerContext.GetContext(myWeb.Site);
UserProfileManager upMgr = new UserProfileManager(context);
UserProfile currentProfile = upMgr.GetUserProfile(loginName);
departmentField.SetValue(currentProfile["Department"].toString());

Some Web Service calls just can't be replaced...so I hid them. For instance, on my form I was contacting our financial system and pulling in the latest budget numbers.  This information needed to appear on my form, and the web service was the only way to get at that information.  What I did was move the budget information off the default view and put it on a new budget view instead. 
That eliminated the need to populate the information on the form load.  My form already uses Button Tabs to move between views.  All I had to do was add my Web Service call to the budget tab.  That way the budget web service doesn't get called until it is needed. The user doesn't notice the slight delay because the page is being redrawn. 
You could use the same approach with a hidden section that is displayed by clicking on a button, or selecting a radio option.

Infopath Performance tips #1

In the next few posts I'm going to share some of my experiences with Infopath performance tuning.  Recently I took a look at one of my earliest forms to try to improve performance. 

The form in question was taking around 20 seconds to load when new, and up to 40 seconds to load when loaded with data.  Obviously, this was not acceptable.  I did a code review and got the load time down to a consistent 2 seconds.

Here's what I found:
1) Web Service calls are sloooooow.  That includes calls to Web Services on the Sharepoint server for things like User Profiles. 
2) File Attachments = Death.  Any file that you attach to your Infopath form is stored as encoded text within your form.  That means if you have a 5MB Word Document attached, your Infopath document is suddenly 5MB bigger.  That leads to:
3) Post Back events make your form look slow.  This is because every time you fire a post-back event, a copy of your form is uploaded to the server...your action is performed...and an updated version of your form is downloaded from the server.  If your form is full of attachments, you can imagine how slow this can become.  Even if your form isn't large, you're still looking at a half-second during which your form is locked up.

The fixes I implemented on my form were:
1) Get rid of all Web Service calls.  Those Web Service calls I couldn't eliminate I hide by putting them into View Changes when the user is already expecting a short wait.  I eliminated all Web Service calls from my On-Load events.
2) Move file attachments outside your form.  All my attachments are now stored in a separate Document Library.  My Infopath form contains links to the files which are displayed to the user as clickable links.
3) Combine multiple Post Back events instead of firing them one at a time.  I did this by using Views to create a Wizard type interface.  Users select several Radio Button options but my post back doesn't fire until they click a Next button at which point I do all my processing at once. 

In my following posts I'll give examples for each, including the code I'm using.  First up will be how I replaced my call to the User Profile web service.

Relink Infopath Documents Web Part

We've been archiving our old documents lately, and we usually do this by moving old Infopath documents to an archive Site.  When you do this, you need to relink the Infopath documents to the new Site Collection. 

Microsoft gives you a tool to do this, the Relink Documents view that's on every Form Library.  The problem with this is that you have to select each document manually, and you're limited to 100 documents at a time.  There are hacks out there that help with both of these problems, but I wanted something better.

After some searching, I found this project: http://sprelinkdocuments.codeplex.com/  This is a Windows app that can be run on the server.  The app asks you for the GUID of the form library.  The creator of the script helpfully provided his source code, so I took that and wrapped it in a Web Part.

When you add the Web Part to your site, it checks the URL of the hosting page to see if it is on the Relink view page.  If it is, it parses the URL of the page to figure out the name of the Form Library.  If it isn't on the Relink view page, it just shows a drop-down menu with a list of all the Form Libraries on the site. 

I've forwarded a copy of my source to the owner of the original project.  It should be posted to the Codeplex site soon.

New Tool Created - Group Synch

I just posted a new tool on the CodePlex this afternoon.  Here's a lin: http://spgroupsynch.codeplex.com/

 

The tool synchronizes groups between Site Collections in Sharepoint.  You show it a Site Collection to start from, it pulls all the Groups and Group Membership and then copies it over to another Site Collection.  It's a one-way push only, although you could get that if you run the tool again with the To and From sites reversed now that I think about it.

 

I wrote the tool over the weekend because my manager was worried about the ability to get our custom workflows running on multiple site collections.  Our workflows use Group Memberships to control access to forms and assign tasks...and we have literally thousands of them.  Basically every job title at every location gets a group, eg Corporate CEO, HospitalA CFO, etc.  Trying to keep all these groups up to date is already a daunting task in our single site collection.  Just thinking of maintaining multiple copies of these groups updated across a couple dozen site collections was giving me nightmares.

 

This was a fun little script to write.  I used the basic framework that I originally came up with building my SPWakeup app.  This handles things like Logging, Run-Time options, etc.  In this version I replaced asking for a mail server IP at runtime and used the SPUtility.Sendemail() instead.  It's a much cleaner way to handle mail, and should avoid some time-out issues I've been having with our internal mail server. 

 

This is also the first time I've really taken advantage of Class constructors. In the past I'd just been createing my class object, and then manually setting the properties. This has been a silly way to do things, so I forced myself to work with constuctors on this project. I'm very happy with the results.

 

I think I can also say that I've finally gotten a real handle on working with SPGroup and SPUser objects.  I've been working a great deal with Permissions lately, so I've been knee-deep in SPMembers for the last few months.  When I was writing this tool, I actually wrote all the major functions without ever debugging or building.  When I launched it for the first time this afternoon, I was really pleased that everything worked the way I expected it to the first time.

 

 

 

 

Standard Submit/Save for Infopath

Last week I was asked to update the submit option on 5 of my old Infopath forms.  Originally these forms just used the built in Save button on the toolbar.  I was asked to give the user a menu to choose their hospital, and then submit the document to a matching folder.

I'd done something similar on another form recently, so I decided to adapt that code, and make it generic enough that I could use the resulting code for all 5 updates.  What I've come up with is something that I plan to use as the basis of all my future forms.

Here's what you need to do if you want to try it:
1) Create a Submit connection with the default name of "Main submit".
2) Create a Web Reference called ProfileService that points to /_vti_bin/UserProfileService.asmx on your server, ie http://myserver/_vti_bin/UserProfileService.asmx
3) Create a reference to Microsoft.Sharepoint.DLL.

Your form will need the following:
1) FileName field:  Used to hold the file name once the form is submitted, ie file - 01-01-2001.xsn.  The file name is built using the Name field + the current date.  If there are duplicates, we add +1 to the name, ie file-2 - 01-01-2001.xsn.
2) Name field:  This can be any text field, and is used to generate the file name.  It should be a field that is likely to be unique, but we do check for duplicate filenames, so it doesn't have to be absolutely unique.
3) [Optional] Log field:  Used to log changes to the form.  My example form logs Creation, Submission, and Saves. 
4) [Optional] Errors field: Used to hold any errors that pop up.  This example form just logs the error.  In most of my real forms, when I encounter an error, I switch to a special Error view where I show the user what went wrong, and give them a button to report the error to our help desk.
5) [Optional] FolderChoices field:  Used to build the list of available folders. (This needs to be a repeating field)
6) [Optional] SelectedFolder field: Grabs the name of the folder that the user has selected. (This should be a drop-down menu which gets its values from the FolderChoices field)

Your form will need a Save button.  Have that button call StartSaveSubmitForm().  The code will figure out whether the document has already been saved, and either generate a new file name and submit, or save to the existing document.

If you want to submit to a folder, use the Loading event below to populate your FolderChoices field with all the folders that exist in your Form Library.

You'll need to disable the Save button if your required fields are still blank.  You'll also want to disable the Name and Folder fields once the document has been submitted to avoid confusion.  (The filename is only generated the first time you save the document, so changing the Folder field will not save the doc to a new folder, and changing the Name field will not rename the document.)

If you don't want to submit to a folder, set SubmitToFolder to False.
If you don't want to use logging, comment out the references.
If you don't want to use error logging, comment out the references.

Here is the code from a stripped down form using this option. 

using Microsoft.Office.InfoPath;
using System;
using System.Windows.Forms;
using System.Xml;
using System.Xml.XPath;
using mshtml;
using Microsoft.SharePoint;
using System.Collections;

namespace submitcode
{
    public partial class FormCode
    {
        public void InternalStartup()
        {
            EventManager.FormEvents.Loading += new LoadingEventHandler(FormEvents_Loading);
            ((ButtonEvent)EventManager.ControlEvents["Save_Button"]).Clicked += new ClickedEventHandler(Save_Button_Clicked);
        }

        public void FormEvents_Loading(object sender, LoadingEventArgs e)
        {
            //////////////////////////////////////////////////////////////////////////////
            //Set the Load variables for this Form
            //////////////////////////////////////////////////////////////////////////////

            string fileName = "/my:myFields/my:FileName";
            string libraryName = "Library"; //Name of the form Library we're submitting to
            string folderChoicesFieldXPath = "/my:myFields/my:FolderChoices"; //XPath for the repeating field that will hold our folder choices           
            bool submitToFolder = true;//If you don't want to use folders, change this to false

            //////////////////////////////////////////////////////////////////////////////
           
            //Check to see if the filename has been set
            XPathNavigator xPathNav = MainDataSource.CreateNavigator();
            XPathNavigator fileNameField = xPathNav.SelectSingleNode(fileName, NamespaceManager);

            if (fileNameField.Value.ToString() == "")//Doc hasn't been saved yet
            {
                LogComment("Document created.");
                if (submitToFolder)//If we are submitting to a folder, build the list of folder names
                {
                    GetSubFolders(libraryName, folderChoicesFieldXPath);
                }
            }
        }

        public void Save_Button_Clicked(object sender, ClickedEventArgs e)
        {
            StartSaveSubmitForm();
        }
       
        private void StartSaveSubmitForm()
        {
            //////////////////////////////////////////////////////////////////////////////
            //Set the Submit variables for this Form
            //////////////////////////////////////////////////////////////////////////////

            string folderXPath = "/my:myFields/my:FolderPicker";//If you are submitting to a folder point to list, otherwise leave blank
            string fileNameXPath = "/my:myFields/my:FileName";//The filename field used to set your form's name, IE: test - 01-01-2009.xsn 
            string nameXPath = "/my:myFields/my:NameField"; //The field used to generate the first part of the filename. The current date will be post-pended           
            string libraryName = "Library"; //Name of the Form Library
            bool submitToFolder = true;//If you don't want to use folders, change this to false

            ///////////////////////////////////////////////////////////////////////////////
           
            //Call Submit or Save method depending on whether the document has been saved
            XPathNavigator xPathNav = MainDataSource.CreateNavigator();
            XPathNavigator fileNameField = xPathNav.SelectSingleNode(fileNameXPath, NamespaceManager);

            if (fileNameField.Value.ToString() == "")//Document hasn't been saved yet, so submit it
            {
                SubmitForm(folderXPath, fileNameXPath, nameXPath, libraryName, submitToFolder);
            }
            else //document has already been submitted, save changes
            {
                SaveForm(folderXPath, fileNameXPath, nameXPath, libraryName, submitToFolder);
            }           
        }

        private void LogComment(string comment)
        {
            //////////////////////////////////////////////////////////////////////////////
            //Set the Log variables for this Form
            //////////////////////////////////////////////////////////////////////////////

            string logXPath = "/my:myFields/my:Log";

            //////////////////////////////////////////////////////////////////////////////
           
            XPathNavigator xPathNav = MainDataSource.CreateNavigator();
            XPathNavigator logField = xPathNav.SelectSingleNode(logXPath, NamespaceManager);

            string log = logField.Value.ToString();

            string newEntry = DateTime.Now.ToString() + " - " + FindCurrentUser() + " - " + comment + "\n\n" + log;
            logField.SetValue(newEntry);
        }

        private void ReportError(string errorMessage, string fullError)
        {
            //////////////////////////////////////////////////////////////////////////////
            //Set the Error Logging variables for this Form
            //////////////////////////////////////////////////////////////////////////////

            string errorXPath = "/my:myFields/my:ErrorMessage";

            //////////////////////////////////////////////////////////////////////////////

            XPathNavigator xPathNav = MainDataSource.CreateNavigator();
            XPathNavigator errorMessageField = xPathNav.SelectSingleNode(errorXPath, NamespaceManager);

            LogComment("Encountered error on form.");
            errorMessageField.SetValue(errorMessage + "\n" + fullError);
        }

        private string FindCurrentUser()
        {
            try
            {
                //Use Web Service to return information about the current user
                ProfileService.UserProfileService profileService = new ProfileService.UserProfileService();
                profileService.UseDefaultCredentials = true;
                profileService.Url = SPContext.Current.Site.Url.ToString() + "/_vti_bin/UserProfileService.asmx";
                ProfileService.PropertyData[] userProps = null;
                try
                {
                    // Passing null to this method causes the profile of the current user to be returned.
                    userProps = profileService.GetUserProfileByName(null);
                }
                catch (Exception ex)
                {
                    string errormessage = "Unable to get current user profile.";
                    ReportError(errormessage, ex.Message.ToString());
                }
                ProfileService.ValueData[] values = userProps[4].Values;//4th item is Preferred Name
                if (values.Length > 0)
                {
                    return values[0].Value.ToString();
                }
                else
                {
                    return this.Application.User.UserName.ToString();
                }
            }
            catch (Exception ex)
            {
                string errorMessage = "Error finding Preferred Name for current user.";
                ReportError(errorMessage, ex.Message.ToString());
                return this.Application.User.UserName.ToString();//Default to returning username
            }
        }

        private void SaveForm(string folderXPath, string fileNameXPath, string nameXPath, string libraryName, bool submitToFolder)
        {
            try
            {
                XPathNavigator xPathNav = MainDataSource.CreateNavigator();
                XPathNavigator folderField = xPathNav.SelectSingleNode(folderXPath, NamespaceManager);

                FileSubmitConnection fs = (FileSubmitConnection)this.DataConnections["Main submit"];

                //Check to see if Submit location ends with a /, if not, add one.
                if (fs.FolderUrl.EndsWith("/"))
                { } //All clear
                else
                {
                    fs.FolderUrl = fs.FolderUrl.ToString() + "/";
                }

                if (fs.FolderUrl.Contains(folderField.Value.ToString()))//Sets right folder when saving immediately after submit
                {
                }
                else
                {
                    if (submitToFolder)//If we are submitting to a folder, and the data conn isn't pointed to it yet, do so now
                    {
                        fs.FolderUrl = fs.FolderUrl.ToString() + folderField.Value.ToString() + '/';
                    }
                }

                LogComment("Saved the document.");

                fs.Execute();
            }
            catch (Exception ex)
            {
                string errorMessage = "Error when saving existing document.";
                ReportError(errorMessage, ex.Message.ToString());
            }

        }

        private void SubmitForm(string folderXPath, string fileNameXPath, string nameXPath, string libraryName, bool submitToFolder)
        {
            try
            {
                XPathNavigator xPathNav = MainDataSource.CreateNavigator();
                XPathNavigator folderField = xPathNav.SelectSingleNode(folderXPath, NamespaceManager);

                FileSubmitConnection fs = (FileSubmitConnection)this.DataConnections["Main submit"];

                //Check to see if Submit location ends with a /, if not, add one.
                if (fs.FolderUrl.EndsWith("/"))
                { } //All clear
                else
                {
                    fs.FolderUrl = fs.FolderUrl.ToString() + "/";
                }

                if (submitToFolder)//If we want to submit to a subfoler, set that folder now
                { fs.FolderUrl = fs.FolderUrl.ToString() + folderField.Value.ToString() + '/'; }

                //Generate a unique filename for our document
                string fileName = SetFileName(fileNameXPath, libraryName, nameXPath);
                LogComment("Submitted document as " + fs.FolderUrl.ToString() + fileName);

                //Submit the document
                fs.Execute();
            }
            catch (Exception ex)
            {
                string errorMessage = "Error when trying to submit new document.";
                ReportError(errorMessage, ex.Message.ToString());
            }
        }

        private string SetFileName(string fileNameXPath, string libraryName, string nameXPath)
        {
            XPathNavigator xPathNav = MainDataSource.CreateNavigator();
            XPathNavigator fileNameField = xPathNav.SelectSingleNode(fileNameXPath, NamespaceManager);
            XPathNavigator nameField = xPathNav.SelectSingleNode(nameXPath, NamespaceManager);

            //Create a filename based on the Name Field and Today's Date
            string fileName = nameField.Value.ToString() + " - " + DateTime.Today.ToShortDateString();
            fileName = fileName.Replace('/', '-');

            //Check to see if a file with the same name already exists
            bool isDuplicate = CheckForDuplicateFileName(fileName, libraryName);

            int x = 2;//If we have a duplicate, we'll add -X to the filename, starting with 2
            while (isDuplicate)//Keep generating filename + X until we get a unique name
            {
                fileName = GenerateNewFileName(nameField.Value.ToString(), Convert.ToString(x));
                isDuplicate = CheckForDuplicateFileName(fileName, libraryName);
                x++;
            }

            //Set File Name Field to our new File Name
            fileNameField.SetValue(fileName);
           
            //Return our filename
            return fileName;
        }

        private string GenerateNewFileName(string nameFieldValue, string x)
        {
            string newFileName = nameFieldValue + "-" + x + " - " + DateTime.Today.ToShortDateString();
            newFileName = newFileName.Replace('/', '-');
            return newFileName;
        }

        private bool CheckForDuplicateFileName(string fileName, string libraryName)
        {
            bool foundduplicate = true;//Return value has to be created and returned outside the elevated security

            //Elevated security sites can't be opened with SPContext, open the site normally and get the GUID
            //Then open the site again with elevated privelages
            Guid siteGuid = new Guid();
            Guid webGuid = new Guid();
            using (SPSite getGuidSite = SPContext.Current.Site)
            {
                using (SPWeb getGuidWeb = getGuidSite.OpenWeb())
                {
                    siteGuid = getGuidSite.ID;
                    webGuid = getGuidWeb.ID;
                }
            }

            SPSecurity.RunWithElevatedPrivileges(delegate()//Run with elevated privelages so we can find docs even if the current user doesn't have access
            {
                using (SPSite mySite = new SPSite(siteGuid))
                {
                    using (SPWeb myWeb = mySite.OpenWeb(webGuid))
                    {
                        SPList requestList = myWeb.Lists[libraryName];
                        SPQuery query = new SPQuery();
                        query.Query = "<Where><Eq><FieldRef Name='Title' /><Value Type='Text'>" + fileName + ".xml</Value></Eq></Where>";
                        query.ViewAttributes = "Scope=\"Recursive\"";//Search subfolders too
                        SPListItemCollection listItemColl = requestList.GetItems(query);
                        if (listItemColl.Count == 0)
                        {
                            foundduplicate = false;//No duplicates found
                        }
                        else
                        {
                            foundduplicate = true;//Found duplicate, we'll need to create a new filename
                        }
                    }
                }
            });

            return foundduplicate;
        }

        private void GetSubFolders(string libraryName, string folderChoicesFieldXPath)
        {
            ArrayList folderArray = GrabFolderList(libraryName);//Get a list of the folders
            PopulateFolders(folderArray, folderChoicesFieldXPath);//Use the folder list to populate our repeating field
        }

        private void PopulateFolders(ArrayList folderArray, string folderChoicesFieldXPath)
        {
            try
            {
                //Attach to repeating field facility list
                XPathNavigator xPathNav = MainDataSource.CreateNavigator();
                XPathNavigator Loc = xPathNav.SelectSingleNode(folderChoicesFieldXPath, this.NamespaceManager);
                Loc.SetValue("");
                XPathNavigator templateNode = Loc.Clone();

                //Loop through all facilities and add them to the repeating field
                for (int r = 0; r < folderArray.Count; r++)
                {
                    string value = folderArray[r].ToString();
                    XPathNavigator newNode = templateNode.Clone();
                    newNode.SetValue(value);
                    Loc.InsertBefore(newNode);
                }
                Loc.DeleteSelf();//Delete close used to populate list
            }
            catch (Exception ex)
            {
                string errorMessage = "Error when trying to populate folder List.";
                ReportError(errorMessage, ex.Message.ToString());
            }
        }

        private ArrayList GrabFolderList(string libraryName)
        {
            try
            {
                ArrayList folderArray = new ArrayList();
                using (SPSite mySite = SPContext.Current.Site)
                {
                    using (SPWeb myWeb = mySite.OpenWeb())
                    {
                        SPFolderCollection folders = myWeb.GetFolder(libraryName).SubFolders;
                        foreach (SPFolder currentFolder in folders)
                        {
                            if (currentFolder.Name.ToString() != "Forms")//Grab every folder name except for Forms
                            {
                                folderArray.Add(currentFolder.Name.ToString());
                            }
                        }

                    }
                }
                return folderArray;
            }
            catch (Exception ex)
            {
                string errorMessage = "Error when trying to get sub folders from Library.";
                ReportError(errorMessage, ex.Message.ToString());
                ArrayList emptyArray = new ArrayList();
                return emptyArray;
            }
        }


    }
}