Deprecated

This blog has been deprecated... I have migrated it here:
http://www.theedgeofall.com/journeys
under the technical tags and categories.

Bending InfoPath and SharePoint to Your Will: Dynamic, Filtered Queries through Data Connections with Minimal Code

It seemed simple enough at first. All I wanted was a way to dynamically pull data from Lists and Document Libaries into InfoPath. Oh, but did I forget to mentioned that I also wanted the data filtered BEFORE it hit InfoPath because the Lists and Libraries might be massive (go figure). Trying several things, it became obvious that I needed a faster/better/easier method than any of the following:

1. Using a baked-in InfoPath data connection to a List is easy, but it retrieves ALL List items and leaves filtering to the controls. This is obviously a horrible method when working with large data sets.
2. Using custom code-behind on the InfoPath form to query SharePoint via the object model is perhaps the cleanest method, but adds code-bloat, is not very modular, and requires extra maintenance.
3. Using a web service data connection from InfoPath to SharePoint web services (like http://[site]/_vti_bin/lists.asmx?op=GetListItem) would be perfect, except that InfoPath does not form the xml in the SOAP request correctly (i.e. the way that SharePoint is expecting) and returns an error. Actually, this boggles my mind. Somebody please correct me if I'm wrong here and/or if there is a clever workaround that I couldn't find / figure out. This seems like an incredible oversight on Microsoft's part for 2 products that are supposedly designed to play nice together.
4. Since I'm in a K2 blackpearl environment, I also thought about using a SmartObject, which rivals option #2 for cleanest method, but it also adds an additional maintenance tail outside of InfoPath and requires additional wiring inside the form.

Looking for a solution, I stumbled on this article - http://blogs.msdn.com/infopath/archive/2007/01/15/populating-form-data-from-sharepoint-list-views.aspx - at the InfoPath Team blog which formed the basis for a new idea:

To summarize – if you don’t want to have to read through the whole article yourself: This little known feature allows you to pass GUIDs for Lists and Views, along with filter parameters, in the URL querystring to owssvr.dll and dynamically retrieve an XML “results” file. Furthermore, the URL itself can be used as the file location for an “XML Document” data connection in InfoPath.

The main hurdle is that, once the data connection is created, it doesn’t seem on the surface like there’s a way to dynamically update this file location to retrieve a different XML Results Document from a List or Library based on related fields in a form that has been opened. Well, it turns out that the InfoPath object model exposes this property and lets us manually execute the query.

In my particular scenario I wanted to retrieve a group of documents (metadata) from a massive Document Library where a DocumentColumn matched a particular FieldValue in the form opened. Here's what I did:

1. [Optional] Create a View of the List or Doc Lib that contains the specific fields you want retrieved into the Form
2. In the Form, add a new Receive > XML Document data connection
3. For the “location of the XML data file...” in the wizard use:
   http://[server]/[site]/[list]/_vti_bin/owssvr.dll?Cmd=Display&List={LIST_GUID}&XMLDATA=TRUE
4. [Optional] Include &View={VIEW_GUID} if you want to pull data and columns from a specific view that already exists or was created in step 1.
5. Note: the GUIDs in the querystring parameters can be in either format:
   a. "{xxxxx-xxxxx-xxxx-…}"
   b. "%7BF498875E%2D5A645B%2D…" (as copied directly from SharePoint generated URLs)
6. Complete the data connection wizard – this will add the unfiltered data connection results schema to the Form (you do want the unfiltered schema). Make sure the following settings are set as you complete the wizard:
   a. “Access the data from the specified location”
   b. [UNcheck] “Automatically retrieve data when form is opened”
7. To add the repeating table that will hold the query results, find the xml/rs:data/z:row repeating group in the new data connection and drag it onto the form
8. To complete the solution, a little code-behind is necessary (but it’s minimal and a LOT less code than method #2 mentioned at the top). There are probably several ways to do this (like wiring the code to a button or another event), but here’s my basic approach:
9. Since the ID will already be stored in the form data by the time the user opens it (in the context of a K2 workflow), I use the form’s Loading event to modify the connection URL and execute the connection.
   a. In InfoPath Designer: Tools > Programming > Loading Event
   b. In Visual Studio Tools for Office: Insert > Loading Event
10. The following code will append - at run time - the filter parameter and value from a field in the form to the XML Data Connection location URL and execute the data connection to retrieve the hits:

// we need to instantiate a typed data connection object
// to modify the query and execute the results
FileQueryConnection dc = (FileQueryConnection)(this.DataConnections
[“NameOfXmlDataConnectionCreatedThroughWizard”]);
// next, we append the filtering values from the form field(s)
// to the existing URL already stored in the data connection
// --see notes for description of readNode()
dc.FileLocation += “&FilterField1=DocumentColumnName&FilterValue1=” +
readNode(“/my:myFields/my:FieldInForm”);
//finally, we execute the data connection to return the results
dc.Execute();

Notes:

• readNode is a little helper function I add to all my InfoPath forms so that I can quickly retrieve the value of a node without having to jump through navigator objects every time I want to get something. It looks like this:
  public string readNode(string xpath)
  {
  return MainDataSource.CreateNavigator().SelectSingleNode(xpath,
  NamespaceManager).InnerXml.ToString();
  }
• I think the SharePoint URL trick supports multiple FilterFieldX and FilterValueX parameters, so in theory, you could build pretty sufficient queries using the querystring alone. For example:
  “&FilterField1=ColumnName1&FilterValue1=Value1&FilterField2=ColumnName2&FilterValue2=Value2”
  This trick might also be leveraged throughout other events in K2 workflows since it returns standard XML and the query results are stored in /xml/rs:data/z:row.

Use this knowledge wisely. And have fun.

Behind Enemy Lines: Document Information Panel Editing in InfoPath

This one really had me wrapped around the axle for a bit, and ye ol' trusty google let me down. Not a single applicable hit when I searched on ["form definition" "for dataObject"] so when I figured it out quite by accident I thought I'd contribute to the googlesphere and maybe help someone else out who might run into this. If anyone does. Ever. Ok, maybe no one else will ever have this happen to them, but it happened to me I promise.

I thought I'd go and customize the Document Information Panel for a Content Type - totally harmless right? Well, InfoPath didn't seem to think so. And when I tried to attach to the Content Type's schema as the basis for a new form (or when I tried to customize it straight from the link in the Content Type settings) i got this:

InfoPath cannot open the following form: C:\Users\[Name]\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.MSO\19D946AE.tmp\customXsn.xsn
The XML schema file specified in the form definition (.xsf) file for dataObject "list_fb2ad403-eb57-43f2-8bee-e7574092cd4d" cannot be used.

Lovely. How informative and useful. It's obvious from that error what exactly needs fixing. Ok, not really. At first I was at a total loss and decided to go home for the weekend. When I got back I realized that "list_" looked a lot like a list GUID. Since I was in a fairly new dev environment without too much clutter yet, I scanned through my lists and found a match on a list used as the source for 2 of the Look-Up Site Columns in the Content Type for which I was trying to customize the Doc Info Panel. Aha!

Removing both Site Columns from the Content Type and trying again, I was able to launch the schema for the form. Grrr... Hmmm...

So, was there anything weird about those columns? Well, they were look-up columns on a list with the multi-select and unlimited length in documents options checked. But what if I added them back to the content type and tried again?

Adding the first one back and trying again it worked! Alright. Adding the second one ... NOPE! Crash and burn. What gives? Well, this must obviously be another stupid Microsoft bug. Incidentally, both look-up columns are based on the same column of the same list. I have a "From" and a "To" column that point to the "Display Name" column a master Names List. But I won't be able to do both and still customize the Doc Info Panel. Bummer. Unless I can come up with some sort of work around (unlikely). But I'll update this if I do.

[a few hours later]

Alrighty then. The enemy can be tricked. If you HAVE to be able to do this (like I do) here's what you do:

1. change one of the site Columns to point at a different look-up list column than the other one
2. this will let you open the doc info panel successfully even with both site columns pointing at the same look-up list (albeit at 2 different columns within that list)
3. in the field properties in infopath, open up the control populated from this look-up list column and change it back to the column you want to pull in
4. in the Site Column change it back to point at the the right look-up column as well.

Voila! Happy ending. If I hadn't wasted hours on something that should work anyway.

Diabolical Microsoft Visual Studio Quirk

Yet again many things have passed that have made me go hmmm... I can't believe I have documented practically none of them. And this one almost passed unaccounted for as well, but I could not let it go. It was simply too much.

But first I must say - I am not a Microsoft hater. I use its software every day and have very few complaints. But when I do, they are huge. It's the little things that are big and the simple things that should be so easy that get botched so royally. And it drives me crazy.

After a couple days of removing a cornucopia of betas, CTPs, and various versions of Visual Studio 2005 (along with WPF, WWF, VSTO and other sundry extenstions) I was finally up and running with a clean slate to do some serious K2 and SharePoint development. Until I discovered I couldn't save any projects.

WHAT!? "The operation could not be completed. No such interface supported." Now there's a useful error. Ummm... So now file system operations are not supported... Uh... right.

So, it took longer than it should have but I finally found this thread:
http://softwarecommunity.intel.com/isn/Community/en-US/forums/thread/30236807.aspx

Which leads me to hypothesize that in all the uninstalling and re-installing perhaps this ridiculous setting in Visual Studio got mysteriously unchecked and that caused VS to stop saving projects? Right.


How much sense does a setting like this make anyway (yes, I get the point, but who in their right mind would ever want to create temporary projects); and more importantly - why does that setting seem to make it impossible to save your "temporary" project later?

After checking the box beside "Save new projects when created" and restarting VS all was well again in Not-Losing-My-Work-Land. Hmmm...

Wild Jungles of Mannual MOSS Beta 2TR Removal

So, I was merrily and methodically following the Microsoft documentation that came with the SharePoint 2007 Beta 2TR downloads, and everything was coming off without a hitch, until the Configuration Wizard choked, crashed, and then burned. In other words: The entire process was nearly complete, the updates had run flawlessly, but it was not happy. This is how I knew is was not happy:

Failed to register SharePoint services. An exception of type
System.Data.SqlClient.SqlException was thrown.
Additional exception information: Could not find stored procedure 'dbo.proc_MSS_Cleanup'.

I discovered that it was incredibly not happy, in fact, when I could not even remove Microsoft Office SharePoint Server (MOSS) 2007 from my system any more. The uninstall app launched from Add/Remove Programs just crashed within a couple seconds of Clicking Remove and then Yes. Every Time. So, I couldn’t fix it. And I couldn’t remove it and start over… at least not the easy way. The following is a record of my journey through the dark and creepy jungles of removing MOSS 2007 manually. Yes, I just said manually. Yes, it can be done.

Lessons Learned:

• The slipstream method for installing the Beta 2TR update seems by far the simplest and most stable option!!! (See “Other Resources” below for good documentation)
• Before trying the steps in this blog try running this command:
  12\bin\psconfig.exe -cmd upgrade -force

UPDATED Lessons Learned (several days after original post):

• B2TR is VERY particular about having a clean slate (no previous installations of B2 on the machine).
• My methods for removing everything mannually still did NOT make B2TR happy. I could install B2TR, configure, and setup all the farm services. I could create Web Apps and Site Collections - team sites worked just fine. But the publishing features and any publishing sites plain did not work. I suspect this has something to do with some WSS B2 stuff surviving the MOSS B2 removal, but I can't be sure.
• Bottom line - rebuild the server to make sure. This was the only way I could get everything to work after days of messing with it.
• Again: The slipstream method is by far the easiest and most reliable method to install B2TR.
• Even the basic setup in the Config Wizard (after a B2TR slipstream install) required that I set up the database (SQLServerExpress in my case) separately.

DISCLAIMER:
The following procedures worked for me to manually remove an installation of MOSS Beta 2 that was corrupted by the Beta 2TR updates. After all this, I was able to successfully reinstall Beta 2TR using the slipstream method. HOWEVER, you might seriously mess up your own system trying this, so be warned. Don't even think about attempting it unless you know exactly what you're doing and/or don't mind rebuilding your machine from scratch (which I actually recommend as an easier and cleaner alternative to what is described below). Of course, if you don't have any choice because you’re avoiding a complete rebuild at all costs (like me), then have at it!

NOTES:

• When I say "delete" I mean (to the recycle bin) unless otherwise stated. Probably not necessary but maybe a cheap form of minimal insurance.
• Backup any relevant parent registry keys before making edits (obviously these don’t go to the recycle bin when deleted :) )
• Hopefully my shorthand wildcard (*) notation will not be confusing. If there are multiple services to stop that start with SharePoint, for example, I might say: stop all SharePoint* Services… Should be intuitive.

Do the following to manually remove MOSS 2007 Beta 2TR:

1. Delete folder C:\Program Files\Common Files\Microsoft Shared\SERVER12
   (This is the key step that tricks your system into thinking MOSS has not been previously installed. After this step you should be able to re-run the setup and it will start from scratch with the Product Key instead of taking you to the Repair / Remove prompt. Don’t get excited yet, you still need to complete the following steps first)
2. In Add/Remove programs find Microsoft Office SharePoint Server 2007 (Beta) and click Remove
   - you should get an error, click OK to remove from list anyway
3. [Optional?] Delete every MOSS 2007 and WSS v3 related key UNDER HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall - do not delete the main parent key : HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall - you will have to go through each sub-key and decide if it applies to MOSS (which, if you've tried to remove MOSS several times like me, might be a bunch) this is easy, just look at the value for the "DisplayName" String to be "Microsoft Office Professional 2007 (Beta)" or something similar
   - most keys to delete will probably end in 0000000FF1CE, but don’t forget about the OSERVER key (might get errors while deleting this one, but press on)
4. Delete folder C:\Program Files\Common Files\Microsoft Shared\web server extensions\12
5. [Optional?] Using a tool like “SQL Server Management Studio Express” stop and/or delete your SharePoint database server instance (in my case, from a default basic installation: OFFICESERVERS). Disable its corresponding service in the system Services.
6. In IIS, delete all MOSS and WSS v3 Web Sites and App Pools (at least 4-5 of them) and run iisreset at the command line.
7. Delete folder C:\Inetpub\wwwroot\wss
8. [If applicable] Kill the following processes:
   - Microsoft.Office.Server.Conversions.Launcher.exe
   - Microsoft.Office.Server.Conversions.LoadBalancer.exe
9. Disable all Windows Sharepoint* and Office Document Conversions* services in system Services
10. Delete the folder C:\Program Files\Microsoft Office Server
11. Delete the folder C:\Program Files\Microsoft Office Servers (if you get access denied, you will probably have to add full control permissions to the local administrator account – or whatever account you installed MOSS with – on that folder and make sure Read Only is unchecked and those settings pushed down to all sub folders. The folder C:\Program Files\Microsoft Office Servers\12.0\Bin\HtmlTrLauncher will not let you access / delete it otherwise)
12. This step is scary. Run a file Search for wss in your c:\windows folder. Delete EVERYTHING that might pertain to wss (but NOT things that are iffy like wssoc.dll and wssbrand.dll which both apply to Windows Storage Services) Repeat using the following searches (there will be lots of stuff in the Windows\inf folder to delete):
    - Sharepoint
    - OSS
    - [do NOT delete anything from the .Net 2.0 folders]
13. Another scary step. Do the same thing with the registry (search and destroy). This will be much easier for you if you do NOT have SharePoint Designer (or Office 2007 Beta 2) installed, heh heh… I found and deleted the following keys (plus a few other similar ones):
    - Several keys under HKEY_CLASSES_ROOT\CLSID\ that hit on a search for sharepoint like {000502B8-C8FE-43C0-B0D9-FD18AB931AE9}
    - and {0451E372-3698-44A4-B7EE-C7B7F448C609}
    - and {04971529-2F83-4173-BD6C-244A897E5A0B}
    - and a whole bunch more... you get the idea. actually, i skipped some of those too, because it was taking too long. the key guid names might be different for you.
    - Roughly 30 Microsoft.Sharepoint.* String Values in the following key: HKEY_CLASSES_ROOT\Installer\Assemblies\Global
    - HKEY_CLASSES_ROOT\Installer\Patches (the whole key)
    - every key ending in F01FEC found under the following key: HKEY_CLASSES_ROOT\Installer\Products to clean up Add/Remove Programs (I found a bunch of weird things got inserted into here because of something I did during this process… you might not have a bunch of entries to delete here)
    - every key ending in F01FEC under the following key: HKEY_CLASSES_ROOT\Installer\UpgradeCodes
    - All Microsoft.Office.Server.* and Microsoft.Sharepoint.* keys from HKEY_CLASSES_ROOT
    - All OSearch.* keys from HKEY_CLASSES_ROOT
    - HKEY_CLASSES_ROOT\Record\{0641D69F-CD7A-3EB5-89B8-2A1741786A64} and all similarly prefixed keys that hit on a search for sharepoint
    - All HKEY_CLASSES_ROOT\SharePoint.* keys
    - All HKEY_CLASSES_ROOT\SPSearch.* keys
14. Reboot and hope for the best.

Try running the setup again from your local folder. After doing the above steps, I followed the instructions here: http://www.sharepointblogs.com/files/12210/download.aspx with the following exceptions:

• During the Advanced setup, I had to take a couple extra steps in the Config Wizard because it couldn’t determine if the server was joined to a Farm or not. I got an error, but clicked OK, and the Config Wizard let me disconnect from the unknown farm. Then I just re-ran the Config Wizard and, much to my amazement, I was in business.
• I did not use SQL2000 SP4 as the instructions did. I used a fresh new instance of SQL Sever Express by running the SQL Server Express Setup. I could have reused the OFFICESERVERS instance left over from the Basic installation I did a while ago, but I wanted a clean slate. In creating the new instance, I specified the [machine]\administrator account (the same account used to install MOSS) to be the sql service account. Once my new instance was installed, I completed the Advanced Setup and just put in my settings like the example – using [Machine]\[New Instance Name] for the Database Server field.

Cleaning up:

• If you have errors running the setup again, double check that the folder C:\Program Files\Common Files\Microsoft Shared\web server extensions\12 is truly deleted. At one point I had an error about not being able to upgrade WSS because the database [which didn’t actually exist] was larger than 4GB… after poking around I found that the folder mentioned had mysteriously re-appeared! Maybe because I ran the setup again somewhere in there as I was experimenting… or something.
• I must have gotten too zealous in the registry because I couldn’t launch Word 2007 Beta 2 after I was done. It said that it wasn’t installed for this user. So, I’ll just have to re-run the setup and do a repair. That should fix it.
• If I have overlooked anything or left anything out, PLEASE let me know! Thanks!

Your time might be better spent just rebuilding the server from scratch. But if that is the route you need to avoid at all costs, hopefully, this will help you get back on your feet.

Other Resources:
http://blogs.msdn.com/sharepoint/archive/2006/09/15/756692.aspx

An Introduction

So, I've been pondering this blog for a while. In the meantime, many things that make me go Hmmm have escaped unrecorded. I initially thought this would be a gigantic melting pot of topics personal and technical. In the end, I could see how difficult it would be to keep that mess untangled, and so I have decided to start on the technical side of things first. Why? Because wrestling an installation of Microsoft Office SharePoint Server 2007 Beta 2TR to the ground has finally pushed me over the edge and I wanted to share what I found with others suffering the same fate (post soon to follow).

edgeDev is me in my technical playground, but hopefully it will also reflect who I am in other playgrounds. I will likely shamelessly link between my other future blogs and this one. But here I will cover things as they arise in my day-job-life as a web developer / SharePoint architect. I do not hate Microsoft, but topics revolving around their products will probably dominate this blog since they are one of the primary sources of things that make me go Hmmm...