Tagged: Add-ins

Updating Legacy SharePoint Customizations to the Add-in Model

If you follow this blog, you have probably read many of my posts through the years dedicated to the Add-in (formerly app) model for SharePoint 2013, 2016, and SharePoint Online. Microsoft introduced the Add-in model in SharePoint 2013 to address the various shortcomings associated with running custom code on the SharePoint server — most notably, of course, that custom code cannot be deployed to SharePoint Online in Office 365. SharePoint-hosted Add-ins are great for simple functionality that can be achieved with 100% client-side solutions (HTML, CSS, and JavaScript), while provider-hosted Add-ins allow you to run custom server-side code in Azure or anywhere other than the SharePoint server.

Using the Add-in model is a no-brainer for most new development efforts, especially if you are developing for SharePoint Online and still need to write server-side code. (At the time of this writing, the new SharePoint Framework is still in Developer Preview but is definitely worth adding to your repertoire for client-side SharePoint development in the years to come!)

But what about your existing solutions? Many organizations using SharePoint maintain large portfolios of custom code solutions that leverage different legacy development approaches advocated by Microsoft through the years, and most of the custom code developed for earlier versions of SharePoint cannot be directly migrated to the cloud. If your organization decided to move to the cloud tomorrow, where would you begin?

Taking inventory of your customizations

I have found that most legacy SharePoint customizations can be placed into the following categories:

  • User interface enhancements (master page, branding, themes, custom web parts)
  • Declarative items (XML for custom site columns, list instances, and content types)
  • Timer jobs (and other administrative extensions to the platform) running in SharePoint Central Administration on-premises

These customizations may take the form of farm or sandboxed solutions that depend on SharePoint’s legacy feature framework. In the case of some user interface customizations, custom HTML markup, CSS, or even JavaScript might be embedded directly within a site’s master page file. For the most part, these are situations we should try to avoid as much as possible when migrating our customizations to the cloud for the following reasons:

  • Farm solutions cannot be deployed to SharePoint Online.
  • Sandboxed solutions that contain managed code can no longer be deployed to SharePoint Online (note that sandboxed solutions contain a managed code assembly by default, even if the solution does not contain any managed code).
  • Custom master pages, while still supported in SharePoint Online, should be avoided unless absolutely necessary. (Microsoft has a history of making significant changes to the default master page in SharePoint Online, such as the addition of the app launcher/waffle/Hollywood Squares in the upper left corner. Sites with heavily customized master pages may not always have access to these changes as they are introduced in the future.)

The role of the Add-in model

When migrating these customizations, keep in mind that you will not necessarily be building and deploying a SharePoint Add-in to replace each customization. In fact, in many cases, you will simply be leveraging techniques from the Add-in model. These include paradigms that were introduced and/or popularized with the introduction of the Add-in model:

  • Using client-side script wherever possible
  • Remote provisioning of assets (everything from CSS/JavaScript files to site columns, content types, etc.) using the .NET Managed client object model (CSOM) or PowerShell
  • Running server-side code outside the SharePoint server that communicates with SharePoint via its APIs

In some cases, all you may need to do is build and deploy a “throwaway” provider-hosted Add-in within a local development environment that performs some one-time remote provisioning tasks, but is not intended to be accessed by end users and can be removed after its work is done.

Add-in model techniques enable us to make our SharePoint customizations with a “lighter touch.” By lighter touch, I am referring to a lighter footprint and/or impact on the SharePoint server. As we decrease the burden on the SharePoint server, we enable SharePoint to run more reliably and efficiently while also making it easier to administer and upgrade (at least on-premises…yes, these techniques all work on-prem as well!) To better explain this concept, I put together the table below contrasting legacy SharePoint development approaches with their modern lighter touch equivalents:

Legacy Approach Lighter Touch
Farm solutions with custom code running in the SharePoint IIS worker process Provider-hosted Add-in with custom code running outside of SharePoint
Farm or sandboxed solutions that deploy declarative artifacts such as site columns, content types, and list instances Remote provisioning these artifacts from a provider-hosted Add-in using CSOM
Custom master pages with embedded markup, styles, and script Remote provisioning of branding assets, embedding of JavaScript UI customizations using ScriptLink custom actions
Custom timer jobs running in Central Administration using the SharePoint server object model Remote timer jobs running as Windows scheduled tasks or Azure WebJobs using CSOM

 
SharePoint Developer PnP: here to help!

Because the Add-in model represents such a strong departure from the way things were done in the past with full-trust code in SharePoint, Microsoft started the SharePoint Developer Patterns and Practices (or PnP) initiative to assist developers with transforming their existing solutions to be cloud-ready as they migrate to SharePoint Online in Office 365.

The PnP team maintains several Github repositories that include reusable, production-ready components, templates, and solution starters that demonstrate the preferred modern approach to SharePoint development — making customizations with a lighter touch wherever possible.

Eric Overfield just published a great blog post detailing the PnP initiative and how you can get involved.

My Pluralsight course

If you are interested in learning more, I recently published a Pluralsight course with lots of demos: Updating Legacy SharePoint Customizations to the Add-in Model. The course covers the process of migrating a heavily customized on-premises SharePoint site to SharePoint Online from start to finish. The clip below is a demo from the course showing how a legacy timer job customization can be made cloud-ready with the help of the PnP Timer Job Framework and Azure WebJobs.

In this course, you will see me leverage the PnP Core Component to drastically simplify writing .NET Managed CSOM code. I also use several great samples and solution starters from the PnP team, all with the objective of easing the pain associated with migrating legacy SharePoint customizations to the Add-in model.

Accessing SharePoint Data from Provider-Hosted Apps: Use the Right Context

I have spent the better part of the last two years working with the new app development model for SharePoint 2013. Recently I have focused on building apps that access, manipulate, and interact with data stored in SharePoint Online with Office 365. If you have done any development using the client-side object model (CSOM) for SharePoint, you understand the importance of instantiating the proper ClientContext object to access data in a particular SharePoint site. The ClientContext constructor takes as an argument the URL of a SharePoint site and allows you to access data stored in the Lists collection of the Web associated with it. In this post, I will discuss the various context objects you should use in your provider-hosted app depending on where the data your app needs to access resides and if the user’s permissions need to be considered. If you have been developing apps for SharePoint for awhile now (and even if you haven’t), I strongly encourage you to use Visual Studio 2013 and the Office Developer Tools for Visual Studio 2013 – March 2014 Update.

Host webs and app webs

When dealing with apps for SharePoint, you will become familiar with host webs and app webs:

  • Host web – the SharePoint site to which an app is installed
  • App web – the special isolated site (a unique app web is provisioned for each installation of the app) where the app for SharePoint’s internal components and content, such as lists, content types, workflows, and pages, are deployed

Note that a provider-hosted app is not required to have an app web, and in fact may not need one depending on your business requirements.

Your app will always have Full Control permissions to its app web. However, your app will need to request (and be granted) permissions by the user installing your app in order to access data in the host web. This is handled through the app manifest.

If your app needs to access data in the SharePoint site where it is being installed, you will be working with a host web context of some sort. As you will see, there are actually two different host web context objects, depending on the app authorization policy you choose.

Life made easy, thanks to SharePointContext.cs

When you create a new provider-hosted app in Visual Studio 2013, you have the option to create a new ASP.NET Web Forms or MVC application to serve as your app’s remote web application. If you are using the Office Developer Tools for Visual Studio 2013 – March 2014 Update, you also have the option to convert an existing ASP.NET web application to an app for SharePoint project (really cool!) In either case, you will notice that SharePointContext.cs is added to the remote web application project. This file contains class definitions for SharePointAcsContext and SharePointHighTrustContext, which allow you to create host web and app web context objects based on whether your trust broker is ACS (which it is with Office 365) or if you are on premises. I won’t delve into too much more detail in this post, but Kirk Evans has an outstanding writeup about these classes.

Accessing data in the app web

To access data in the SharePoint app web from your app, use the following pattern:

CSOM (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);

using (var clientContext = spContext.CreateUserClientContextForSPAppWeb())
{
    Web web = clientContext.Web;
    clientContext.Load(web);
    clientContext.ExecuteQuery();

    ListCollection lists = web.Lists;
    clientContext.Load<ListCollection>(lists);
    clientContext.ExecuteQuery();
}

REST (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);
HttpWebRequest listRequest = (HttpWebRequest)HttpWebRequest.Create(spContext.SPAppWebUrl + "/_api/web/lists");
listRequest.Method = "GET";
listRequest.Accept = "application/atom+xml";
listRequest.ContentType = "application/atom+xml;type=entry";
listRequest.Headers.Add("Authorization", "Bearer " + spContext.UserAccessTokenForSPAppWeb);

JSOM

var appweburl = decodeURIComponent(getQueryStringParameter("SPAppWebUrl"));
var clientContext = new SP.ClientContext(appweburl);
var appWeb = clientContext.get_web();
var appWebListColl = appWeb.get_lists();
clientContext.load(appWebListColl);
clientContext.executeQueryAsync(onAppWebGetListSuccess, onError);

Accessing data in the host web

To access data in the SharePoint host web (the SharePoint site where your app is installed) from your app, use the following pattern:

CSOM (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);

using (var clientContext = spContext.CreateUserClientContextForSPHost())
{
    Web web = clientContext.Web;
    clientContext.Load(web);
    clientContext.ExecuteQuery();

    ListCollection lists = web.Lists;
    clientContext.Load<ListCollection>(lists);
    clientContext.ExecuteQuery();
}

REST (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);
HttpWebRequest listRequest = (HttpWebRequest)HttpWebRequest.Create(spContext.SPAppWebUrl + "/_api/web/lists");
listRequest.Method = "GET";
listRequest.Accept = "application/atom+xml";
listRequest.ContentType = "application/atom+xml;type=entry";
listRequest.Headers.Add("Authorization", "Bearer " + spContext.UserAccessTokenForSPHost);

JSOM

var appweburl = decodeURIComponent(getQueryStringParameter("SPAppWebUrl"));
var hostweburl = decodeURIComponent(getQueryStringParameter("SPHostUrl"));
var clientContext = new SP.ClientContext(appweburl);
var factory = new SP.ProxyWebRequestExecutorFactory(appweburl);
clientContext.set_webRequestExecutorFactory(factory);
var appContextSite = new SP.AppContextSite(clientContext, hostweburl);
var hostWeb = appContextSite.get_web();
hostWebListColl = hostWeb.get_lists();
clientContext.load(hostWebListColl);
clientContext.executeQueryAsync(onHostWebGetListSuccess, onJSOMError);

Note that using JSOM, we still need to construct a ClientContext for the app web before we generate an AppContextSite for the host web, made possible through the SP.ProxyWebRequestExecutorFactory.

A note about the app-only authorization policy

By default, authorization checks in the host web succeed only if both the current user and the app have sufficient permissions to perform the action in question, such as reading from or writing to a list. We are reminded that the user’s permissions are taken into account based on the names of the context and access token objects we use in these scenarios: for instance, CreateUserClientContextForSPHost and UserAccessTokenForSPHost. However, your app has the ability to do something akin to running with elevated privileges using the app-only policy for authorization. Also controlled through the app manifest, the app-only policy is useful when an app doesn’t need or want to consider the permissions of the current user. In Visual Studio 2013, you can specify that your app would like to have the ability to use the app-only policy by checking this box in the AppManifest.xml editor, on the Permissions tab:

apponly

That being said, just because your app is granted this permission does not mean that you can use the same host web context or access token as before to automatically leverage it. To access data from the SharePoint host web (taking only your app’s permissions into account and ignoring the current user’s permissions) from your app, use the following pattern:

CSOM (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);

using (var clientContext = spContext.CreateAppOnlyClientContextForSPHost())
{
    Web web = clientContext.Web;
    clientContext.Load(web);
    clientContext.ExecuteQuery();

    ListCollection lists = web.Lists;
    clientContext.Load<ListCollection>(lists);
    clientContext.ExecuteQuery();
}

REST (C#)

var spContext = SharePointContextProvider.Current.GetSharePointContext(Context);
HttpWebRequest listRequest = (HttpWebRequest)HttpWebRequest.Create(spContext.SPAppWebUrl + "/_api/web/lists");
listRequest.Method = "GET";
listRequest.Accept = "application/atom+xml";
listRequest.ContentType = "application/atom+xml;type=entry";
listRequest.Headers.Add("Authorization", "Bearer " + spContext.AppOnlyAccessTokenForSPHost);

Remember that in order to use the app-only policy, your app must request and be granted this permission by the site owner who installs your app. Also note that there is no JSOM example using the app-only policy because apps that do not make OAuth authenticated calls (such as apps that are only JavaScript running in the app web) cannot use the app-only policy.

As you can see, the code you write in each of the above scenarios (accessing data in the app web, host web, or using the app-only authorization policy) is identical except for the method or property you use from the SharePointContext class to get the appropriate context or access token. Understanding these subtle differences is vitally important when making sure your app has the ability to access and manipulate the SharePoint data it needs.

Using JSOM to write (small) files to a SharePoint 2013 document library

Update 6/9/2014: The code for this app is available for download here.

Update 3/3/2013: Problem solved! The culprit: “Protected View” in Office 2013. Thanks to this blog post from Tobias Lekman, after disabling protected view in each of the Office 2013 applications (I had to update these settings separately in Word, Excel, and PowerPoint), every document I uploaded using the code below opened without any errors on my CloudShare VM.

Update 2/16/2013: The issue I had opening files uploaded into SharePoint described below may not be an issue after all. As it turns out, the CloudShare VM where I was doing my development and testing is generating the same error for ANY document I upload, even “the old fashioned way.” I accessed the same document library from a remote client and the same products that generated errors when being uploaded using the code above opened just fine, without any issues or errors. I will update this post as I learn more about the root cause of this problem.

A recent post on Yammer lamented the lack of examples in the SharePoint 2013 API documentation that use the JavaScript Object Model (JSOM) to do anything more than create a very basic text file in a SharePoint document library.

After lots of digging and a fair amount of trial and error, I now understand why that is the case.

The use case seems simple enough: allow the user to select a file from his or her local machine using an HTML DOM FileUpload object on a form, then use JSOM to upload this file into a document library. It’s certainly easy enough to do using the Client Script Object Model (CSOM). As it turns out, there are a couple of very good reasons why your document upload capability (whether you package it into an app for SharePoint or something else) should NOT leverage JSOM:

  • Per MSDN, you can only work with files up to 1.5 MB when using JSOM. It is recommended that you use REST to deal with larger files. (Incidentally, there is a comment in the article on using the REST endpoints that reads “See How to: Complete basic operations using JavaScript library code in SharePoint 2013 for a code example that shows you how to upload a binary file that is smaller than 1.5 MB by using the SharePoint 2013 Javascript object model.” Unfortunately, the only code example in that article creates a very rudimentary plain text file.)
  • Unless your browser supports the File APIs introduced in HTML5 (specifically the FileReader API), you are out of luck. As a general rule, browsers will block attempts by JavaScript to access and read files from the local file system for security reasons. If you are using IE, only version 10 supports the FileReader API.

Although I was somewhat discouraged by this news, I was determined to develop an app for SharePoint 2013 that presented a simple file upload control to the user and stored the file in a document library (as long as it was smaller than 1.5 MB, of course). I figured as long as I could save Office documents to the library (i.e., more than a simple plain text file), I would have succeeded.

To accomplish this, I knew I would need to make use of the HTML5 FileReader API. (Because of that, I also knew I would need to test this solution using IE 10, Firefox, or Chrome!) Based on the MSDN documentation, I knew I would be setting the contents of the file by using a new SP.Base64EncodedByteArray. The FileReader API exposes three methods for reading the contents of a file:

  1. readAsText() – this method reads the plain text contents of a file, but does not properly handle binary files.
  2. readAsArrayBuffer() – this seemed to be the most promising option, but no matter how I tried to cast the contents of the ArrayBuffer to a Base64-encoded byte array, I was not able to successfully reproduce a file from the file system in a document library. If anyone out there has any suggestions that might enable readAsArrayBuffer() to work, please let me know in the comments!
  3. readAsDataURL() – this method returns the contents of the file using the Data URI scheme. Thanks to a handy utility method I found here, I can convert this Base64-encoded string into a JavaScript Uint8Array and use that to populate the SP.Base64EncodedByteArray that the JSOM expects.

Here is the JavaScript I ended up using:

$(document).ready(function ()
{
    // Get the URI decoded host web URL
    // We will use this to get a context here to write data
    hostweburl = decodeURIComponent(getQueryStringParameter("SPHostUrl"));
});

function CreateFile()
{
    // Ensure the HTML5 FileReader API is supported
    if (window.FileReader)
    {
        input = document.getElementById("fileinput");
        if (input)
        {
            file = input.files[0];
            fr = new FileReader();
            fr.onload = receivedBinary;
            fr.readAsDataURL(file);
        }
    }
    else
    {
        alert("The HTML5 FileSystem APIs are not fully supported in this browser.");
    }
}

// Callback function for onload event of FileReader
function receivedBinary()
{
    // Get the ClientContext for the app web
    clientContext = new SP.ClientContext.get_current();
    // Use the host web URL to get a parent context - this allows us to get data from the parent
    parentCtx = new SP.AppContextSite(clientContext, hostweburl);
    parentWeb = parentCtx.get_web();
    parentList = parentWeb.get_lists().getByTitle("Documents");

    fileCreateInfo = new SP.FileCreationInformation();
    fileCreateInfo.set_url(file.name);
    fileCreateInfo.set_overwrite(true);
    fileCreateInfo.set_content(new SP.Base64EncodedByteArray());

    // Read the binary contents of the base 64 data URL into a Uint8Array
    // Append the contents of this array to the SP.FileCreationInformation
    var arr = convertDataURIToBinary(this.result);
    for (var i = 0; i < arr.length; ++i)
    {
        fileCreateInfo.get_content().append(arr[i]);
    }

    // Upload the file to the root folder of the document library
    this.newFile = parentList.get_rootFolder().get_files().add(fileCreateInfo);

    clientContext.load(this.newFile);
    clientContext.executeQueryAsync(onSuccess, onFailure);
}

function onSuccess()
{
    // File successfully uploaded
    alert("Success!");
}

function onFailure()
{
    // Error occurred
    alert("Request failed: " + arguments[1].get_message());
}

// Utility function to remove base64 URL prefix and store base64-encoded string in a Uint8Array
// Courtesy: https://gist.github.com/borismus/1032746
function convertDataURIToBinary(dataURI)
{
    var BASE64_MARKER = ';base64,';
    var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
    var base64 = dataURI.substring(base64Index);
    var raw = window.atob(base64);
    var rawLength = raw.length;
    var array = new Uint8Array(new ArrayBuffer(rawLength));

    for (i = 0; i < rawLength; i++)
    {
        array[i] = raw.charCodeAt(i);
    }
    return array;
}

This code works!–mostly. In my environment, Excel Services was able to successfully open my basic test spreadsheet in the browser:

excelservices

I could also use the Download a Copy option to save local copies and successfully open files of any type:

dlcopy pptx

For a simple Word document, though, I was unable to click the link from the document library and have it open successfully in Word. Instead, Word reported an error when trying to open the document:

If you receive the error shown below when opening a document from SharePoint, it is due to “Protected View” in Office 2013. To disable Protected View, follow the steps outlined here.

worderr

Regardless of your Office 2013 Protected View settings, the Download a Copy option will open the document without the annoying error message.

saveas

testdoc

Presenting Apps for SharePoint 2013 at @CapAreaSP!

Thanks to everyone who braved the elements (e.g., the winter storm that never materialized) to see me present “Get Started with Apps for SharePoint 2013!” at the Capital Area .NET SharePoint Special Interest Group at Knowlogy in Tysons Corner tonight! I was very impressed with the level of audience participation and interaction during the session. Feel free to reach out to me anytime (either in the comments here or on Twitter) if you have any questions or just want to chat about Apps for SharePoint 2013!

Slides and code from my session

Code for the sample app I demonstrated can be downloaded here.

To learn more about when and when not to use the new app model for SharePoint 2013 development, attend next month’s @CapAreaSP session featuring Alex Randall from Microsoft!

Kicking off @RestonSPUG with Apps for SharePoint 2013!

Thanks to everyone who joined me, Scott Hoag, and Dan Usher as we kicked off the new Reston SharePoint User Group today! The group will meet around lunchtime at the Microsoft office in Reston on the first Monday of each month. The group’s meeting time and location should be ideal for the many SharePoint professionals in the Reston area who can’t make it to the various regional SPUGs that traditionally meet in the evenings.

I had the privilege of being the group’s inaugural speaker today, presenting my session “Get Started with Apps for SharePoint 2013!” Much like the session I led last weekend at SPS Virginia Beach, there was great audience participation and interaction. There were a couple of great questions in particular that I wanted to address further here:

If an app requests the Write right at the List scope, is there any way I can install or configure multiple instances of the app to grant it write permissions against two or more lists on a site?

This was an interesting question because the guidance I gave last weekend was that in order to grant the Write right against multiple lists, your app must request permissions at the next highest scope (in the case of this example, that would be the Web scope). However, it is not possible to install multiple instances of an app for SharePoint to the same site. The UI will not show the app in the “Apps you can add” section for that site once an instance of that app is already installed. I even tried forcing my around that by using PowerShell, but was forbidden:

powershell

What is the relationship between OAuth and tokens with apps for SharePoint? Is there a token for the user as well as one for the app that factors into authorization decisions?

The OAuth protocol is used to authenticate and authorize apps for SharePoint. Authentication and authorization of users and apps in SharePoint 2013 is a very advanced subject with many potentially complicated scenarios and unique considerations. To learn more about these topics, I strongly encourage you to read the following articles on MSDN:

Slides and code from my session

Code for the sample app I demonstrated can be downloaded here.

Thanks again to everyone who was part of today’s Reston SPUG event. I look forward to working with the group much more in the future!