Installing a Let’s Encrypt SSL certificate on Winhost with only a Windows 10 workstation

Author’s note: We will return to our regularly scheduled SharePoint programming soon, but I wanted to share this story for anyone in my shoes who wants to create and install a free Let’s Encrypt SSL certificate in a cPanel/shared hosting environment such as Winhost without direct server access. This is the post I wish I could have found when I started on this journey.

I have hosted this blog on Winhost since 2014. I am a huge fan of their affordable, reliable web hosting and friendly, responsive customer support. However, I am not a fan of paying $39-$49 (or more) per year just to have a simple SSL certificate for a personal blog site.

Let’s Encrypt is a free and automated Certificate Authority (CA) that has major community backing. I am an ardent supporter of their mission and believe we should do everything possible to support “HTTPS everywhere.” Many web hosting providers integrate Let’s Encrypt support right out of the box and if yours is one of them, you need not read any further. However, if you are using a shared hosting provider with a cPanel such as Winhost that is somewhere between lukewarm and slightly hostile toward Let’s Encrypt, do not fear. It is still possible to create and install a FREE SSL certificate for your site.

I would like to add that despite the “slightly hostile” Winhost blog post linked above, Winhost support was extremely helpful with a couple of issues I had during this process. If they were upset with me for not buying my SSL certificate through them, they didn’t let it show. 🙂

Certificate Signing Request (CSR)

When requesting an SSL certificate, you must specify a Certificate Signing Request (CSR). I don’t pretend to fully understand how all of this works, but through this process I have learned that the CSR is based on a public/private key pair generated on the server. Therefore, when using a hosting provider like Winhost you must allow them to generate the CSR and not attempt to generate one yourself (unfortunately, this rules out using alternative methods for getting a free SSL certificate such as gethttpsforfree.com, which requires you to have access to the private key).

Within your cPanel, navigate to the SSL Manager and click Generate New CSR. Enter the domain name you wish to protect in the Common Name field. After filling in all the fields, press Submit CSR Information.

This will produce output like what you see below:

Copy this text into a file on your local workstation. I called mine domain.csr. You will need to reference this file when submitting your certificate request.

Certbot

Let’s Encrypt recommends the use of Certbot to get your certificate. Certbot is only available for UNIX-like operating systems (e.g., Linux and MacOS). If you are running Windows 10, however, you can enable and run a Bash shell on Ubuntu 16.04!

Certbot is used to request the certificate and is especially useful when it can be run directly on the web server. Obviously I could not do this, but fortunately Certbot also offers a “manual mode” that lets you specify the CSR to use when submitting your request and prove ownership of your domain without running it directly on the server where the certificate is to be installed.

After you have followed the steps to get Bash on Windows 10, launch Bash on Ubuntu on Windows and run the following commands to install Certbot:

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot

Once this is complete, you can begin the process of requesting your certificate. The command to run is:

$ certbot certonly --manual --csr <path to domain.csr file>

NOTE: Your Windows C: drive is mounted as /mnt/c in Ubuntu.

You must consent to your IP being logged in association with this certificate request.

Once that is done, Certbot asks that you prove ownership of the domain for which you are requesting a certificate by serving a specific string of plain-text content from a URL in the /.well-known/acme-challenge subdirectory of your domain.

To accomplish this, I created a plain-text file in Notepad and named it according to the URL specified. I pasted in the string of content that is supposed to be served from that URL and uploaded it to the appropriate directory of my web site using FTP. Before pressing Enter, I navigated to the URL to ensure the appropriate content was being served, but I received the following error:

Because the file has no extension and is being served out from IIS, I also need to create a new web.config file in this directory, telling IIS to treat this extensionless file as plain text when serving it out:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <staticContent>
            <mimeMap fileExtension="." mimeType="text/plain" />
        </staticContent>
    </system.webServer>
</configuration>

After saving and uploading the web.config file, I confirmed that I was serving out the correct content from the specified URL. I returned to my Bash shell and pressed Enter.

Now that you have proven ownership of your domain, Certbot will request the certificate for your domain and write it out to a .pem file on your workstation.

We can now copy the contents of this file (in this case, 0000_cert.prm) and paste it in our cPanel where it requests the SSL certificate:

Press Submit Certificate. It took about 20 seconds for me, and you won’t see any fancy success or confirmation messages, but after you are redirected away from this page you can navigate to your site and see that it is secured with the Let’s Encrypt certificate.

Caveats

There are some caveats worth pointing out below that may not make this the best solution for you.

– At least for Winhost, the CSR you generate can only contain a single Common Name. Unfortunately, you cannot get a certificate that protects both yourdomain.com and yourotherdomain.com, or even yourdomain.com and www.yourdomain.com.
– The certificate expires every 90 days, so you will need to repeat this process to renew the certificate fairly often. This is worth the cost savings, at least for me.

Despite these caveats, I am thrilled to have my free SSL certificate from Let’s Encrypt protecting my Winhost-hosted site and I hope these steps are useful for someone else.

One final author’s note: I am a SharePoint developer by trade and am far from an expert when it comes to web servers and SSL. If I have misstated anything in this post, please leave a correction in the comments and I will update this post. In addition to hopefully being useful, I want to ensure I am being accurate as well. Thanks!

Tooling up for SharePoint Framework

Over a year ago, Microsoft first announced the SharePoint Framework (SPFx) at the Future of SharePoint event. Accompanying the introduction was this blog post describing the SharePoint Framework as “an open and connected platform” that represents the next step in a natural evolution toward client-side development and customization on top of SharePoint. The SharePoint Framework allows developers to use modern web paradigms and JavaScript frameworks and moves away from the more Microsoft-specific legacy server-side customizations of the past.

The SharePoint Framework finally reached GA in February 2017 and is currently available in SharePoint Online. SharePoint Framework support will be available to SharePoint 2016 on-premises users with the release of Feature Pack 2 later this year.

Getting started with the new developer toolchain

Perhaps the largest barrier to entry when it comes to getting started with development using the SharePoint Framework is the new developer toolchain, which bears very little resemblance to any SharePoint development tools you may have used in the past. Gone are the days of just cracking open Visual Studio to build a SharePoint solution (although Eric Shupps has just announced the SharePoint Framework Extension for Visual Studio, so definitely check that out if you want to continue leveraging Visual Studio as part of your development workflow).

In exchange for the added complexity introduced by this new toolchain we get a set of modern, open-source, cross-platform tools and technologies that finally levels the playing field between SharePoint development and modern web development (try leveraging React or Angular in a Script Editor Web Part!) and opens the door for developers on Linux and Mac OS to build SharePoint customizations. It also eliminates the need for a local SharePoint farm on the developer workstation for on-premises development. Moving forward, the SharePoint Framework is your only option for building custom web parts that can be added to the new modern page experiences.

Even if you decide to leverage the SharePoint Framework Extension for Visual Studio, you will still need to install–and have a basic understanding of–all of the prerequisite tools and technologies that comprise this developer toolchain, which include:

Tool Purpose
Node.js (LTS version) JavaScript runtime; most of the other tools in the toolchain are built on Node.js
npm Package and dependency management
Yeoman Scaffolding engine for new projects
TypeScript Typed superset of JavaScript; transpiles to plain JavaScript
Gulp Extensible build task runner

 
My Pluralsight course

Feeling overwhelmed? I recently published a Pluralsight course that contains short, easy-to-digest modules and demos, each introducing a different item in the toolchain: Tooling up for SharePoint Framework. The course is geared toward “newbies” with little to no prior experience using any of these new tools and technologies. If you follow along with the course, within about an hour you will be able to stand up a new, fully-configured SharePoint Framework development workstation and create, build, and test your first client-side web part. The clip below is a demo from the course showing how to use the Workbench that is included with the SharePoint Framework to test customizations locally without having to deploy anything to SharePoint (another advantage over “legacy” SharePoint development).

What the future holds

Client-side web parts are the first of what will ultimately be many types of JavaScript customizations supported by the SharePoint Framework. According to this blog post from the Office Team, we can expect to see support for the following customizations coming to the SharePoint Framework soon:

  • Web Part Connections that allow client-side web parts to communicate and exchange information with each other
  • ApplicationCustomizers to embed JavaScript on pages within SharePoint sites
  • FieldCustomizers to update the way data is displayed in a list view, similar to JSLink
  • CommandSets to add custom ECB actions, toolbar buttons, etc.

Finally: SPFx vs. the Add-in Model

As the SharePoint Framework continues to evolve and mature, you may encounter scenarios where you cannot use it to achieve a particular objective. In these cases (and many other cases, such as those where server-side code is still necessary), the Add-in model is still viable and fully supported. Microsoft intends for the SharePoint Framework to be complementary to the Add-in model, not a replacement for it. If you’re interested in learning more about updating your legacy SharePoint customizations to take advantage of the Add-in model, please check out my blog post and Pluralsight course.

Slides from today’s Reston SPUG presentation

Thanks to everyone who came out to see me speak at the Reston SharePoint User Group today! Today was my fourth (!) time speaking at Reston SPUG and I was so excited to chat with everyone who had such great questions about updating legacy SharePoint customizations to the add-in model.

Special thanks to Kurt Greening and Dan Usher for all the hard work they put in to running this outstanding group, to LaBrina Loving and Microsoft for hosting us, and an additional shout out to Kurt for saving the day with his 4G mobile hotspot–giving me the internet access I desperately needed for all of my demos!

There was some discussion around customizing the new “modern” team sites and experiences in SharePoint Online. I wanted to share this article on customizing modern team sites if you have not seen it already. I mentioned that the SharePoint Framework provides a viable option for updating legacy custom web parts above and beyond what can be accomplished using the add-in model. As Vesa Juvonen explains in this article, SharePoint Framework will also eventually provide a “more controlled way” to embed JavaScript customizations on modern pages, so please keep an eye on that as it continues to evolve. The good news is that you can still use the PnP Provisioning Engine with modern team sites!

Here are the slides from my presentation today. Feel free to leave any questions you may have in the comments. Thanks again to everyone who came out and spent some time with me at Reston SPUG today!

Updating Legacy SharePoint Customizations to the Add-in Model

If you follow this blog, you have probably read many of my posts through the years dedicated to the Add-in (formerly app) model for SharePoint 2013, 2016, and SharePoint Online. Microsoft introduced the Add-in model in SharePoint 2013 to address the various shortcomings associated with running custom code on the SharePoint server — most notably, of course, that custom code cannot be deployed to SharePoint Online in Office 365. SharePoint-hosted Add-ins are great for simple functionality that can be achieved with 100% client-side solutions (HTML, CSS, and JavaScript), while provider-hosted Add-ins allow you to run custom server-side code in Azure or anywhere other than the SharePoint server.

Using the Add-in model is a no-brainer for most new development efforts, especially if you are developing for SharePoint Online and still need to write server-side code. (At the time of this writing, the new SharePoint Framework is still in Developer Preview but is definitely worth adding to your repertoire for client-side SharePoint development in the years to come!)

But what about your existing solutions? Many organizations using SharePoint maintain large portfolios of custom code solutions that leverage different legacy development approaches advocated by Microsoft through the years, and most of the custom code developed for earlier versions of SharePoint cannot be directly migrated to the cloud. If your organization decided to move to the cloud tomorrow, where would you begin?

Taking inventory of your customizations

I have found that most legacy SharePoint customizations can be placed into the following categories:

  • User interface enhancements (master page, branding, themes, custom web parts)
  • Declarative items (XML for custom site columns, list instances, and content types)
  • Timer jobs (and other administrative extensions to the platform) running in SharePoint Central Administration on-premises

These customizations may take the form of farm or sandboxed solutions that depend on SharePoint’s legacy feature framework. In the case of some user interface customizations, custom HTML markup, CSS, or even JavaScript might be embedded directly within a site’s master page file. For the most part, these are situations we should try to avoid as much as possible when migrating our customizations to the cloud for the following reasons:

  • Farm solutions cannot be deployed to SharePoint Online.
  • Sandboxed solutions that contain managed code can no longer be deployed to SharePoint Online (note that sandboxed solutions contain a managed code assembly by default, even if the solution does not contain any managed code).
  • Custom master pages, while still supported in SharePoint Online, should be avoided unless absolutely necessary. (Microsoft has a history of making significant changes to the default master page in SharePoint Online, such as the addition of the app launcher/waffle/Hollywood Squares in the upper left corner. Sites with heavily customized master pages may not always have access to these changes as they are introduced in the future.)

The role of the Add-in model

When migrating these customizations, keep in mind that you will not necessarily be building and deploying a SharePoint Add-in to replace each customization. In fact, in many cases, you will simply be leveraging techniques from the Add-in model. These include paradigms that were introduced and/or popularized with the introduction of the Add-in model:

  • Using client-side script wherever possible
  • Remote provisioning of assets (everything from CSS/JavaScript files to site columns, content types, etc.) using the .NET Managed client object model (CSOM) or PowerShell
  • Running server-side code outside the SharePoint server that communicates with SharePoint via its APIs

In some cases, all you may need to do is build and deploy a “throwaway” provider-hosted Add-in within a local development environment that performs some one-time remote provisioning tasks, but is not intended to be accessed by end users and can be removed after its work is done.

Add-in model techniques enable us to make our SharePoint customizations with a “lighter touch.” By lighter touch, I am referring to a lighter footprint and/or impact on the SharePoint server. As we decrease the burden on the SharePoint server, we enable SharePoint to run more reliably and efficiently while also making it easier to administer and upgrade (at least on-premises…yes, these techniques all work on-prem as well!) To better explain this concept, I put together the table below contrasting legacy SharePoint development approaches with their modern lighter touch equivalents:

Legacy Approach Lighter Touch
Farm solutions with custom code running in the SharePoint IIS worker process Provider-hosted Add-in with custom code running outside of SharePoint
Farm or sandboxed solutions that deploy declarative artifacts such as site columns, content types, and list instances Remote provisioning these artifacts from a provider-hosted Add-in using CSOM
Custom master pages with embedded markup, styles, and script Remote provisioning of branding assets, embedding of JavaScript UI customizations using ScriptLink custom actions
Custom timer jobs running in Central Administration using the SharePoint server object model Remote timer jobs running as Windows scheduled tasks or Azure WebJobs using CSOM

 
SharePoint Developer PnP: here to help!

Because the Add-in model represents such a strong departure from the way things were done in the past with full-trust code in SharePoint, Microsoft started the SharePoint Developer Patterns and Practices (or PnP) initiative to assist developers with transforming their existing solutions to be cloud-ready as they migrate to SharePoint Online in Office 365.

The PnP team maintains several Github repositories that include reusable, production-ready components, templates, and solution starters that demonstrate the preferred modern approach to SharePoint development — making customizations with a lighter touch wherever possible.

Eric Overfield just published a great blog post detailing the PnP initiative and how you can get involved.

My Pluralsight course

If you are interested in learning more, I recently published a Pluralsight course with lots of demos: Updating Legacy SharePoint Customizations to the Add-in Model. The course covers the process of migrating a heavily customized on-premises SharePoint site to SharePoint Online from start to finish. The clip below is a demo from the course showing how a legacy timer job customization can be made cloud-ready with the help of the PnP Timer Job Framework and Azure WebJobs.

In this course, you will see me leverage the PnP Core Component to drastically simplify writing .NET Managed CSOM code. I also use several great samples and solution starters from the PnP team, all with the objective of easing the pain associated with migrating legacy SharePoint customizations to the Add-in model.

Speaking at SharePoint Fest DC 2017!

This April, I will be joining SharePoint experts from around the world at SharePoint Fest DC 2017! I am very excited to be debuting a new session this year:

DEV 200 – Updating Legacy SharePoint Customizations to the Add-in Model

Many organizations using SharePoint maintain large portfolios of custom code solutions that leverage different legacy development approaches advocated by Microsoft through the years, and most of the custom code developed for earlier versions of SharePoint cannot be directly migrated to the cloud. In this session, you will learn how the SharePoint Add-in model allows you to develop maintainable customizations that have traditionally been accomplished using custom server-side code. You will learn how to categorize the types of user interface, declarative, and timer job customizations you may have performed with farm and sandboxed solutions in the past and understand how to approach developing each of them using techniques from the Add-in model.

If you register now and use code JesseeDC200 at checkout, you can save $200 off the purchase of a Gold, Platinum, or Diamond pass!

This will be my third SharePoint Fest event and my second SharePoint Fest DC. I absolutely love this conference and the DC location is as convenient as it gets for mass transit…just step off the train and you are there! I look forward to seeing you there.

Custom site header and footer using a SharePoint-hosted add-in

In this post, I will show you how to create a custom site header and footer for SharePoint Online/Office 365 that will render on all pages using a SharePoint-hosted add-in. This could be used to post critical alerts or to specify the level of business impact of the content stored within a particular site (e.g., high, medium, or low).

This add-in will accomplish the following, all without a single line of server-side code:

  • Upon installation, deploy 3 JavaScript files to the Style Library of the host web
  • Register 3 user custom actions in the host web to ensure these JavaScript files are loaded on every page
  • Via an add-in part, provide a mechanism for users to enable/disable and set properties of the custom header and footer (e.g., message text, background/text color)
  • Store and retrieve configuration parameters for the site header and footer in the host web’s property bag

Rendering a custom header and footer

We might be tempted to edit the master page directly to add a header and a footer, but this is not considered a best practice (especially in SharePoint Online, where master page changes can be rolled out quite frequently). Instead, our add-in will make use of JavaScript techniques to insert <div> elements at the top and bottom of the page for our header and footer:

  • In SharePoint Online/Office 365, our header <div> (with ID customHeader) will be inserted above the <div> with ID suiteBarTop (for SharePoint 2013 on-premises, you’ll need to update the script to reference ID suiteBar instead).
  • Our footer <div> (with ID customFooter) will be inserted below the <div> with ID s4-bodyContainer.
  • NOTE: The <div> IDs in SharePoint Online are never set in stone and could change at any time. If they do, you will need to update HeaderFooter.js to reference the new <div> IDs.

jQuery makes it easy to create our custom header and footer <div> elements and insert them in the appropriate location within the DOM with its .insertBefore() and .insertAfter() functions following this approach:

$("<div id='customHeader' class='ms-dialogHidden'>HEADER TEXT</div>").insertBefore("#suiteBarTop");
$("<div id='customFooter' class='ms-dialogHidden'>FOOTER TEXT</div>").insertAfter("#s4-bodyContainer");

Making the footer “sticky”

Huge shout-out to Randy Drisgill for his SharePoint 2013 Sticky Footer implementation. The “sticky” footer anchors the footer <div> to the bottom of the page (or the bottom of the browser window if the available screen real estate exceeds the amount of content on a given page). I only needed to make one change to Randy’s StickyFooter.js implementation, and that was to account for the height offset imposed by the addition of our customHeader <div>:

var difference = windowheight - (bodyheight + ribbonheight + footerheight + $("#customHeader").outerHeight());

Provisioning files to the host web via JavaScript

For anyone following the OfficeDev PnP, the concept of remote provisioning to allow add-ins to deploy files to the host web should be a familiar one. However, the PnP examples make use of the .NET Managed CSOM to do this, which is a perfectly valid technique but would require us to develop a provider-hosted add-in (allowing that code to run in Azure or some other web server). Since I wanted to create a SharePoint-hosted add-in, I had to find a way to accomplish this using only JavaScript. Thankfully, I found Chris O’Brien’s post with code showing how to provision files to the host web using JavaScript. You will see that my code is based heavily on the example he provides and provisions the following files from the add-in web to the host web:

  • jQuery-1.9.1.min.js – jQuery
  • HeaderFooter.js – our logic to read the header/footer configuration data from the host web property bag and render the header <div> and footer <div> elements
  • StickyFooter.js – Randy Drisgill’s Sticky Footer implementation for SharePoint 2013 (with the one tweak described above)

Add-in part for setting configuration values

The add-in also includes an add-in part (deployed to and rendered from the add-in web) that allows users to enable/disable the header/footer and set the text and colors:

addinpartWhen the add-in part loads, it uses JavaScript to query the property bag of the host web to see if these settings already exist and if so, prepopulates the values in the form. Once the user has made the necessary changes, clicking Set Values will save the changes back to the property bag (again, using JavaScript).

NOTE: The need for our add-in to read and write data to and from the host web property bag requires us to request the FullControl permission at the Web scope in our add-in manifest (AppManifest.xml):

<AppPermissionRequests>
  <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="FullControl" />
</AppPermissionRequests>

What about MDS?

SharePoint 2013 introduced the Minimal Download Strategy (MDS), which reduces page load time by sending only the differences when users navigate to a new page. While this is a wonderful benefit, it wreaks havoc with solutions that need to manipulate the DOM every time a new page is loaded. In other words, our header and footer may render perfectly on the home page when we first load it, but thanks to MDS when we navigate to the “Documents” library, only part of the page will actually be re-rendered. Our header and footer will not display properly (if at all) when only part of the page is re-rendered to support a new page request.

For much deeper reading on this subject, I encourage you to go through Wictor Wilen’s blog posts introducing the MDS and explaining the correct way to execute JavaScript functions on MDS-enabled sites. My code is based on Wictor’s solution and works properly in scenarios where MDS is enabled and where it is not (e.g., on Publishing sites or any site where the MDS feature has been deactivated).

We handle MDS by calling RegisterModuleInit() with the function we need to execute on every page load in HeaderFooter.js:

if (typeof _spPageContextInfo != "undefined" && _spPageContextInfo != null) {
    // MDS enabled
    RegisterModuleInit(_spPageContextInfo.siteServerRelativeUrl + 'Style Library/Headerfooter.js', DJ.HeaderFooter.Render.initializeHeaderFooter);
}
// Run now on this page (and non-MDS scenarios)
DJ.HeaderFooter.Render.initializeHeaderFooter();

The code

I have posted the code for this add-in to GitHub at the following location:

https://github.com/dannyjessee/SiteHeaderFooter

I encourage you to download it, try it out in your environment, and let me know if you run into any issues with it. My sincere thanks go out to Chris O’Brien, Randy Drisgill, and Wictor Wilen for giving me the building blocks needed to put this add-in together.

Yes, you can still use the AjaxControlToolkit with SharePoint 2013 on-premises

For years, it seems developers have struggled with finding exactly the right set of steps to leverage the AjaxControlToolkit in their SharePoint applications. In SharePoint 2010, you had to download a specific legacy version of the toolkit and make a handful of <SafeControls> entries in your solution manifest (obviously we can only use the AjaxControlToolkit on-premises since it requires deploying a DLL to the Global Assembly Cache). You also had to update your master page to leverage the toolkit’s own ToolkitScriptManager. If you’re looking for a verified set of steps for using the AjaxControlToolkit with SharePoint 2010, see Sohel’s blog post here. It was a lifesaver for me a few years ago.

But what if you’re stuck having to upgrade a solution that depends on the AjaxControlToolkit to SharePoint 2013 today? I found myself in precisely this situation not too long ago. Based on my web searching, there doesn’t appear to be a great deal of interest surrounding this topic anymore, but in case you find yourself in my predicament, here is what to do:

  1. Download the latest version of the AjaxControlToolkit here. At the time of this writing, I have verified these steps with the May 2015 update – v15.1.2.
  2. Run the installer in your development environment.
  3. Find the location where the files were installed and grab a copy of AjaxControlToolkit.dll from the ASP.NET AJAX Control Toolkit\Bin directory. Place this file somewhere you can easily reference it from your SharePoint project. I like to make an “External References” folder at the same level as my .sln file to keep the path references simple.
  4. In Visual Studio, create a new empty SharePoint 2013 project as a Farm Solution.
  5. Add a reference to AjaxControlToolkit.dll by right-clicking References and selecting Add Reference…
    addreference
  6. Browse to the location of AjaxControlToolkit.dll. This will allow you to reference the toolkit controls in your code.
  7. We need to ensure that AjaxControlToolkit.dll is deployed to the GAC with our farm solution. To do this, we must update the package manifest. In Solution Explorer, expand the Package folder and double-click Package.package. Go to the Advanced tab and click Add, then Add Existing Assembly…
    addassembly
  8. In the Source Path area, navigate to the location of AjaxControlToolkit.dll and ensure GlobalAssemblyCache is selected as the Deployment Target. Press Click here to add a new item. Enter AjaxControlToolkit in the Namespace and Assembly Name areas. When complete, the dialog should look like this:
    addassembly2
  9. Deploy your solution. You can verify the AjaxControlToolkit assembly has been deployed to the GAC (C:\Windows\Microsoft.NET\assembly\GAC_MSIL).
  10. Anywhere you would like to leverage the toolkit, ensure you add the following declaration to your page/visual web part user control:
    <%@ Register Assembly="AjaxControlToolkit, Version=15.1.2.0, Culture=neutral, PublicKeyToken=28f01b0e84b6d53e" Namespace="AjaxControlToolkit" TagPrefix="asp" %>
    

You do NOT need to update any master pages to use the ToolkitScriptManager because the v15.1 update of the AjaxControlToolkit removed it in favor of the standard ScriptManager. Rejoice!

wp2Although the AjaxControlToolkit may be waning in popularity among modern SharePoint developers, it still contains some pretty useful controls (CalendarExtender, ColorPickerExtender, etc.) For any developers who need to update a legacy solution leveraging the AjaxControlToolkit for SharePoint 2013 without the budget or resources to adopt a different approach, know that it can be done without many of the headaches that existed in the past.

What’s in a name? The challenges posed by renaming SharePoint “apps” to “Add-ins”

As you probably know by now, the following message started appearing on all MSDN pages related to apps for SharePoint in early May (the wording has recently been tweaked; the latest version of the message is shown below):

apps1The Note above links to New name for apps for Office and SharePoint.

My initial reaction to this news was something to the effect of, “OK, that’s interesting but no biggie. I’ll just make a mental note to say Add-in now instead of app (oh, and do a global search and replace in all my slide decks)…”

The more I thought about it, though, this simple name change was going to pose many challenges for customers and consultants alike. I have been speaking about “apps for SharePoint” for almost three years now and one of the most trivial, yet biggest stumbling blocks seemed to be:

“So when I want to add a custom list to my site, I go to ‘Add an app’ now?”
“Well, yes, but doesn’t that at least make more sense than going to ‘More Options…’?”
“I guess. But do I need to configure the app domain and do all that other stuff you talked about just to add a list now?
“No, you only need to do that if you will be connecting to the Office Store or if your developers will be leveraging the app model.”

It’s already been more than a little awkward as I feel the need to interject whenever someone says something about “apps for SharePoint” or the “SharePoint app model.”

Along with a few other developers and consultants, I shared my concerns about the name change in semi-public discussions on Facebook and Yammer. As a result, Jeremy Thake was kind enough to publish a blog post containing the Office 365 ‘App Model’ rename cheat sheet. This post is fantastic and gives some more background about why the name change was necessary. Jeremy has certainly taken a beating from some of the folks out there who are frustrated, confused, and/or bewildered by the name change, and I applaud his efforts in listening to our concerns and providing this level of information to us.

Having spent the better part of the last three years educating our customers and development teams about the “app model,” we must now update our vocabulary to ensure we are speaking with accuracy and precision about the many facets of custom development for SharePoint and Office 365. The “cheat sheet” is a good start, but it omits at least a few relevant “app-isms” that I have listed below by what I assume will be their new names:

    • Add-in prefix
    • Add-in domain
    • Add-in catalog
    • The file extension of the generated Add-in package (currently *.app)
    • Add-in manifest (AppManifest.xml)
    • Add-in principal
    • Add-in-only calls to SharePoint
    • Add-in authorization policy types (Add-in + User, Add-in-only)
    • Add-in Management Service Application
    • Add-in registration and management pages (currently /_layouts/15/AppRegNew.aspx, AppInv.aspx, AppPrincipals.aspx)
    • Countless other Add-in-related PowerShell cmdlets and APIs (NOTE: word on the street is that the app-related names referenced in the APIs will not be changing)

After three years, there is obviously a great deal of sample code and documentation out there that references the “app model.” The Office 365 PnP repository is no exception:

People who are willing to invest the time and energy to ensure that documentation and tooling around SharePoint Add-ins is up-to-date with the proper naming should be lauded, as there will no doubt be a great deal of confusion among customers and developers as they figure out what terminology to use and which examples are relevant when searching for accurate, up-to-date information about building SharePoint Add-ins. It appears that folks who are fortunate enough to be under NDA with Microsoft also have a better sense of why this name change was necessary, so let’s hope that information makes its way out to the rest of the community soon so we can answer the inevitable questions from our customers about “Why?” or “What’s the difference between an app and an Add-in?” or “Now where do I go to add a custom list to my site?”

(Yes, you guessed it…to add a custom list to your site, you will soon have to go to “Add an Add-in.”)

What are your thoughts on this change? Let’s keep the discussion going. I will be updating this post as we experience the impact of these changes over time.

Getting “Unexpected response data from server” error using SharePoint JSOM?

I was clearing the dust off an old demo for my session on cloud-hosted apps at SharePointFest DC next month when I ran into a problem. As I walked through the demo (which I had given three times before at SPS Events in Boston, Baltimore, and DC), I was getting an error when testing my ability to query for list data in my cloud-hosted app’s app web in my Office 365 developer tenant using JSOM:jsrequestfailedKnowing this script had worked for months in the past, I turned to the Internet. Sure enough, others appeared to be experiencing the same problem. One app developer even worked around the problem by abandoning the use of JSOM altogether in favor of the REST APIs.

What’s the problem here?

In a cloud-hosted app, the app’s business logic is deployed to a remote web (separate from its app web, which resides in SharePoint). This remote web is likely on a different domain (and is definitely on a different domain if you are using SharePoint Online). Querying for data that resides in the app web from a script running in your remote web requires the use of cross-domain JavaScript calls, which are generally blocked for security reasons. To work around this requires the use of the JavaScript Cross-Domain Library (sp.requestexecutor.js).

My demo script looked like this:

var clientContext = new SP.ClientContext(appweburl);
var appWeb = clientContext.get_web();

appWebList = appWeb.get_lists().getByTitle(listname);
var camlQuery = new SP.CamlQuery();
camlQuery.set_viewXml("<View><RowLimit>50</RowLimit></View>");
appWebListItems = appWebList.getItems(camlQuery);
clientContext.load(appWebList);
clientContext.load(appWebListItems);
clientContext.executeQueryAsync(onAppWebGetListDataSuccess, onJSOMError);

Once I initialized the ProxyWebRequestExecutorFactory object and set it as the factory of my ClientContext object, everything worked as expected. (NOTE: The script in my demo that queried for data in the host web was already making proper use of the cross-domain library.)

Here is the proper script to initialize the ClientContext and associate the appropriate ProxyWebRequestExecutorFactory with it:

var clientContext = new SP.ClientContext(appweburl);
var factory = new SP.ProxyWebRequestExecutorFactory(appweburl);
clientContext.set_webRequestExecutorFactory(factory);
var appWeb = clientContext.get_web();

// Remaining code is the same as above

Conclusion

I’m not sure exactly what changed that would cause this problem to start occurring. Common sense dictates that the ProxyWebRequestExecutorFactory should be needed for all cross-domain calls, whether they be to the host web or the app web. I’m less concerned about that than why the old code ever worked (that did not make use of it). Nonetheless, if you find yourself in a situation where a script that definitely worked in the past now fails with an “Unexpected response data from server” error when using the JSOM, ensure you are making proper use of the cross-domain library as well.

Move list items with attachments between SharePoint farms: Access to the Rescue!

Update 8/27/2017: I updated the title of this post to avoid confusion about which versions of SharePoint and Access can be used. I also added a screenshot to this post showing where to find the option to import data from a SharePoint list using Access 2016 (it’s buried a little deeper in the UI than it was in Access 2013). I also verified that all the other steps in this article remain the same for Access 2016. One user posted an issue in the comments with the exported list displaying a “something went wrong” error in either classic or modern view, but I was unable to reproduce this with the list I was using to test. There are some documented issues (see section towards bottom of article) that will force a list to render using the classic experience, usually due to the data types of one or more fields in the list.

Original post appears below.

I was recently faced with the unenviable task of moving the full contents of a custom SharePoint list with about 60 columns (including attachments totaling about 400 MB) between a hosted instance of SharePoint 2007 and SharePoint Online. While there is no shortage of options for getting data from a list in one environment into another, I was constrained by the fact that this was a cloud-hosted instance of SharePoint 2007 where I had no server or administrator-level access. I went through a mental checklist of my options:

  • Save the list as a template with contents – Nope. Way too big with those 400 MB worth of attachments.
  • Export the list to Excel – Nope. Only includes the fields defined in the list view and does not include attachments.
  • Create an empty version of the list in the destination environment, then use datasheet/quick view to copy data between the two lists – Nope. Again, the attachment problem (not to mention how tedious it would be to create identical views that would support datasheet/quick view in both environments).
  • Write code to programmatically handle the export from SharePoint 2007 – Eh. With no server-level access in the cloud-hosted SharePoint 2007 environment, writing full trust code with the server object model was off the table. Theoretically I could have written code that leveraged SharePoint 2007’s .asmx web services, but I only wanted to go there as a last resort.

Then, as if by providence, I accidentally clicked to open Access 2013 when I actually meant to open Excel. I knew SharePoint 2013 had the hooks to import and export data using Access, but could I somehow use Access to pull the data out of SharePoint 2007? I clicked to create a new Blank Desktop Database and started my journey:

blankdesktopdb

I gave the database a name and pressed Create:

newdb1

Time to start poking around. I went to the External Data tab, and clicked More to reveal SharePoint List in the “Import & Link” section of the ribbon:

importsp1

Importing a SharePoint list using Access 2013.

In Access 2016, select New Data Source from the Import section of the ribbon, then choose From Online Services and then SharePoint List:

Importing a SharePoint list using Access 2016.

I entered the URL to my SharePoint 2007 site and chose to Import the source data into a new table in the current database.

importsp2

I selected the list I wanted to export and clicked OK.

importsp3

Access then proceeded to lock up my computer for several minutes as it imported the data, but it was worth the wait. Keep in mind that although logically we are exporting the data from SharePoint 2007, from Access’ perspective we are importing this data into Access. From here, we will export this data to a list in our SharePoint Online site.

importsp4

When the import completed, I clicked Close and noticed in the “All Access Objects” view in the left pane, my database now contained a table named after the SharePoint list I just brought over. I double-clicked it and sure enough, all the data was there including the list item attachments!

attachments

Stunned by my good fortune, I double-clicked some of these cells to confirm the attachments were actually there. They were!

attachments2

Time to get this data into SharePoint 2013. I went back to the External Data tab, and clicked More to reveal SharePoint List in the “Export” section of the ribbon:

exportsp1

Much like the import wizard earlier, the export wizard allows me to connect to a SharePoint site (I chose my SharePoint Online site this time) and specify a name for the new list.

exportsp2

I pressed OK and allowed Access to do its thing (it took about 15 minutes). When it was done, my browser opened to the newly imported list in SharePoint Online. Not only was the list schema perfect (if you’ve ever imported a spreadsheet to create a new list, you know how frustrating it can be to have all of your Choice columns converted to Single line of text), but all the attachments to the list items were there as well!

You will notice a few extra columns are created as part of the process:

  • Encoded Absolute URL
  • Item Type
  • Path
  • URL Path
  • Workflow Instance ID
  • Comments
  • File Type
  • _OldID

You can safely remove these from your new list if you do not want or need them.

finalexport

Mission accomplished! Thanks, Access 2013!