SharePoint Online Continuous Delivery with Visual Studio Team Services (part 1)

It's time for full cloud DevOps with Office365, Team Services and Azure!
In this guide I will cover how to setup a continuous delivery for SharePoint Online from ground up using Visual Studio Team Services (former Visual Studio Online).

I'm about to cover several topics in order to get to the point, to simplify the writing (and maybe the reading) I will split this into several parts. I will come back and update links as posts are made.

Link to post 2/3 : https://sharepointologic.blogspot.fr/2017/02/sharepoint-online-continuous-delivery.html
Link to post 3/3 : https://sharepointologic.blogspot.fr/2017/03/sharepoint-online-continuous-delivery.html

The context of all this

In order to give you a better view I'll try and build this guide on a real world use case.
I'm making a custom portal with SharePoint Online, this is the "traditional" target release pipeline I want to achieve to try and maintain quality over successive releases:

Nothing crazy here, the devs and the project manager use the integration (INT) platform for integrity check and the customer will proceed with testing on the UAT.
I have several developers, they each have a developer site collection:

• https://<tenant>.sharepoint.com/sites/dev-a
• https://<tenant>.sharepoint.com/sites/dev-b
• https://<tenant>.sharepoint.com/sites/dev-c

The integration has it's own site collection : https://<tenant>.sharepoint.com/sites/int-portal
So does the "User Acceptance and Testing" site : https://<tenant>.sharepoint.com/sites/uat-portal
And production (you get the idea).
The workflow involves devs building on their onw site. Then the source code is used build a release that is deployed to INT to check for broken code, merge issues and deployment flaws. Once this is cleared the release is deployed to UAT for testers to proceed with validation of business requirements and bug hunting. After UAT testing if the release meets expectations, it can be deployed to production.

Wait this is SharePoint Online, I come from on-premises how do I "package" ?

Well one solution is to use the guidance from SharePoint Patterns & Practice (https://github.com/SharePoint/PnP).
PnP offers PowerShell Commands (https://github.com/SharePoint/PnP-PowerShell), it's flexible and offers a very cool way of provisioning stuff to SharePoint and specially the "PnP Provisioning Schema" (https://github.com/OfficeDev/PnP-provisioning-schema) that will sound very familiar to what was done with XML modules with onpremises full trust wsp packages.
The basics are making a hierarchy of folders and files to upload and a main powershell script. This is your "release package".

Making a SPO "package" with PnP

I will demonstrate a very simple example here:

• Enable publishing features
• Upload a masterpage file
• Upload a css file
• Upload an image file
• Define my masterpage to be used on the site

Ok so first create a new folder that will be the root of our solution. Then make a subfolder that will contain your assets (masterpage, css, image), mine will be "_catalogs" to match SP structure. Copy html, css and jpg file in the appropriate sub folders.
Then make an xml file and use PnP Provisioning schema to do the step described.

provtemplate.xml :

<pnp:Provisioning xmlns:pnp="http://schemas.dev.office.com/PnP/2016/05/ProvisioningSchema"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- All the schema contents -->
  <pnp:Preferences Generator="Djavan ROA" />
  <pnp:Templates>
    <pnp:ProvisioningTemplate ID="DRO-TEMPLATE-1" Version="1" DisplayName="DROTEMPLATE">
      <pnp:WebSettings SiteLogo="" AlternateCSS="" MasterPageUrl="{masterpagecatalog}/seattle.master" CustomMasterPageUrl="{masterpagecatalog}/dro/mymasterpage.master" />

      <pnp:Features>
        <pnp:SiteFeatures>
          <pnp:Feature ID="f6924d36-2fa8-4f0b-b16d-06b7250180fa" Description="SharePoint Server Publishing Infrastructure"/>
        </pnp:SiteFeatures>
        <pnp:WebFeatures>
          <pnp:Feature ID="94c94ca6-b32f-4da9-a9e3-1f3d343d7ecb" Description="SharePoint Server Publishing"/>
        </pnp:WebFeatures>
      </pnp:Features>

      <pnp:Files>
        <!--Masterpage-->
        <pnp:File Src="_catalogs\masterpage\dro\mymasterpage.html" Folder="{masterpagecatalog}/dro" Overwrite="true" Level="Published">
          <pnp:Properties>
            <pnp:Property Key="ContentTypeId" Value="{contenttypeid:Html Master Page}" />
            <pnp:Property Key="HtmlDesignFromMaster" Value="{masterpagecatalog}/dro/mymasterpage.html, https://drodemo365.sharepoint.com{site}{masterpagecatalog}/dro/mymasterpage.html" />
            <pnp:Property Key="HtmlDesignAssociated" Value="True" />
            <pnp:Property Key="HtmlDesignStatusAndPreview" Value="{masterpagecatalog}/dro/mymasterpage.html, Conversion successful." />
            <pnp:Property Key="BSN" Value="Microsoft.SharePoint.Client.FieldLookupValue" />
          </pnp:Properties>
        </pnp:File>

        <!--CSS-->
        <pnp:File Src="_catalogs\masterpage\dro\css\styles.css" Folder="{masterpagecatalog}/dro/css" Overwrite="true" Level="Published">
            <pnp:Properties>
                <pnp:Property Key="ContentTypeId" Value="{contenttypeid:Design File}" />
            </pnp:Properties>
        </pnp:File>

        <!--Images-->
        <pnp:File Src="_catalogs\masterpage\dro\img\logo.jpg" Folder="{masterpagecatalog}/dro/img" Overwrite="true" Level="Published">
            <pnp:Properties>
                <pnp:Property Key="ContentTypeId" Value="{contenttypeid:Design File}" />
            </pnp:Properties>
        </pnp:File>
        
      </pnp:Files>
    </pnp:ProvisioningTemplate>
  </pnp:Templates>
</pnp:Provisioning>

Then make a script file "prov.ps1", note that I used environment vars this will be required by Team Services Release later.

Import-Module SharePointPnPPowerShellOnline

# Get site url
set-variable -scope Global -name siteUrl -value $env:siteUrl

# Get credentials
$password = $env:deploypassword | ConvertTo-SecureString -asPlainText -Force
$username = $env:deployusername
$cred = New-Object System.Management.Automation.PSCredential($username,$password)

# cd to root script folder
cd $PSScriptRoot


##
# Root site collection
##
Write-Host "> Connect to site collection root" $siteUrl
Connect-PnPOnline –Url $siteUrl -Credentials $cred

# Provisioning
Write-Host "> Provisioning"
Apply-PnPProvisioningTemplate -Path .\provtemplate.xml

The resulting structure should look like this:

.
|-- _catalogs
|   |-- masterpage
|       |-- dro
|           |-- css
|           |   |-- styles.css
|           |
|           |-- img
|           |   |-- logo.jpg
|           |
|           |-- mymasterpage.html
|
|-- prov.ps1
|-- provtemplate.xml

Now upload all this to Team Services source control.



It's all for now, in part 2 I'll show how to setup a custom Team Service Agent, stay tuned.