Recently I posted a series of three articles describing our approach to DevOps for Microsoft Dynamics 365, and the technology behind it.
After giving a session on this topic at CRM Saturday in Madrid, Spain, it is now time to announce “public preview” of our tools.

If you want the full story – these are the articles describing the background and technology behind our tools:

Part I – Background and how our DevOps tools evolved before we knew about it
Part II – Automation of the build and deploy process using custom VSTS Build Tasks
Part III – Demo of complete build and release definitions taking you from A to Z

Helping tools

As described in the previous articles, the core of our tools is the Shuffle. The behavior of this is controlled by Shuffle Definition files, which is an XML file with a custom schema. Since XML may not be everyone’s native language, there is a tool for XrmToolBox called Shuffle Builder, that will help compose these XML files in the same way as SiteMap Editor and FetchXML Builder.

There is another tool called Shuffle Runner that can be used to test the Shuffle Definitions. As these tools are “not for anyone”, they are not at this time published in the XrmToolBox Plugins Store. Instead you can download them and follow the instructions here: https://github.com/Innofactor/Innofactor.Crm.CI/releases/

Project set-up

The solution in this example is basically built from a plugin project, and folders containing web resources. In addition to that there is a folder DevOps that contain files that specify details of the build and deploy mechanics.

WebResourcePatterns.txt defines which files and folders to include and exclude when updating web resources in the CRM DEV environment.
Shuffle Jonas Tools Solution.xml is the Shuffle Definition defining which and how solutions should be exported during build and imported during deploy.
Shuffle Jonas Tools Configuration.xml is the Shuffle Definition defining which and how configuration data should be exported during build and imported during deploy.
Jonas Tools.cdpkg is the definition file used by the CRM Shuffle Deployer during manual deployment of the complete package.

Solution Shuffle Definition

This is a sample Shuffle Definition defining export and import of one solution called Jonas Tools. This is used by the generic VSTS Build tasks and by the CRM Shuffle Deployer package.

<ShuffleDefinition StopOnError="true">
  <Blocks>
    <SolutionBlock Name="JonasTools">
      <Export Type="Both" SetVersion="{ShuffleVar:version}" PublishBeforeExport="true">
        <Settings IsvConfig="false" RelationshipRoles="false" OutlookSync="false" Marketing="false" General="false" EmailTracking="false" Customization="false" Calendar="false" AutoNumbering="false" />
      </Export>
      <Import Type="Managed" PublishAll="false" OverwriteCustomizations="true" ActivateServersideCode="true" OverwriteNewerVersion="false" OverwriteSameVersion="false" />
    </SolutionBlock>
  </Blocks>
</ShuffleDefinition>

Data Shuffle Definition

Below is a sample of a Shuffle Definition used to define export and import details for configuration data used in my sample solution. Just like the definition above, this is used during automated build and deploy, and for manual deployment of the package.

<ShuffleDefinition StopOnError='true' >
  <Blocks>
    <DataBlock Name='SettingValue' Entity='jonas_setting' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_name' />
          <Attribute Name='jonas_value' />
        </Attributes>
      </Export>
      <Import CreateWithId='false' Save='CreateOnly' >
        <Match PreRetrieveAll='true' >
          <Attribute Name='jonas_name' />
        </Match>
      </Import>
    </DataBlock>
    <DataBlock Name='SettingDescr' Entity='jonas_setting' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_name' />
          <Attribute Name='jonas_description' />
        </Attributes>
      </Export>
      <Import CreateWithId='false' Save='UpdateOnly' UpdateInactive='false' UpdateIdentical='false' >
        <Match PreRetrieveAll='true' >
          <Attribute Name='jonas_name' />
        </Match>
      </Import>
    </DataBlock>
    <DataBlock Name='Country' Entity='jonas_country' >
      <Export>
        <Filter Attribute='jonas_continent' Operator='Equal' Type='int' Value='147450000' />
        <Attributes>
          <Attribute Name='jonas_*' />
        </Attributes>
      </Export>
      <Import CreateWithId='false' Save='CreateUpdate' UpdateInactive='false' UpdateIdentical='false' >
        <Match>
          <Attribute Name='jonas_code' Display='jonas_name' />
        </Match>
      </Import>
    </DataBlock>
    <DataBlock Name='County' Entity='jonas_county' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_*' IncludeNull='false' />
        </Attributes>
      </Export>
      <Import CreateWithId='false' Save='CreateUpdate' Delete='All' UpdateInactive='false' UpdateIdentical='false' >
        <Match>
          <Attribute Name='jonas_country_id' Display='' />
          <Attribute Name='jonas_name' Display='' />
        </Match>
      </Import>
      <Relation Block='Country' Attribute='jonas_country_id' IncludeNull='false' />
    </DataBlock>
    <DataBlock Name='City' Entity='jonas_city' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_*' IncludeNull='false' />
        </Attributes>
      </Export>
      <Import CreateWithId='false' Save='CreateUpdate' UpdateInactive='false' UpdateIdentical='false' >
        <Match>
          <Attribute Name='jonas_county_id' Display='' />
          <Attribute Name='jonas_name' Display='' />
        </Match>
      </Import>
      <Relation Block='County' Attribute='jonas_county_id' IncludeNull='false' />
    </DataBlock>
    <DataBlock Name='River' Entity='jonas_river' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_*' IncludeNull='false' />
        </Attributes>
      </Export>
      <Import>
        <Match>
          <Attribute Name='jonas_name' Display='' />
        </Match>
      </Import>
    </DataBlock>
    <DataBlock Name='RiverCounty' Entity='jonas_river_jonas_county' Type='Intersect' >
      <Export>
        <Attributes>
          <Attribute Name='jonas_riverid' IncludeNull='false' />
          <Attribute Name='jonas_countyid' />
        </Attributes>
      </Export>
      <Import CreateWithId='true' Save='CreateUpdate' UpdateInactive='false' UpdateIdentical='false' >
        <Match>
          <Attribute Name='jonas_riverid' Display='' />
          <Attribute Name='jonas_countyid' Display='' />
        </Match>
      </Import>
      <Relation Block='County' Attribute='jonas_countyid' IncludeNull='false' />
      <Relation Block='River' Attribute='jonas_riverid' IncludeNull='false' />
    </DataBlock>
  </Blocks>
</ShuffleDefinition>

Digging into the details of the XML above could be relevant if you really want to make good use of the data shuffling capabilities, but for now you may move on and instead look at the same Shuffle Definition in Shuffle Builder in XrmToolBox, which would look like this.

And this is an example of exporting with the Shuffle Runner.

Deployer Package

This file defines which Shuffle Definitions that are included in the deployment package.

<Package>
  <Modules>
    <Module Name="Jonas Tools Solution" Type="ShuffleDefinition" Description="This module contains the solutions required for the Jonas Tools functions" File="Shuffle Jonas Tools Solution.xml" DataFile="" Default="true" Optional="false" />
    <Module Name="Jonas Tools Configuration" Type="ShuffleDefinition" Description="Default configuration data for Jonas Tools" File="Shuffle Jonas Tools Configuration.xml" DataFile="Shuffle Jonas Tools Configuration.data.xml" Default="true" Optional="true" />
  </Modules>
</Package>

The package can also be edited by opening the package file with the CRM Shuffle Deployer.

Admin pro tip: Press <ctrl>+<shift>+A to display the Build deploy packages tab.

Connecting the dots

To get a head start to creating build and release definitions, I strongly encourage to read the last article from the series describing the evolution of our DevOps: Part III – Demo of complete build and release definitions taking you from A to Z.This will give you an overview of the different steps to include in the definitions, to compose a fully automated process.

Install VSTS extensions

To install our DevOps tools for VSTS Build and Release management, install (or download) the extension from the marketplace:
https://marketplace.visualstudio.com/items?itemName=InnofactorSE.cinteros-devutils-ci-build-tasks

Follow and contribute

Source code for the Shuffle core and VSTS extensions are published on GitHub – feel free to submit issues for improvements, contribute, or just fork and dig into it to see how we do things:https://github.com/Innofactor/Innofactor.Crm.CI

What’s next?

Next improvement to our tools is to include typescript handling, so we don’t have to add the generated js files into the repository, but instead build them during the automated build.

Need more?

If you are interested in more information, help to get started, or have suggestions for improvements – don’t hesitate to contact me through the usual channels; Twitter @rappen, comments below, or send me an e-mail.