Retention in SharePoint Online: the HOW
There are many things to consider when applying retention to your SharePoint Online content, and each decision you make will have follow-on effects to be aware of. I’m writing 4 posts to highlight 4 key questions to answer when configuring Office 365 retention and will identify some pros and cons of decisions surrounding each:
This post is the third in the series, Retention in SharePoint Online: The Where, What, and How, and When, and answers the all-important “HOW” question:
- Part I: Retention in SharePoint Online: The “WHERE” answers the question “Where should retained content live?”
- Part II: Retention in SharePoint Online: The “WHAT” answers the question “What type of retention should be applied?”
- Part III: Retention in SharePoint Online: The “HOW” (THIS POST) answers the question “How should the retention be applied?”
- Part IV: Retention in SharePoint Online: The “WHEN” answers the question “When should the retention start?”
I think the image below sums up the retention scenario across Microsoft 365. Knowing you need it is only scratching the surface (which is partly what this blog post series is addressing)…
Let’s answer “The HOW”…
You’ve decided where you’ll retain your content and what type of retention you’ll apply, and now you need to decide how you’re going to apply it. This primarily comes down to whether or not you want to involve an end-user in applying retention, your risk tolerance, your comfort with custom code, and the license you have in your tenant. I’ll cover several options for both retention policies and retention labels to retain content stored in SharePoint Online… some automated, some manual and pros/cons for each. At the end of the post, I’ll summarize the options and the license required for each.
There are 5 ways to apply a Retention policy and 10 ways to apply a Retention label!
5 Ways to apply a Retention Policy
#1 — Org-wide all Workloads
Include all content locations in the Retention Policy configuration. You’re allowed 10 of these per tenant. There is NO LIMIT to the number of each workload that will be included in the policy. This means as new SharePoint sites and OneDrives (and mailboxes) are added to your environment, they will be automatically included and the retention policy will apply to content within.
Pros:
- simple to configure — everything falls under the same retention rules
- good model to use in tandem with other retention policies/labels across your tenant
Cons:
- there will likely be exceptions to this retention so you must ensure you are also applying more granular retention controls to handle exceptions (either thru a policy exclusion/inclusion OR a retention label)
#2 — Org-wide Workload specific
If you select Let me choose specific locations, you can choose All of one or more specific types. In this example, we could select All SharePoint sites and All Office 365 Groups to apply retention automatically to them. Similar to an org-wide retention policy, there is NO LIMIT to the number of each workload that will be included in the policy. This means as new SharePoint sites and Office 365 Groups are added to your environment, they will be automatically included and the retention policy will apply to content within
Did you know? Sites included when you toggle the SharePoint sites are Communication sites, modern team sites not backed by a Group, and classic sites. If you want to include a Modern Team site backed by a Group in retention, you must use the Office 365 groups toggle.
Pros:
- simple to configure as everything falls under the same retention rules
- good model to use in tandem with other retention policies/labels across your tenant
Cons:
- there will likely be exceptions to this retention so you must ensure you are also applying more granular retention controls to handle those exceptions (either thru a policy exclusion/inclusion OR a retention label)
#3 — Manually include/exclude specific Site/Group
You can manually include or exclude a SharePoint site or Office 365 Group in a Retention Policy prior to publishing. There are some limitations to this you must be aware of for the maximum number you can include or exclude per retention policy:
- 100 sites (either OneDrive or SharePoint)
- 1,000 Office 365 groups
Due to this, you must work within these bounds if you are automating a solution that includes/excludes SharePoint sites/Office 365 Groups in a retention policy.
Pros:
- simple to configure when you want to target an entire site/Group for a unique retention requirement
Cons:
- with the limits above, you may run into these in large tenants
- if doing this in an automated way, you must account for and word around the limits
#4 — Conditionally apply
Just like you can conditionally apply a retention label (you can read about this in the next section below), you can also conditionally apply a retention policy to a piece of content. At the time of this writing, the condition can be either a keyword or a sensitive information type.
This can also be done via PowerShell with the Set-RetentionComplianceRule cmdlet.
Pros:
- more targeted retention so not as much storage space will be consumed by the Preservation Hold Library (PHL) on the SharePoint site
- good model to use in tandem with automation when provisioning sites and adding them into the policy
Cons:
- there may be exceptions to this retention so you must ensure you are also applying more granular retention controls to handle them (either thru a policy exclusion/inclusion OR a retention label)
#5 — PowerShell
You can add a SharePoint site/Office 365 Group to a retention policy during a provisioning process. Reference: PowerShell cmdlets for Retention Policies
For a NEW Retention policy… use the New-RetentionCompliancePolicy cmdlet to create a new retention policy in the Compliance Center and associate a location to it.
For an EXISTING Retention policy… use the Set-RetentionCompliancePolicy cmdlet to update an existing Retention policy to add a new location or remove an existing location.
If you are using Azure Automation to provision a site, you could include a new site in a retention policy with the Set-RetentionCompliancePolicy cmdlet. Keep in mind the limitations identified above for the maximum number of inclusions allowed.
Pros:
- doesn’t require an Administrator to manually add a site/Group into a policy
- good model to use in tandem with site provisioning solution
Cons:
- technical debt for the custom code
10 Ways to apply a Retention Label
I’ll summarize the 10 ways, however if you’re looking for more detail, check out the link at the end of the section for a presentation where I include videos for most of the ways!
#1 — Manually apply the Retention Label
An end-user can select a document (or several documents) from a SharePoint library, open the detail pane and set the retention label. In the Apply retention label drop-down, you will only see retention labels previously published to the site. In the image below, 5 retention labels have been published to this site.
Did you know? You cannot make a retention label required. Any user with at least contribute permission level can remove a retention label or change it. An exception to this is if the retention label has made the document a record. In this case, the retention label cannot be removed or changed unless done by a Site Collection Admin (includes Group owners).
Pros:
- end-users can self-serve (a pro AND a con)
- good model to use in tandem with other retention policies/labels
Cons:
- end-users don’t always know what retention label to apply which means they may apply the wrong one or none at all
#2 — Automatically set at a library level
On every document library, there is a setting to set a default retention label. Select this and choose the retention label you would like all documents to be defaulted with when added to the library. You can (optionally) assign all existing documents this same label as well.
Did you know? Even if you don’t have the license for this feature, you will still see this as an option? Check your license before you use this feature! #DontShootTheMessenger
Pros:
- all documents in the library inherit the label so end-users don’t have to apply it manually
- good model to use in tandem with other targeted retention labels for more targeted retention scenarios
Cons:
- often document libraries aren’t set up to align with your retention requirements
- requires an elevated level of licensing (see license table at end)
#3 — Automatically set at a folder level
Similar to applying a retention label to a document, an end-user can select a folder (or document set) from within a SharePoint library, open the detail pane and set the retention label at a folder level. In the Apply retention label drop-down, you will only see retention labels previously published to the site. In the image below, 5 retention labels have been published to this site.
Did you know? Once you apply a retention label to a folder, all documents within the folder automatically inherit the retention label (unless the document already had another retention label applied). If you remove the retention label from the folder, the retention label will also be removed from the documents within (unless a document had another retention label applied)
Pros:
- all documents in the folder inherit the label so end-users don’t have to apply it manually
- good model to use in tandem with a high-level folder structure in a library if the folders align with your retention requirements
Cons:
- requires an elevated level of licensing (see license table at end)
- end-users will be able to remove the retention label from the folder (unless it’s a record) which will also remove the retention label from documents within the folder
Auto-apply Options (#4 thru #8)
#4 thru #8 all use the auto-apply capability for a retention label. You can see this option after you’ve defined a retention label and are prompted for further actions to take. In this example, the Board record retention label has the Auto-apply a label option available.
Did you know? For any of the auto-apply options, once you define the Auto-apply condition, it can take up to 7 days for content across your tenant to have the retention label applied. Also, the content must be indexed by search! If you exclude a site or library from search, it won’t be covered in an auto-apply label policy.
#4 — Auto-apply for a sensitive information type
Select the Auto-apply a label button and then choose the first option below:
This auto-apply capability can be used to detect sensitive information types across content and apply a retention label when detected. In this example, I’m looking for Canadian Financial information to apply a retention label:
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
Cons:
- requires an elevated level of licensing (see license table at end)
#5 — Auto-apply based on keyword query
Select the Auto-apply a label button and then choose the second option below:
A keyword query uses a special syntax called Keyword Query Language(KQL) and it can look for several things. On a SharePoint site, the most common use-case I see is Path and Site. Using these properties, you can auto-apply retention labels to a site starting with a prefix. This is something you could do if you are also controlling the provisioning process for sites, including the name. (E.g. Project sites)
Example:
This would apply a retention label to any site starting with the above URL prefix.
Reference: SharePoint searchable site properties
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
- could use this in tandem with a site-provisioning solution since you can control a site’s URL (and other properties)
Cons:
- requires an elevated level of licensing (see license table at end)
#6 — Auto-apply based on a content type
Select the Auto-apply a label button and then choose the second option below:
This also uses KQL and allows you to target a specific content type in your environment. This is a golden opportunity to leverage any information architecture you have set up across your tenant.
Example: ContentType:”Project Document”
Did you know? You can use compound conditions in your KQL. As long as they are searchable properties, you’re gold!
Example: ContentType: AND Author:joeblow@contoso.com
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
- leverages content types you may already have defined across your environment
Cons:
- requires an elevated level of licensing (see license table at end)
#7 — Auto-apply based on a metadata value
Select the Auto-apply a label button and then choose the second option below:
This also uses KQL and allows you to target a specific managed property for metadata across your environment. There are some limitations to this feature on the data types allowed. As of the time of this writing, the following column types are allowed: choice, managed metadata if published from the Content Type Hub, and Date.
This is another golden opportunity to leverage information architecture you have set up in places across your tenant for the above column types.
Example: RefinableDate##<TODAY
Did you know? For some data types, you need to map the crawled property generated from the metadata column to a predefined Refinable managed property and then use the Refinable managed property in the query.
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
- leverages metadata you have defined across your environment
Cons:
- requires an elevated level of licensing (see license table at end)
#8 — Automatically set based on Classifier
Select the Auto-apply a label button and then choose the third option below:
Currently in preview, you can use Classifiers (5 are built-in) or build your own to be able to intelligently detect specific types of content across data in your tenant.
“This method of classification is particularly well suited to content that isn’t easily identified by either the manual or automated pattern matching methods.”
Reference: Getting started with trainable classifiers (preview)
Defining the classifier is outside of the scope of this post, however once defined, you can use it as an auto-apply condition for a retention label.
Use-cases I can see for custom Classifiers are: employee forms, contracts, budgets, customer forms, etc.
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
- leverages classifiers you may already have defined across your environment (classifiers can be used in other areas of Office 365)
Cons:
- requires a custom classifier to be configured in your environment (can also use a built-in one, however the most prevalent use-case I see for this is a custom classifier)
- requires an elevated level of licensing (see license table at end)
#9 — Automatically set using Power Automate
You can invoke the Send an HTTP Request action from a Power Automate Flow to make a REST call against the document using the SetComplianceTag method.
Did you know? The retention label must already be published to the site before the above action will work. Also, you must set the above retention label settings correctly to match the retention label settings.
- isTagPolicyHold: don’t know what this maps to, perhaps disposition review? If you do, let me know and I’ll update post. In the interim, I set this to false
- isTagPolicyRecord: if this is a Record Retention label, it’s true
- isEventBasedTag: if this is an event-based Retention label, it’s true
- isTagSuperLock: don’t know what this maps to. If you do, let me know and I’ll update post. In the meantime, I set this to false
Pros:
- reduces the amount of manual intervention by end-users and therefore improves your compliance posture
- can use any kind of custom logic to accommodate your unique business scenarios for applying retention
Cons:
- technical debt of building/maintaining the Flow using Power Automate
- requires an elevated level of licensing (see license table at end)
#10 — Automatically set using custom code/PowerShell
There’s several other ways to set a retention label if you’re open to using some custom code or PowerShell. These are the current options:
HTTP Client to call the REST API
This is the same REST call made from the Power Automate Flow in #9. Any HTTP client can invoke the same REST call to apply a retention label on a document:
/_api/web/Lists/GetByTitle(‘LibraryName’)/items//SetComplianceTag
With this as the body:
{
“complianceTag”:”Retention Label name”,
“isTagPolicyHold”: “False”,
“isTagPolicyRecord”: “False”,
“isEventBasedTag”: “False”,
“isTagSuperLock”: “False”
}
SharePoint PnP (Patterns & Practices) PowerShell
You can use Set-PnPLabel to default a document library and Set-PnPListItem to set an individual document’s retention label. I have used both of these PnP cmdlets in a production environment in Azure Automation and it works extremely well:
Set-PnPLabel -List “List name” -Label “Retention Label name” -SyncToItems $true
Set-PnPListItem –List “List name” –Identity -Label “Retention Label name”
SharePoint CSOM (Client-Side Object Model)
You can use SPPolicyStoreProxy.SetListComplianceTag to default a document library and ListItem.SetComplianceTag to set an individual document’s retention label.
Note: I have not used the CSOM method in a production SPO environment.
For ALL custom code solutions:
Pros:
- reduces the amount of manual intervention by end-users and therefore improves the compliance posture
- can use any kind of custom logic to accommodate your unique business scenarios for applying retention
Cons:
- technical debt of custom code/solution/script
Too long, didn’t read (TLDR)…
I get it, we’re all short on time nowadays…
Here’s the list of all the ways to apply Retention Policies and Retention Labels and their licensing requirements (as of May 2020):
Please refer here for a Detailed Microsoft 365 Compliance Licensing Comparison.
Thanks for sticking with me. I hope you found this helpful and I’d love to know if you have any other techniques you’ve come across on HOW to apply retention across SharePoint.
This Retention series covered where to store your retained content, what type of retention to apply once you put it there, and how to apply it. I hope you found this series helpful and can use it to start building your own retention solutions across your tenant.
Good luck and reach out for comments, feedback, and questions!
-JCK
Originally posted at: https://joannecklein.com/2020/05/07/retention-in-sharepoint-online-the-how/
