From f895050314405a4638f477c6f63dc7539dc8b499 Mon Sep 17 00:00:00 2001 From: Tyler Gill Date: Fri, 23 Mar 2018 16:27:11 -0600 Subject: [PATCH 1/5] Update section on filters to talk about filter groups. Also added some more examples and moved things around, hopefully in a way that makes things clearer. --- docs/appendix-d-filters.md | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-) diff --git a/docs/appendix-d-filters.md b/docs/appendix-d-filters.md index a1697ba..590cfc9 100644 --- a/docs/appendix-d-filters.md +++ b/docs/appendix-d-filters.md @@ -7,7 +7,7 @@ Duplicati 2.0 can include or exclude files and folders from your backup based on Duplicati's filter engine processes folders first then files. The reason for that behavior is performance. If a folder is excluded from a backup, the files inside that folder don't have to be processed anymore. -When filter rules have been defined the first folder is taken and the filter rules are processed one by one. The first rule that matches is applied and the following rules are not processed anymore. For instance, if the first rule excludes a folder, then this folder and all files within will be excluded from the backup even if following rules include this folder or its files. +When filter rules have been defined the first folder is taken and the filter rules are processed one by one. The first rule that matches is applied and the following rules are not processed anymore. For instance, if the first rule excludes a folder, then this folder and all files within will be excluded from the backup even if following rules include this folder or its files. Likewise, if the first rule includes a folder, then it will be included even if a following rule would exclude it. It is recommended to write folder rules first and file rules afterwards. That way rules are written in the same order as they will be effective when Duplicati processes them and Duplicati's filters are easier to understand that way. @@ -15,11 +15,41 @@ Per default, all files and folders will be backed up. That means, if no rule mat ### Syntax -If you want to use file globbing to specify rules, `?` and `*` are allowed placeholders. `?` matches any single character. `*` specifies none or multiple characters. Rules can also be specified as regular expression. In this case put the regular expression (using .NET syntax) into hard brackets `[]`. Folder names always end with a slash `/` on Linux or Mac and a backslash `\` on Windows. For instance, `log` is a file, `log/` is a folder. In the UI a rule to include is started with a `+`, a rule to exclude is started with a `-`. Using the command-line there are specific settings to specify include or exclude rules. These are `--include` and `--exclude`. Using the command-line various rules can be specified using `--include` or `--exclude` repeatedly. +Folder names always end with a slash `/` on Linux or Mac and a backslash `\` on Windows. For instance, `log` is a file, `log/` is a folder. + +If you want to use file globbing to specify rules, `?` and `*` are allowed placeholders. `?` matches any single character. `*` specifies none or multiple characters. For example, `*.txt` would specify all `.txt` files. + +Rules can also be specified as regular expression. In this case put the regular expression (using [.NET syntax](https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference)) into hard brackets `[]`. For example, `[.*/IMG_\d{4}\.jpg]` would specify all files named `IMG_0000.jpg` through `IMG_9999.jpg`. + +Duplicati also has several built-in filter groups, which define set of well-known files and folders for different operating systems. These can be specified by putting the names or aliases of the desired groups inside curly brackets `{}` separated by commas `,`, for example, `{SystemFiles,OperatingSystem}`. The following groups are currently defined: + +- **SystemFiles** +Files and paths that are owned by the operating system or are not real files. +For example, `System Volume Information` on Windows and `/proc/` on Linux. +- **OperatingSystem** +Files and folders that are normally part of the operating system. +For example, `C:\Windows` on Windows and `/bin/` on Linux. +- **CacheFiles** +Files and folders that are known to be cache locations. +For example, Google Chrome and Firefox web caches. +- **TemporaryFiles** +Files and folders that are temporary. +For example, the recycle bin on Windows and `/tmp/` on Linux. +- **Applications** +Files and folders that are application binaries (i.e., installed programs). +For example, `C:\Program Files` on Windows and `/lib/` on Linux. +- **DefaultExcludes** +Meta group which contains all groups that are commonly excluded, which is currently all other groups. + +### Using filters + +In the UI, filters can be created using drop down boxes for common rule types. More advanced users can also use a textbox to specify multiple rules, with a rules to include starting with `+` and rules to exclude starting with `-`. + +Using the command-line there are specific settings to specify include or exclude rules. These are `--include` and `--exclude`. Multiple rules can be specified by using `--include` or `--exclude` repeatedly. ### Settings -Besides filter rules there are settings that can exclude specific files by their attributes. Those settings are `--skip-files-larger-than` and `--exclude-files-attributes`. The latter is able to exclude files that have any of the following attributes: ReadOnly, Hidden, System, Directory, Archive, Device, Normal, Temporary. Those settings are applied to all files of the backup. +Besides filter rules there are settings that can exclude specific files by their attributes. Those settings are `--skip-files-larger-than` and `--exclude-files-attributes`. The latter is able to exclude files that have any of the following attributes: `ReadOnly`, `Hidden`, `System`, `Directory`, `Archive`, `Device`, `Normal`, `Temporary`. Those settings are applied to all files of the backup. ### Common use cases From 9b672d94be09835465dc05ed762ff7b564fa9d8e Mon Sep 17 00:00:00 2001 From: Tyler Gill Date: Tue, 10 Apr 2018 10:57:21 -0600 Subject: [PATCH 2/5] Add note pointing out that the MS Groups backend requires admin permissions to use. --- docs/05-storage-providers.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/05-storage-providers.md b/docs/05-storage-providers.md index ecb4343..d88d607 100644 --- a/docs/05-storage-providers.md +++ b/docs/05-storage-providers.md @@ -457,6 +457,10 @@ The username used to connect to the server. This may also be supplied as the env ## Microsoft Office 365 Group (Microsoft Graph API) +***** +> ![](icon_important.png) Using this backend requires the [`Groups.ReadWrite.All` permission scope](https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference#group-permissions) in the Microsoft Graph APIs. These permissions require [admin consent](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-scopes#admin-restricted-scopes) to enable. +***** + This backend can store backups in the document library associated with a Microsoft Office 365 Group, using the following URL format: `msgroup://folder/subfolder` From 2b89e62b00c51175a69279f57638e2fc0febaf1a Mon Sep 17 00:00:00 2001 From: Tyler Gill Date: Tue, 10 Apr 2018 12:19:23 -0600 Subject: [PATCH 3/5] Fix typo in permission name. --- docs/05-storage-providers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/05-storage-providers.md b/docs/05-storage-providers.md index d88d607..6c0b912 100644 --- a/docs/05-storage-providers.md +++ b/docs/05-storage-providers.md @@ -458,7 +458,7 @@ The username used to connect to the server. This may also be supplied as the env ## Microsoft Office 365 Group (Microsoft Graph API) ***** -> ![](icon_important.png) Using this backend requires the [`Groups.ReadWrite.All` permission scope](https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference#group-permissions) in the Microsoft Graph APIs. These permissions require [admin consent](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-scopes#admin-restricted-scopes) to enable. +> ![](icon_important.png) Using this backend requires the [`Group.ReadWrite.All` permission scope](https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference#group-permissions) in the Microsoft Graph APIs. These permissions require [admin consent](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-scopes#admin-restricted-scopes) to enable. ***** This backend can store backups in the document library associated with a Microsoft Office 365 Group, using the following URL format: From 25163f2b1e89ef81912df32c83e4b46973a2e89f Mon Sep 17 00:00:00 2001 From: Tyler Gill Date: Fri, 13 Apr 2018 10:32:27 -0600 Subject: [PATCH 4/5] Add message about required 'Group.Read.All' scope to MS Group backend. --- docs/05-storage-providers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/05-storage-providers.md b/docs/05-storage-providers.md index 6c0b912..b90bedf 100644 --- a/docs/05-storage-providers.md +++ b/docs/05-storage-providers.md @@ -458,7 +458,8 @@ The username used to connect to the server. This may also be supplied as the env ## Microsoft Office 365 Group (Microsoft Graph API) ***** -> ![](icon_important.png) Using this backend requires the [`Group.ReadWrite.All` permission scope](https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference#group-permissions) in the Microsoft Graph APIs. These permissions require [admin consent](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-scopes#admin-restricted-scopes) to enable. +> ![](icon_important.png) Specifying the group's email address when using this backend requires the [`Group.Read.All` permission scope](https://developer.microsoft.com/en-us/graph/docs/concepts/permissions_reference#group-permissions) in the Microsoft Graph APIs. These permissions require [admin consent](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-scopes#admin-restricted-scopes) to enable. +You can use this backend without this permission by using an AuthID generated for the `SharePoint v2` backend and passing the `--group-id` instead of `--group-email`. You can lookup the group ID by using the [Microsoft Graph Explorer](https://developer.microsoft.com/en-us/graph/graph-explorer). ***** This backend can store backups in the document library associated with a Microsoft Office 365 Group, using the following URL format: From aa7124da3d04449a3cc0a6c845565fa444a3425d Mon Sep 17 00:00:00 2001 From: kees-z Date: Sat, 14 Apr 2018 10:36:01 +0200 Subject: [PATCH 5/5] Update index.md --- docs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index 3b5a9e8..771bd74 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,7 +6,7 @@ ![](duplicatilogo.png) -2018/04/06 +2018/04/14