Uploads Not Going on External Drive Nextcloud

Backup

This App creates and stores backup images of your Nextcloud.

(Documentation is all the same in writing)

• Important notes
• Restoring Points
• Hardware Requirement
• How the Backup App manage your information
• Export configuration
• Important details well-nigh your data
• Upload packed backup to External Storages
• Quick compression during 1st pass
• AppData on External Storage
• Restoring a fill-in
• Available occ commands
• Faq
• Known issues

Important notes

• Read the full documentation,
• During the generation of the fill-in, the app will put your example in maintenance mode,
• This app generates a lot of information and tin can fill your hard drive,
• Past default, your data are encrypted meaning you will demand to export the configuration of the App as soon every bit possible or you will not be able to decrypt your backups.

Restoring Points

A restoring bespeak is an paradigm of your Nextcloud at a specific time. A restoring point tin be:

• 'Full' (or Complete) and contains a backup of :

  • The instance of Nextcloud, its apps and config/config.php,
  • A dump of the database,
  • The local folder defined as data of the Nextcloud.

• 'Partial' (or Differential) that contains a fill-in of :

  • The case of Nextcloud, its apps and config/config.php:
    • A dump of the database,
    • Local data that have been modified/generated since the last Full Backup.

What data are available in a Restoring Point

Allow's first start with the fact that the Backup App will not shop ALL data from your Nextcloud.
As an example, remote files won't be stored.

This is a list of what can be restored and what cannot be restored when using the Backup App:

A restoring point will store:

• Your current Nextcloud,
• The configuration in config/config.php,
• The apps/ folder and whatsoever other custom_apps/,
• Your local information/, defined by 'datadirectory' in config/config.php,
• A full sqldump of your database,
• List of files and localisation within the backup,
• Metadata about the instance.

A restoring point volition NOT store:

• Data from External Storages, fifty-fifty if the mounted filesystem is available locally.

Metadata

Each Restoring Point includes a file named restoring-point.information that contains metadata well-nigh the backup:

• Version of your Nextcloud,
• The ID of the parent backup in case of partial fill-in,
• The listing of data file that compose the restoring point, the format for this data depends on the current status of the restoring point (packed/unpacked) and the settings (compression, encryption),
• Absolute paths for each information file,
• Checksum for each file of the backup itself,
• The date of the restoring point,
• Comments,
• Information related to the health of the files during the last check.

While the file restoring-point.data is require to confirm the integrity of all files and parts of the backup, information technology is still possible to generate a restoring point based on the available files. Yet :

• There is no way to confirm the integrity of the restoring point,
• The restoring procedure will require some cognition from the admin near the original infrastructure from The original instance that generated the fill-in.

Generate Metadata from backup files

• Upload the files of your restoring point on your case of Nextcloud with the Backup App installed, in a specific folder in your Files,

• At the root of this specific folder, create a file named restoring-indicate.data with this content:

                                        {"action": "generate", "id": "20211023234222-full-TFTBQewCEdcQ3cS"}                                  

• Customize your id; while information technology is brash to use the correct Id of the Restoring Indicate (if known), any cord would work. If kept empty, a new Id will be generated using the current fourth dimension,

• Right-click the file restoring-point.data and select Scan Backup Folder.

Subsequently few seconds, the metadata file volition exist generated and stored within the aforementioned restoring-point.information itself.

Hardware requirement

• Diskspace: Creating and storing backups require a lot, a lot, of disk-infinite,

• AES Hardware Dispatch: If your processor does not support AES pedagogy set up, the encryption procedure will autumn back to aes-256-cbc.
  This should only affect you lot if using the Backup App to migrate your instance from an AES-supporting CPU to a non-AES-supporting CPU (ie. old arm proc). Enforcing the employ of aes-256-cbc earlier the packing of the restoring point on a AES-supporting CPU will fix this:

  • Run: ./occ config:app:set backup force_cbc --value '1' ,
  • Pack the restoring point: ./occ backup:indicate:pack <pointId>.

Configure the handle of your information

The timing

From the Admin Settings/Backup page, you tin configure the fourth dimension slot and the rate for the generation of your futurity backups:

The time slot define the time of the day the Fill-in App might run its 1st pass to generate a backup, it is based on the local fourth dimension on the server.

Go on in mind that, when generating a new fill-in, your instance will be in maintenance fashion for the total duration of the 1st pass. This is the reason why, by default, Total Fill-in volition just be started during the week-end, while Partial Backup are also run during week days.

If you lot curl downwards to the bottom of this page, you can have an interpretation of the appointment for your next backup based on your settings:

The outset pass (the backup process)

During the First Pass, data are speedily stored in the appdata binder of the Backup App.
At this point, at that place is no compression nor encryption; the starting time pass needs to exist as fast as possible to release the maintenance mode on the example.
The information are stored in a list of nil files (named chunk), each i with a maximum size of 4GB (unless it contains a file bigger than 4GB).

Because there is no pinch during the first pass, the appdata binder of the Fill-in App will crave at least the same size of your current setup of Nextcloud: the content of the core, its apps and local information.

By default, the appdata folder of the Backup App is located in the aforementioned folder than the balance of the data of your instance defined in datadirectory. It is estimated that the Backup App needs 65% of the available diskspace of the datadirectory

In instance there is not enough space, you can:

• Mountain an External Storage and move the appdata binder of the Backup app there.

The 2d pass (the packing procedure)

The second pass does not require to put your example in maintenance fashion. The second laissez passer consist in the packing of the restoring point and somewhen its upload on external storage.

You tin can configure the type of packing in the Admin Settings of the app:

The packing will list each chunk of your backup and:

• Shrink them (if enabled),
• Carve up the consequence in multiple files (named part) of 100MB,
• Encrypt each part (if enabled),
• Once completed without consequence, remove the original zip file of the chunk to free space.

Storing on a different hard drive

Once packed, restoring points can be stored on a different location. Locally or remotely.

if configured, the Backup App will start storing your restoring points externally, and check their integrity every twenty-four hours.

The uploading process will check that each function of the packed restoring betoken are healthy, based on the checksum stored in the metadata file and volition retry to upload whatsoever faulty part.
On top of that, the content from the metadata file is signed, making the Fill-in App able to confirm the total integrity of the backup.

Exporting your configuration

Of import details about your data

• Disk-infinite: The 1st pass does not compress anything, significant that you volition need at least the equivalent of currently used infinite by your Nextcloud as available disk-infinite. If you have no disk-space available, y'all tin can configure the app to use an external storage to shop all its data.

• Temporary Files: during the 2nd laissez passer (packing process), the compression and encryption require the cosmos of temporary files. while those files are temporary and deleted when they become useless, they are notwithstanding bachelor for few seconds. Meaning that the temp directory should not be shared with other application.

• Export your setup: If the option is not disabled, backups are encrypted with a key that is stored in the database of your current instance of Nextcloud. The key is mandatory to recover any data from your backups.

  You can export your setup from the Admin Settings/Backup page, or using occ. If encrypted, the export process will generate and returns its own key that volition be required during the import when restoring your instance. As an admin, y'all will demand to shop the export file and its key, preferably in dissimilar locations.

• .nobackup: The presence of a .nobackup file in a binder will exclude all content from the electric current folder and its subfolders from the backup.

Upload to External Storages

Uploading your packed restoring point on one (or multiple) external storages is the perfect solution when facing a huge loss of information from your disk whether it is of human or hardware origin.

Those external data are fully managed by the app itself.

The configuration is done in 2 steps:

• The first step is to setup a folder from the External Storage Settings Folio, information technology is strongly adviced to limit the availability of the folder to the admin group only:

• The second step is done in the Fill-in Settings Page, the configured External Storage should be displayed in the listing of available storage location:

• Set the folder where to shop your fill-in files on the external storage, and click on 'Add new external location'.

AppData on External Storage

If you have no disk-space available, you can configure the app to use an external storage to store all its data:

• The information generated during the 1st laissez passer are not encrypted, Your information leaves the internal data binder from your instance and are now available on an external storage,
• The 1st-laissez passer volition require more resources and your instance will stays in maintenance mode for a longer fourth dimension,
• If your external storage is not a local folder, huge network resources will be required,
• During the configuration, onetime restoring points from the previous storage will exist deleted.

From a concluding, run ./occ backup:external:appdata and follow the instructions to select the external storage and the folder.
From the Backup Settings Folio, y'all can do the same:

Restoring a fill-in

You lot tin can restore a unmarried file or the whole instance to a previous state:

              ./occ backup:point:restore <pointId> [--file <filename>] [--information <dataPack>] [--chunk chunkName]                          

Please note that you lot tin can get back to a previous backup of your instance from any version of Nextcloud uniform with the Backup App. There is no need to install the exact same version as it will be reverted to the one used when creating the Restoring Point. Pregnant that you can fully restore your instance of Nextcloud fifty-fifty if you lot lost your harddrive, every bit long as you lot kept a copy of the Restoring Point (upload to another remote instance).

Restoring my Nextcloud from scratch.

In this scenario, you lot have lost everything and desire to fully recover your Nextcloud.

The first footstep would exist to have a bones setup of Nextcloud+Backup:

• Install a version of Nextcloud compatible with the Backup App,
• Install the Backup App,
• Import your setup that contains the signature and encryption keys from your previous example. Y'all can bypass this footstep just if your fill-in are not encrypted and y'all practice not need the ability to ostend the integrity of files:
  • ./occ backup:setup:import [--key <cardinal>] < ~/backup_setup
• Enable and Configure the External Storage App, if your backups are on a external storage.

Then, you accept to add your last valid restoring points from your previous case. Both the last Full-Backup and the last Partial-Backup you take in mitt:

• Restoring my Nextcloud from a remote storage
• Restoring my Nextcloud from my workstation

Restoring my Nextcloud from a remote storage

If your backups are on an external storage, bold you have already configured it in both External Storage and Backup, your restoring points should already be available.

Note: in that location is a known bug that might require you to scan the root of your Nextcloud Files later the cosmos of the external storage.

• List your bachelor restoring points:

              $ ./occ backup:point:list - Retrieving data from local - Retrieving data from external:3 > Found RestoringPoint 20211031232710-full-Tu4H6vOtxKoLLb9 > Establish RestoringPoint 20211101014009-full-QeTziynggIuaaD2 +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+ | Restoring Point                       | Date                | Parent | Comment | Status                      | Instance   | Health       |  | +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+ | A 20211031232710-full-Tu4H6vOtxKoLLb9 | 2021-10-31 23:27:10 |        | beta2   | packed,compressed,encrypted | external:three | 12H, 23M ago |  | |  20211101014009-full-QeTziynggIuaaD2  | 2021-xi-01 01:xl:09 |        |         | packed,compressed,encrypted | external:3 | 10H, 53M agone |  | +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+                          

• Download the one y'all want to restore; if you desire to restore multiple backups, it is a good idea to download them all at that signal:

              $ ./occ fill-in:indicate:download 20211031232710-full-Tu4H6vOtxKoLLb9 --external three > downloading metadata check wellness status: 0 correct, 43 missing and 0 faulty files   * Downloading data/data-0540e4d6-9d7f-4c84-a8d8-ca40764257d1/00001-B57XWKJQe5Xg1sd: ok   * Downloading data/data-0540e4d6-9d7f-4c84-a8d8-ca40764257d1/00002-PXHPeS6t6OXFwkP: ok [...]                          

• Unpack the downloaded restoring indicate:

              $ ./occ backup:point:unpack 20211031232710-full-Tu4H6vOtxKoLLb9 Unpacking Restoring Point 20211031232710-full-Tu4H6vOtxKoLLb9  > lock and ready status to unpacking  > Browsing RestoringData data    > Unpacking RestoringChunk data-0540e4d6-9d7f-4c84-a8d8-ca40764257d1: proceeding      * Copying parts to temp files        - 00001-B57XWKJQe5Xg1sd: /tmp/phpNnYCbZ        - 00002-PXHPeS6t6OXFwkP: /tmp/phpYqRSPW [...]                          

• Starting time the restoring process. Please annotation that:
  • For each data pack, you volition be able to choose the location of the extraction, (you can bypass using --practice-not-ask-data),
  • If a sqldump is available, y'all will exist prompt to use the electric current configuration from the instance, the one from the config/config.php freshly restored or use another database (you can bypass using --do-not-ask-sql),
  • If the information from the file config/config.php are in disharmonize with the path or sql settings specified during the extraction, you will be notified that the restoring process wants to update them:

              $ ./occ backup:signal:restore 20211031232710-full-Tu4H6vOtxKoLLb9 Restoring Point: 20211031232710-full-Tu4H6vOtxKoLLb9 Date: 2021-10-31 23:27:10 Checking Health status: ok   Alarm! You lot are virtually to initiate the complete restoration of your instance! All information generated since the creation of the selected backup volition be lost...  Your case volition come up back to a previous state from xiii hours, 17 minutes and 48 seconds ago.  Do you really want to continue this operation ? (y/North)                          

Restoring my Nextcloud from my workstation

If your backups are on your workstation, yous can upload them on your Nextcloud Files, on your own Nextcloud account. One time uploaded, open the folder containing the restoring bespeak to find the metadata file restoring-indicate.information.

Right-click the file restoring-point.information and select Scan Backup Folder.

The scan of the restoring volition be initiated at the side by side tick of your crontab. The background task volition browse the full folder and its content, copy pertinent data into the app'due south appdata and create a new entry in the database.

In one case available in the listing of your available restoring points, the process will exist the aforementioned as described in Restoring my Nextcloud from Appdata.

Restoring my Nextcloud from AppData

After restoring a previous restoring point, you lot tin can face a situation where some restoring points are available in your appdata but not displayed in the listing. This de-synchronisation can be fixed by running ./occ backup:indicate:scan. This will scan your appdata and add the restoring points to your current database.

Once available in the listing, you can have a better overview of the current status of the restoring point:

              $ ./occ fill-in:bespeak:listing +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+ | Restoring Point                       | Date                | Parent | Annotate | Status                      | Case   | Health       |  | +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+ | A 20211031232710-full-Tu4H6vOtxKoLLb9 | 2021-10-31 23:27:10 |        | beta2   | packed,compressed,encrypted | external:iii | 15H, 36M ago |  | |  20211101014009-full-QeTziynggIuaaD2  | 2021-11-01 01:forty:09 |        |         | not packed                  | local      | 1H, 13M ago  |  | +---------------------------------------+---------------------+--------+---------+-----------------------------+------------+--------------+--+                          

In order to restore a fill-in, the restoring point needs to be not packed, if the restoring indicate you lot want to restore has the packed condition, you lot will need to unpack it outset:

              $ ./occ backup:bespeak:unpack 20211031232710-full-Tu4H6vOtxKoLLb9 Unpacking Restoring Bespeak 20211031232710-full-Tu4H6vOtxKoLLb9  > Lock and set status to unpacking  > Browsing RestoringData data    > Unpacking RestoringChunk data-0540e4d6-9d7f-4c84-a8d8-ca40764257d1: proceeding      * Copying parts to temp files        - 00001-B57XWKJQe5Xg1sd: /tmp/phpNnYCbZ        - 00002-PXHPeS6t6OXFwkP: /tmp/phpYqRSPW [...]                          

On your restoring indicate is marked as not packed you tin proceed to the restoring. Please annotation that:

• For each data pack, you will be able to choose the location of the extraction, (you lot tin can bypass using --do-not-ask-data),
• If a sqldump is available, you will exist prompt to use the current configuration from the example, the i from the config/config.php freshly restored or use some other database (you can bypass using --exercise-not-ask-sql),
• If the data from the file config/config.php are in disharmonize with the path or sql settings specified during the extraction, you will be notified that the restoring procedure wants to update them:

              $ ./occ fill-in:point:restore 20211031232710-full-Tu4H6vOtxKoLLb9 Restoring Point: 20211031232710-full-Tu4H6vOtxKoLLb9 Date: 2021-ten-31 23:27:x Checking Health status: ok   WARNING! You are about to initiate the complete restoration of your case! All data generated since the cosmos of the selected backup will be lost...  Your case will come back to a previous state from 13 hours, 17 minutes and 48 seconds ago.  Practice you really want to continue this operation ? (y/Due north)                          

Restoring role of my Nextcloud

Bachelor occ commands:

Export/Import the configuration of your app.

It is mandatory to export the configuration of the app equally information technology contains the encryption keys for your encrypted backup and you will not exist able to restore your backups from a data lost.

You tin can exercise that from the Admin Settings page or using the occ command:

              ./occ backup:setup:consign [--key] > ~/fill-in.setup                          

This will create the file ~/backup.setup.
When using the selection --key the setup will be encrypted and an encryption_key volition exist generated and returned by the occ command. This key needs to exist stored somewhere and will be required to decrypt the saved configuration.
Information technology is strongly (again) brash to use the --key option.

To restore the exported configuration:

                              ./occ fill-in:setup:import [--key encryption_key] < ~/fill-in.setup                          

Information technology is mandatory to export the configuration of the app as it contains the encryption keys for your encrypted backup and you will not exist able to restore your backups from a data lost.

You tin do that from the Admin Settings folio or using the occ command:

              ./occ backup:setup:export [--key] > ~/backup.setup                          

This will create the file ~/backup.setup.
When using the option --cardinal the setup will be encrypted and an encryption_key will be generated and returned past the occ command. This key needs to be stored somewhere and will exist required to decrypt the saved configuration.
It is strongly (again) advised to utilize the --cardinal option.

To restore the exported configuration:

                              ./occ backup:setup:import [--key encryption_key] < ~/fill-in.setup                          

Manage your restoring indicate

Create a new Restoring Bespeak

While this is managed by a background task, yous can still generate a restoring bespeak manually:

              ./occ backup:indicate:create [--differential]                          

The --differential option will create an differential fill-in.

Upload a Restoring Point

              ./occ backup:point:upload <pointId>                          

This will asking all configured remote instances to check the sanity of any previous upload for this Restoring Point, and will only upload missing/faulty file.

List restoring points

You tin can search and compare restoring point bachelor locally and on configured remote instance.

Search for a specific file:

              ./occ fill-in:file:search [--since|--until|--point] <string>                          

Search for a file, based on its name.

Example: ./occ backup:file:search test.jpg --since 2021-09-23

History of specific a file:

              ./occ fill-in:file:history [--since|--until] <dataPack> <fullPath>                          

Display the history of a file.

Case: ./occ backup:file:history information cult/files/backup1.md

Import a Restoring Point

If you showtime using the app, you lot will face up at i point a situation where an important Restoring Point is available somewhere but cannot be find in your database. As an example, when restoring a Backup, all Restoring Point created after this backup won't be in database anymore. This is normal every bit restoring the backup fully overwrite your database. In that case, you tin can run this command:

              ./occ backup:betoken:scan <pointId>                          

If it cannot exist found, y'all volition demand to manually copy the binder that contains the Restoring Point in the appdata folder: information/appdata_qwerty123/backup/.

Verify integrity of a Restoring Point

              ./occ backup:point:details <pointId>                          

Exporting configuration

This is an important stride of your configuration of the Backup App Some information volition exist needed in instance you start storing your fill-in on remote instances:

• The identity of your Nextcloud,
• The encryption key used to encrypt your fill-in.

While the identity can be inverse and your access to the remote files can be restored past executing some control on the remote instance to update your new identity, a missing encryption key means that your remote backup cannot be decrypted and are totally useless.

Please note that creating a new identity will disable the sanity check on the metadata file.

              ./occ backup:setup:export [--fundamental] > ~/backup_setup.json                          

Using the --primal option will generate a Key, used to encrypt/decrypt the data of your setup. The key generated during the consign of your setup needs to exist stored somewhere safe!

              ./occ backup:setup:import [--key <cardinal>] < ~/backup_setup.json                          

Questions ?

- Tin can the app be used to migrate an instance of Nextcloud ?

Yes, during the restoration you tin can change the absolute path of your files and the configuration relative to the database.
No, you cannot switch the blazon of the database server (mysql, postgres, ...).

However, the app should not be used to indistinguishable setup in production as each case will be fully identical (instanceid, ...).

Known problems:

• When adding a new external storage from the files_external, the folder needs to be mounted using the Files App. Browsing the root folder should be enough.

defoorsomard.blogspot.com

Source: https://github.com/nextcloud/backup/blob/master/README.md

Share this post