eXist-db Database Recovery

For full documentation on backup, restore, consistency checks, and emergency export procedures, see https://exist-db.org/exist/apps/doc/backup.

Create (emergency) backup

With the "Emergency Export tool" (see https://exist-db.org/exist/apps/doc/backup#emergency-export-tool) it is possible to create a 'regular' backup of the database, but it can also handle many situations with a database corruption.

During the export process the database should not be running.

./bin/export.sh --direct --check-docs

By default a ZIP file is written into the 'export' directory. Please check ./bin/export.sh --help for available command-line options.

Alternatively, a backup can be done using the following command while the database is running. Replace {username}, {admin-password}, {path-to-destination-of-backup} and optionally also xmldb:exist://localhost:8080/exist/xmlrpc according your installation/settings, e.g. xmldb:exist://localhost:8877/exist/xmlrpc.

./bin/backup.sh --user {username} --password {admin-password} --backup /db --dir {path-to-destination-directory-of-backup} --optionuri=xmldb:exist://localhost:8080/exist/xmlrpc

Recovering from a corrupted index

To recover from a corrupted index, perform the following steps:

1. Stop the running eXist database instance

2. Change into directory $EXIST_HOME/data or another directory you specified as data directory in the configuration ($EXIST_HOME/etc/conf.xml).

3. Delete the following resources:

   • All ".dbx" files EXCEPT blob.dbx, collections.dbx, dom.dbx, and symbols.dbx
   • All ".lck" lock files
   • All ".log" transaction log files
   • The lucene and range directories

4. Trigger a reindex in order to reconstruct the secondary indexes, via the eXist-db XQuery function xmldb:reindex("/db"). This can be executed directly via the Java admin client, or, alternatively, on the command line via the following command:

   ./bin/client.sh --xpath "xmldb:reindex('/db')" --optionuri=xmldb:exist:///
   

   Possibly it is needed to specify username and password as well:

   ./bin/client.sh --xpath "xmldb:reindex('/db')" -u ... -P '...' --optionuri=xmldb:exist:///
   

Please check ./bin/client.sh --help for available command-line options, e.g., for supplying a username (-u option) and password (-P option).

WARNING

Please make sure that the file and directory ownership of the data directory regarding user existdb and group existdb is set properly. For example, executing the commands above as user root will spoil the user and group ownership of files and directories.

A suggested option is to chnage user and group ownership of files and directories in data by issuing the following command in the data directory

chown -R existdb:existdb *

Deleting all data in the database

DANGER

This really will delete the data in the database!

Please make a backup of the data directory.

To completely wipe a database — for example, in order to restore a backup onto a clean database — perform the following steps:

1. Stop the running eXist database instance
2. Please make a backup of the data directory
3. Delete all files and directories from the data directory

You can then continue with a restore of the data (see "Restoring from a backup").

Restoring from a backup

To restore the database files from a backup, you can use either the Java Admin Client or the backup command line utility. A backup file (or directory) can be efficiently restored via the command-line:

./bin/backup.sh --restore full20210925-1600.zip --dba-password <dba-password> --rebuild --optionuri=xmldb:exist:///

Note that for this command the database may not be running as a server.

Please check ./bin/backup.sh --help for available command-line options.

NOTE

Note that restoring from a backup (or parts of it) does not automatically delete all data in the database, nor does it necessarily require manually wiping the existing data using this procedure.

If a restore is performed on a database with data, the restore process will upload the collections and documents contained in the backup. Collections and documents which exist in the database but are not part of the backup will not be modified.

"Naked Restore"

You can combine a backup with the deletion of all data in the all data in the database and a restore to create a clean database into an empty data directory.

Backup the database, e.g. by using the dashboard.

This produces a file in the data/export directory of the exist home folder, e.g. in /usr/local/exist_atp/data/export with a name like with fullxxxxxxxxxx.zip, for example, full20231129-0100.zip.

Stop the database.

Go to the exist home folder, e.g. in /usr/local/exist_atp.

Move the new backup, e.g. /usr/local/exist_atp/data/full20231129-0100.zip to /usr/local/exist_atp

Move the folder /usr/local/exist_atp/data to a save place (if something goes wrong during restore) so now you have a /usr/local/exist_atp folder without a data folder.

Create a new data folder.

Start the "naked restore" by issuing the command line restore

./bin/backup.sh --restore full20231129-0100.zip --dba-password '...' --rebuild --optionuri=xmldb:exist:///

The zip is used to populate the empty data folder and to restore all data.

After the restore, please check whether the file and folder persmission are correct. In most cases the data folder shall have existdb user and existdb group as ownerships. You can set this by issuing the command

chown -R existdb:existdb data

After all this is done, start the database again.

konec