Migrating to Release 3 (or up)
NOTE
This page only applies to users that use code directly from our git repository that resides at Bitbucket.
This document is for Release 2 only.
Introduction
Release 2 is tied into a particular version of the database: eXist-db 2.2.
Release 3 runs on eXist-db 5.4.1, 6.0.1 and 6.2.0 at time of writing. Don't use eXist-db 6.1.0. Please see the Release 3 instructions for installing the newer database.
The eXist-db 2.2 database is not binary compatible and not code compatible with eXist-db >= 5. This means that it is not possible to just move the eXist-db 2.2 database files into the new eXist-db application, nor is it possible to do a simple backup/restore of the eXist-db 2.2 database contents.
Migration Steps
This instruction assumes you have the new database installed and have not committed any data to it as that will be lost upon restore.
Preparations on Server carrying Release 2
First, create full backup of your eXist-db 2.2 installation. See eXist-db documentation for further instructions and details.
Second, make a list of the packages you have installed in the package manager.
NOTE
Take special note of the terminology packages and contact us for instructions on SNOMED CT LOINC and other common packages.
Third, make sure you have the unzipped backup available on the Release 3 machine
Installation on Server carrying Release 3
Prepare Database
Shutdown the database, e.g.
systemctl stop eXist-db.service
Make a physical backup of the database directory, e.g. similar to the install instructions:
cd /opt/art-decor-linux/tooling/backups
VERSIONSHORT=6.0.1
tar zcvf exist_3_before_data_migration_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_*
Then disable any scheduled tasks in your Release 3 eXist-db conf.xml
by placing those in xml comments. Most notably:
<!--
<job type="user" name="scheduled-tasks" xquery="/db/apps/api/modules/library/scheduled-tasks.xql" period="15000" unschedule-on-exception="false"/>
-->
Start the database, e.g.
systemctl start eXist-db.service
Prepare Package Manager
Check /db/apps/packageservice/configuration.xml
. It should have one active repository in the repository section that points to the Release packages:
<repository>https://repository.art-decor.org/stable3</repository>
Using the package manager in the Dashboard install required and optional packages. The order of execution is relevant:
- Advanced Requirements Tooling
- DECOR core files
- Terminology Applications
- OID Registry tools
- ART-DECOR System Services
- ART-DECOR API (new for Release 3)
- any other packages using the list from your Release 2 environment
- any additional packages you may wish to install
Unveil the Export
First create a migration directory if it does not exist.
mkdir -p /opt/art-decor-linux/tooling/migration
Change to that directory.
cd /opt/art-decor-linux/tooling/migration
Get the export of the old Release 2 server that is typcially named like in this example: full20221029-1119.zip
and put it in /opt/art-decor-linux/tooling/migration
.
Unzip the export. After unzip is complete you typically have a folder call db
which carries all backed up files unpacked. You should see the following files and directories.
full20221029-1119.zip
db
Restore Accounts and Groups
Restore all user accounts and groups from the backup. See eXist-db documentation for further instructions and details.
NOTE
Before this step you need to use your admin password of the new server.
After this step the admin password will be equal to your "old" Release 2 installation.
Corresponding error messages may appear ("org.apache.xmlrpc.client.XmlRpcHttpTransportException: HTTP server returned unexpected status: Unauthorized"). After using the correct Release 2 admin password, all further steps should work properly.
The path in the backup is
/db/system/security/__contents__.xml
For the restore you must preceed this path with the absoulte pathname on your server. The restore can be achieved by issuing a command similar to the following one. Note that your working directory is migration directory created above, so absolute paths are needed.
/usr/local/exist_atp/bin/backup.sh -r /opt/art-decor-linux/tooling/migration/db/system/security/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u {administrator} -p {password}
You should see something similar like this output.
Creating Zip of restore data (uncompressed=101 KB)...
Finished creating Zip of restore data.
Transferring restore data to remote server (size=50 KB)...
Finished transferring restore data to remote server.
...
HTTP server returned unexpected status: Unauthorized <-- temporary error message (see above)
Restore Data Collections
Restore all data collections from the backup. Do not to overwrite newer packages during restore. You may not have some of the collections mentioned below, e.g. fhir-data
or xis-data
, when you never used any of the FHIR servers or the XIS package. Don't worry about those.
The paths in the backup are
/db/apps/art-data/__contents__.xml
/db/apps/decor/data/projects/__contents__.xml
/db/apps/decor/history/__contents__.xml
/db/apps/decor/releases/__contents__.xml
/db/apps/ada-data/__contents__.xml
/db/apps/fhir-data/__contents__.xml
/db/apps/temple-data/__contents__.xml
/db/apps/tools/oids-data/__contents__.xml
/db/apps/xis-data/__contents__.xml
This can be achieved by issuing several commands similar to this one:
/usr/local/exist_atp/bin/backup.sh -r /opt/art-decor-linux/tooling/migration/db/apps/decor/data/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u {administrator} -p {password}
For convenience reasons you might want to install the following little shell script upl.sh
that prints out all necessary commands after setting the first four variables properly. Note that it does not really upload the files but rather prints the commands. You have to copy&paste all or one by one of the printed commands and execute them. You could put the script and also the database export in /opt/art-decor-linux/tooling/migration
.
#!/bin/sh
# short script to print out uploading commands for migration from ART-DECOR Release 2 to 3
# see also https://docs.art-decor.org/release2/migration/
# please set these four variables properly
PATH2BAKDB=/opt/art-decor-linux/tooling/migration/db
USER='...'
PASSWORD='...'
BIN=/usr/local/exist_atp/bin/backup.sh
# no changes needed below this line
OBJ=(art-data decor/data/projects decor/history decor/releases ada-data fhir-data temple-data tools/oids-data xis-data)
echo "-----"
for f in "${OBJ[@]}"
do
echo ${BIN} -r ${PATH2BAKDB}/apps/${f}/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u ${USER} -p "'"${PASSWORD}"'"
done
echo "-----"
NOTE
The list above purposefully does not contain any terminology data collections: these should be installed through the package manager rather than restore.
Re-install Terminology Packages
Consider to re-install all needed Terminology Packages. These should be installed through the package manager.
NOTE
The Terminology Packages for ART-DECOR Release 2 are different from those for Release 3. You might end up with installing a Terminology Packages for both environments if you want to use them in Release 2 and 3.
All ART-DECOR Release 3 are belonging to the Cenztralized ART-DECOR Terminology Services (CADTS), our naming convention is that all ART-DECOR Release 3 terminology packages has "CADTS" in their names to easily identify them.
Re-install OID Package
From the package manager (re)install OID Registry - HL7 International. The restore of oids-data
as mentioned above causes an override of the earlier installation. Uninstall – if relevant – and re-install fixes this problem.
Repair and Reindex
Finally you need to repair all repositories. Use eXistdb's eXide app to submit the following XQuery.
import module namespace repair="http://exist-db.org/xquery/repo/repair"
at "resource:org/exist/xquery/modules/expathrepo/repair.xql";
repair:clean-all(),
repair:repair()
We also recommend to re-index the database (again). Use eXistdb's eXide app to submit the following XQuery.
xmldb:reindex('/db/')
Fix ART Permissions
NOTE
This section applies to installations only that use ART-DECOR Release 2.
Logged in as a System Administrator use the Server Administartion / Server Management Menu and click on Fix ART permissions to fix access right settings.
Check correct Orbeon Version
NOTE
This section applies to installations only that use ART-DECOR Release 2.
If you have a duplex server (hybrid system with both ART-DECOR Release 2 and 3 together on one database), please emake sure that you have checked the correct "Orbeon 2019" Version. Log in as admin and choose Server Settings from the Application menu. Under "Orbeon Version" choose Orbeon 2019.x. You choice is stored immediately.

Re-establish scheduled tasks
Shutdown the database, e.g.
systemctl stop eXist-db.service
Re-enable any scheduled tasks in your Release 3 eXist-db conf.xml you placed in xml comments. Most notably:
<job type="user" name="scheduled-tasks" xquery="/db/apps/api/modules/library/scheduled-tasks.xql" period="15000" unschedule-on-exception="false"/>
Start the database again...
systemctl start eXist-db.service
...and enjoy.