Migrating to Release 3 (or up)

NOTE

This page only applies to users that use code directly from our git repository that resides at Bitbucketopen in new window.

This document is for Release 2 only.

Introduction

Release 2 is tied into a particular version of the database: eXist-db 2.2.

Release 3 runs on eXist-db 5.4.1, 6.0.1 and 6.2.0open in new window at time of writing. Don't use eXist-db 6.1.0. Please see the Release 3 instructions for installing the newer database.

The eXist-db 2.2 database is not binary compatible and not code compatible with eXist-db >= 5. This means that it is not possible to just move the eXist-db 2.2 database files into the new eXist-db application, nor is it possible to do a simple backup/restore of the eXist-db 2.2 database contents.

Migration Steps

This instruction assumes you have the new database installed and have not committed any data to it as that will be lost upon restore.

Preparations on Server carrying Release 2

First, create full backup of your eXist-db 2.2 installation. See eXist-db documentationopen in new window for further instructions and details.

Second, make a list of the packages you have installed in the package manager.

NOTE

Take special note of the terminology packages and contact us for instructions on SNOMED CT LOINC and other common packages.

Third, make sure you have the unzipped backup available on the Release 3 machine

Installation on Server carrying Release 3

Prepare Database

Shutdown the database, e.g.

systemctl stop eXist-db.service

Make a physical backup of the database directory, e.g. similar to the install instructions:

cd /opt/art-decor-linux/tooling/backups
VERSIONSHORT=6.0.1
tar zcvf exist_3_before_data_migration_${VERSIONSHORT}_$(date '+%Y%m%d%H%M').tgz /usr/local/exist_atp_*

Then disable any scheduled tasks in your Release 3 eXist-db conf.xml by placing those in xml comments. Most notably:

<!--
<job type="user" name="scheduled-tasks" xquery="/db/apps/api/modules/library/scheduled-tasks.xql" period="15000" unschedule-on-exception="false"/>
-->

Start the database, e.g.

systemctl start eXist-db.service

Prepare Package Manager

Check /db/apps/packageservice/configuration.xml. It should have one active repository in the repository section that points to the Release packages:

<repository>https://repository.art-decor.org/stable3</repository>

Using the package manager in the Dashboardopen in new window install required and optional packages. The order of execution is relevant:

Advanced Requirements Tooling
DECOR core files
Terminology Applications
OID Registry tools
ART-DECOR System Services
ART-DECOR API (new for Release 3)
any other packages using the list from your Release 2 environment
any additional packages you may wish to install

Unveil the Export

First create a migration directory if it does not exist.

mkdir -p /opt/art-decor-linux/tooling/migration

Change to that directory.

cd /opt/art-decor-linux/tooling/migration

Get the export of the old Release 2 server that is typcially named like in this example: full20221029-1119.zip and put it in /opt/art-decor-linux/tooling/migration.

Unzip the export. After unzip is complete you typically have a folder call db which carries all backed up files unpacked. You should see the following files and directories.

full20221029-1119.zip
db

Restore Accounts and Groups

Restore all user accounts and groups from the backup. See eXist-db documentationopen in new window for further instructions and details.

NOTE

Before this step you need to use your admin password of the new server.

After this step the admin password will be equal to your "old" Release 2 installation.

Corresponding error messages may appear ("org.apache.xmlrpc.client.XmlRpcHttpTransportException: HTTP server returned unexpected status: Unauthorized"). After using the correct Release 2 admin password, all further steps should work properly.

The path in the backup is

/db/system/security/__contents__.xml

For the restore you must preceed this path with the absoulte pathname on your server. The restore can be achieved by issuing a command similar to the following one. Note that your working directory is migration directory created above, so absolute paths are needed.

/usr/local/exist_atp/bin/backup.sh -r /opt/art-decor-linux/tooling/migration/db/system/security/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u {administrator} -p {password}

You should see something similar like this output.

Creating Zip of restore data (uncompressed=101 KB)...
Finished creating Zip of restore data.
Transferring restore data to remote server (size=50 KB)...
Finished transferring restore data to remote server.
...
HTTP server returned unexpected status: Unauthorized <-- temporary error message (see above)

Restore Data Collections

Restore all data collections from the backup. Do not to overwrite newer packages during restore. You may not have some of the collections mentioned below, e.g. fhir-data or xis-data, when you never used any of the FHIR servers or the XIS package. Don't worry about those.

The paths in the backup are

/db/apps/art-data/__contents__.xml
/db/apps/decor/data/projects/__contents__.xml
/db/apps/decor/history/__contents__.xml
/db/apps/decor/releases/__contents__.xml
/db/apps/ada-data/__contents__.xml
/db/apps/fhir-data/__contents__.xml
/db/apps/temple-data/__contents__.xml
/db/apps/tools/oids-data/__contents__.xml
/db/apps/xis-data/__contents__.xml

This can be achieved by issuing several commands similar to this one:

/usr/local/exist_atp/bin/backup.sh -r /opt/art-decor-linux/tooling/migration/db/apps/decor/data/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u {administrator} -p {password}

For convenience reasons you might want to install the following little shell script upl.sh that prints out all necessary commands after setting the first four variables properly. Note that it does not really upload the files but rather prints the commands. You have to copy&paste all or one by one of the printed commands and execute them. You could put the script and also the database export in /opt/art-decor-linux/tooling/migration.

#!/bin/sh
# short script to print out uploading commands for migration from ART-DECOR Release 2 to 3
# see also https://docs.art-decor.org/release2/migration/

# please set these four variables properly
PATH2BAKDB=/opt/art-decor-linux/tooling/migration/db
USER='...'
PASSWORD='...'
BIN=/usr/local/exist_atp/bin/backup.sh

# no changes needed below this line
OBJ=(art-data decor/data/projects decor/history decor/releases ada-data fhir-data temple-data tools/oids-data xis-data)

echo "-----"
for f in "${OBJ[@]}"
do
  echo ${BIN} -r ${PATH2BAKDB}/apps/${f}/__contents__.xml -ouri=xmldb:exist://localhost:8877/exist/xmlrpc -u ${USER} -p "'"${PASSWORD}"'"
done
echo "-----"

NOTE

The list above purposefully does not contain any terminology data collections: these should be installed through the package manager rather than restore.

Re-install Terminology Packages

Consider to re-install all needed Terminology Packages. These should be installed through the package manager.

NOTE

The Terminology Packages for ART-DECOR Release 2 are different from those for Release 3. You might end up with installing a Terminology Packages for both environments if you want to use them in Release 2 and 3.

All ART-DECOR Release 3 are belonging to the Cenztralized ART-DECOR Terminology Services (CADTS), our naming convention is that all ART-DECOR Release 3 terminology packages has "CADTS" in their names to easily identify them.

Re-install OID Package

From the package manager (re)install OID Registry - HL7 International. The restore of oids-data as mentioned above causes an override of the earlier installation. Uninstall – if relevant – and re-install fixes this problem.

Repair and Reindex

Finally you need to repair all repositories. Use eXistdb's eXide app to submit the following XQuery.

import module namespace repair="http://exist-db.org/xquery/repo/repair"
at "resource:org/exist/xquery/modules/expathrepo/repair.xql";
repair:clean-all(),
repair:repair()

We also recommend to re-index the database (again). Use eXistdb's eXide app to submit the following XQuery.

xmldb:reindex('/db/')

Fix ART Permissions

NOTE

This section applies to installations only that use ART-DECOR Release 2.

Logged in as a System Administrator use the Server Administartion / Server Management Menu and click on Fix ART permissions to fix access right settings.

Check correct Orbeon Version

NOTE

This section applies to installations only that use ART-DECOR Release 2.

If you have a duplex server (hybrid system with both ART-DECOR Release 2 and 3 together on one database), please emake sure that you have checked the correct "Orbeon 2019" Version. Log in as admin and choose Server Settings from the Application menu. Under "Orbeon Version" choose Orbeon 2019.x. You choice is stored immediately.

Re-establish scheduled tasks

Shutdown the database, e.g.

systemctl stop eXist-db.service

Re-enable any scheduled tasks in your Release 3 eXist-db conf.xml you placed in xml comments. Most notably:

<job type="user" name="scheduled-tasks" xquery="/db/apps/api/modules/library/scheduled-tasks.xql" period="15000" unschedule-on-exception="false"/>

Start the database again...

systemctl start eXist-db.service

...and enjoy.

konec

Permalink | Updated 2023-03-24 | Visits: 872

# Migrating to Release 3 (or up)

# Introduction

# Migration Steps

# Preparations on Server carrying Release 2

# Installation on Server carrying Release 3

# Prepare Database

# Prepare Package Manager

# Unveil the Export

# Restore Accounts and Groups

# Restore Data Collections

# Re-install Terminology Packages

# Re-install OID Package

# Repair and Reindex

# Fix ART Permissions

# Check correct Orbeon Version

# Re-establish scheduled tasks

Migrating to Release 3 (or up)

Introduction

Migration Steps

Preparations on Server carrying Release 2

Installation on Server carrying Release 3

Prepare Database

Prepare Package Manager

Unveil the Export

Restore Accounts and Groups

Restore Data Collections

Re-install Terminology Packages

Re-install OID Package

Repair and Reindex

Fix ART Permissions

Check correct Orbeon Version

Re-establish scheduled tasks