1. Overview

2. Schema Versioning

Most interoperability scopes and messaging requirements tend to evolve over time. Versioned schemas keep track of these evolutionary steps. This evolution however, if poorly managed, can quickly create compatibility issues hampering interoperability, or even bring the operational message exchanges to a grinding halt.

Consider this real-life scenario: Multiple organisations are exchanging messages which are based on one particular schema version. Then a new schema version becomes available allowing the exchange of additional data elements. Forcing all organisations to upgrade at once may be difficult, if not impossible. If the organisations upgrade asynchronously, new version messages with those additional data elements will not schema-validate at recipient organisations that have not upgraded their version yet. The roll-out of a new schema version requires a signifiant amount of management and coordination, even if the technical aspect of the schema change itself is simple.

This predicament is caused by tight coupling. Coupling is a term that describes the level of shared knowledge necessary by participants in a distributed computing exchange. There are various facets to coupling. Here, we focus on the coupling of schema versions. In a tightly coupled exchange, the participants must have detailed knowledge about the other party's deployed schema version in order to successfully exchange messages. In a loosely coupled exchange, the participants still have specific, but less detailed knowledge about each other's schema version.

Multiple concurrent schema versions across exchange participants are often inevitable. But how can we ensure continued interoperability without the cost and risks of tight coupling?

A clear schema versioning strategy is needed to govern backward compatibility, which in turn enables forward compatibility to establish loosely coupled schema versions.

This page will outline MXV's schema versioning strategy, which is directly derived from:

• OASIS UBL NDR 2.1 (*)

• Crane Softwrights' paper "Versions in UBL." Presented at The International Symposium on Versioning XML Vocabularies and Systems, Montréal, Canada, August 11, 2008

With special thanks to the authors of these documents.

_________________________

(*) UBL NDR 2.1: Please note that as of November 2012 the UBL NDR 2.1 document has not been published yet. At this stage, MXV emulates UBL NDR 2.1. A detailed comparison of UBL NDR 2.1 rules against MXV will be made available once the UBL NDR 2.1 document has been finalised.

MXV is implementing the same schema versioning strategy as defined by OASIS UBL NDR 2.1.

There are two types of schema versions:

• Major versions

• Minor versions

"UBL distinguishes between major versions and minor versions. Major versions are not backwards compatible. Minor versions do not break backwards compatibility. In other words, a document instance that validates against version 1 of the schema must also validate against version 1.1 of the schema, where version 1.1 is a minor version change based on version 1. However, the same document instances would not necessarily be valid against version 2 of the schema, where version 2 is a major version change. Versioning information is indicated both in the namespace URI and in the version attribute of the schema module." (Source: OASIS UBL NDR)

Version information can also be included in each document instance itself using the element cbc:UBLVersionID or cbc:MXVVersionID respectively.

UBL NDR 2.1 contains a set of rules governing the versioning approach. These rules cannot be violated.

Let's have a closer look at the implications of major and minor versions:

- Major versions

"As is widely accepted, a major version change is required when instances of the older version no longer validate against a schema. This implies that applications are obliged to be changed in order to recognize the vocabulary constructs it may already support. From the schema validity perspective, this means an application is unable to inspect the content of an old instance because it cannot validate the instance with the new schema." (Source: Crane Softwrights' "Versions in UBL")

- Minor versions

"New document types can be added to UBL simply by creating the definitions of new document elements and the children they use. Should there be a need for a new business objects, this can simply be added to the library without impacting any of the existing document types. Should there be a need to modify an existing business object, only additions that have optional cardinality can be made in order not to impact the schema validity of an instance of an existing document type." (Source: Crane Softwrights' "Versions in UBL")

"New business requirements for an existing document type may mandate a change to a business object already being used. As when modifying objects to accommodate new documents, such changes can be made in a minor UBL revision provided they are all made with an optional cardinality. Nothing that is mandatory can be added in a minor version specification, as instances of prevous minor versions would no longer be schema valid.

In this approach, for example, a system set up to validate version 2.7 instances will also validate a version 2.5 instance. Whether a user wishes to accept the business validity of a version 2.5 instance is an out-of-band business decision. Nevertheless, there is nothing in the system preventing it from being able to inspect the instance."  (Source: Crane Softwrights' "Versions in UBL")

The published UBL processing model validates instances of minor versions either equal to or older than the recipient's installed minor version:

Figure 1: The published processing model for UBL (Diagram courtesy of Crane Softwrights)

For example, if the system is set up to validate version 2.7 instances, it will validate all instance versions from 2.0 to 2.7.

MXV applies UBL's minor versioning approach and generates Windows command files supporting UBL's published processing model (figure 1).

"A receiving application is assumed to have been programmed to be aware of only those constructs of a particular version of UBL. It would therefore be deployed with the schemas for that UBL version and will typically employ validation of received documents in advance of acting on the semantics represented by the information structured and identified in the XML. The application receiving an instance of a later UBL version may find either unrecognized constructs or recognized constructs in unexpected places. For example, a UBL 2.5 application would not recognize constructs introduced by the schema for UBL 2.7.

Shown previously in figure 1, UBL 2.1 describes a two-step validation processing model. The first of these two steps confirms the structural and lexical constraints (the lexical being, effectively, the structure of the content) of the document. The second step confirms the value constraints being imposed by business requirements beyond the purview of the committee, but within the context of using UBL between trading partners. At either stage of validation, a failure indicates that the message is to be rejected, either because the document structure or value constraints have been violated.

This processing model for like-versioned UBL systems does not serve a UBL 2.5 application receiving a UBL 2.7 instance with unexpected content.

Figure 2 illustrates a customized processing model that supports forward compatibility of UBL instances. This augments the published processing model in figure 2.1.

Figure 2: A customized processing model supporting forward compatibility (Diagram courtesy of Crane Softwrights)

This customized processing model for the receiving system uses only that version of UBL schema supported by the receiving system, and does not involve any inspection of the XML instance in advance of validation. In this model an initial schema validation failure indication is recognized to possibly have been triggered by an instance using features added in a schema later than the version supported by the system. After such a failure, an instance pruning process takes away unknown constructs from the instance being validated. The resulting pruned instance can then be checked for schema validity. If successful, the pruned instance is passed to the second stage value validation.

As with the published model, passing value validation grants delivery of the instance to the application. In this model, a second piece of information accompanies the instance being passed to the application. The application can already assume that value constraints in the document are satisfied. An "initial pass/fail" indication tells the application that the instance it is working with satisfies the structure constraints in either an unmodified ("initial pass") or a modified ("initial fail") state.

An unmodified instance can be acceptable for business processing regardless of the stated version number found in the UBLVersionID element if all of the business objects found in the instance conform to the constraints of the application, unused additions in a later version notwithstanding. The application can use out-of-band decision making, including the UBLVersionID element as input, to accept or reject a modified instance for the purposes of doing business.

In both cases if the instance is delivered to an application, such an application relying on schema validity for inspecting instance content can successfully extract any information therein.

Considering the example above, a UBL 2.7 instance without constructs unrecognized by the UBL 2.5 schema would validate using the receiving application's schemas. The instance would be passed to the UBL 2.5-aware application untouched and with an "initial pass" indication. In this case that the instance is marked 2.7 is irrelevant. A UBL 2.7 instance with unrecognized constructs would fail to validate with the UBL 2.5 schema and would be passed to the application after being pruned to the UBL 2.5 subset and with an "initial fail" indication. In this case that the instance is marked 2.7 is relevant to the application and user deciding how to proceed.

In support of this customized processing model, instance pruning processes for UBL 2.0 are already publicly available [Crane Resources]. Such processes are programmatically derived from the UBL specification rather than being hand-coded. This approach gives the pruning process a higher level of assurance of accuracy than coding the filter by hand."  (Source: Crane Softwrights' "Versions in UBL")

MXV optionally generates Windows command files supporting the customized processing model for forward compatibility (figure 2). See next chapter for MXV pruning processes.

MXV applies and enforces UBL NDR versioning rules. This allows to programmatically derive MXV pruning processes from MXV models. An optional MXV plug-in (circled blue in figure 3) generates at the push of a button a version filter (marked 'F' in figure 2) for the selected document schemas. These pruning filters are activated in the generated MXV Windows command files. Both filters and command files are included in the published delivery packages, in case an implementing organisation opts to enable the customized processing model for forward compatibility.

Figure3: MXV Tools menu for a document schema model

MXV adheres to critical UBL NDR rules and thus inherits UBL's approach to versioning as well as backward and forward compatibility between minor versions.

All data exchange partners implementing an MXV developed schema and the published processing model in figure 1 automatically obtain backward compatibility between minor versions. In other words, a schema valid instance based on prior minor schema version can always be schema-validated against the newer schema version. A basic level of loose coupling minor schema versions is thus achieved.

The developer of an MXV schema has the option to generate forward compatibility filters for inclusion in the published delivery packages. Each data exchange partner implementing an MXV developed schema has the option to implement the provided forward compatibility filters in conjunction with the customized processing model in figure 2. In other words, an instance, which is based on a newer minor schema version, can be pruned back to the known constructs of the currently implemented minor version for a second schema validation attempt. If successful, the instance has now become schema-valid by eliminating the unknown constructs of a newer minor version. This optional, higher level of loose coupling minor schema versions is in fact easy to achieve.

As a result, operational message exchanges can continue without interruption even when minor schema versions are out of sync. The participating organisations do not have to upgrade simultaneously, and each organisation decides for itself whether incoming newer minor versions are to be processed or rejected. Figure 4 illustrates an example of exchanged instances based on heterogenous schema versions.

Figure 4: Loose coupling minor schema versions for heterogenous instance versions

Assume the following scenarios:

a) Company 2 sends an instance A based on schema version 2.4 to Company 1. At Company 1, the first validation fails, and the MXV filter attempts pruning it back to version 2.3. The second validation succeeds, and after passing value validation, the instance can be processed by Application 1.

b) Company 1 sends an instance A based on their schema version 2.3 to Company 2. At Company 2, the first validation succeeds, and after passing value validation, the instance can be processed by Application 2 and 3.

c) Company 1 sends an instance A based on their version 2.3 to Company 3. At Company 3, the first validation fails, and because the MXV filter is not installed, the instance  cannot be processed. Company 3 may opt to install MXV filters and the associated processing model.

d) At Company 3 Application 5 sends an instance A based on schema version 2.2. The receiving Applications 1, 2 and 3 at Company 1 and 2 can validate the version 2.2 against the backward compatible schema versions 2.3 and 2.4. The instance can be processed without the need for filtering.

e) At Company 3 Application 5 sends an instance C based on its version 3.6 to the internal Application 6. The first validation fails, and the MXV filter for version 3.4 attempts pruning it back to version 3.4. The second validation fails again, meaning there are issues beyond the scope of forward compatibility. The instance is rejected with an error sent to Application 5.

Note how Company 1 and 2 can process both incoming prior and newer minor versions, while Company 3 cannot process incoming newer external minor versions until MXV filters are enabled. 

Note how the lack of enabled filters at Company 3 affects certain incoming messages, but not outgoing messages, because filters need not running when sending out instances.