Jolt.Testing : Assertions for XML Data Types

Note: This page describes the core XML assertions in the Jolt.Testing library. For a description of assertion adapters for a third party test framework, please visit this page.

As a basic feature, unit test frameworks must provide a set of assertion constructs that allow one to compare values of a given type. Support for intrinsic types is a neccesity, and as the framework develops, more complicated data types should be added; doing so makes the framework easeier to use by end-users. To illustrate, consider the NUnit and Microsoft's Visual Studio Test Edition frameworks. Both support assertions on intrinsic types as well as a declarative style of asserting the state of items in a collection (is subset of, are unique, etc...).

At the time of authoring this document, one feature that has not been prevalent in test frameworks is the ability to assert aspects of XML data. Existing libraries such as XmlUnit and Microsoft's XmlDiffPatch attempt to fill this gap, however they either target a specific unit testing framework, or provide output that is too detailed for the use through unit tests. Jolt XML assertions aim to overcome these limitations by providing a set of assertions and operations that are easy to use and which may be easily integrated with any third-party unit test framework.

Table of Contents

  1. Assertion Failure Reporting
  2. Design Scope
  3. Usage Examples
    1. Asserting the validity of an XML document
    2. Controlling the style of validation used by the validity assertion
    3. Asserting the equality of two XML elements
    4. Asserting the equivalency of two XML elements
    5. Customizing the equivalency strictness level used by equivalency assertion

Assertion Failure Reporting

The return value of an XML assertion function will contain enough context to determine the location of the failure in the XML document, along with a message that describes the reason for the failure. The XmlComparisonResult class implements this idiom, and extends from the general-purpose AssertionResult class. The following example demonstrates how XmlComparisonResult is used with the XmlEquivalencyAssertion class.

using System.Xml;
using System.Xml.Linq;
using Jolt.Testing.Assertions;
 
public class XmlEquivalencyAssertion
{
  public XmlComparisonResult AreEquivalent(XmlReader expected, XmlReader actual);
}
 
public class XmlComparisonResult : AssertionResult
{
  // The following two members are inherited.
  // public bool Result { get; }
  // public string Message { get; }

  public XElement ExpectedElement { get; }
  public XElement ActualElement { get; }
  public string XPathHint { get; }
}

Design Scope

The Jolt core XML assertion library is designed to support the following features.
  • Facilitates integration with many test frameworks by providing framework-agnostic assertions and operations
  • Simplify usage and implementation by always ignoring insignificant whitespace, processing instructions, and comments
  • Asserting XML validity
  • Asserting equality between two XML elements (or documents)
    • In the event of inequality, report the specific nodes that cause the assertion failure
  • Asserting user-defined equivalency between two XML elements (or documents)
    • Ignore namespaces
    • Ignore attributes
    • Ignore element values
    • Ignore element ordering
    • Any combination of the above
    • In the event of inequivalency, report the specific nodes that cause the assertion failure

Usage Examples


The XmlValidityAssertion class produces a list of all errors that occured during validation. If the produced list is empty, then the given document is considered valid. The following example demonstrates how to use the XmlValidityAssertion to assert the validity of an XML document.

using System.Collections.Generic;
using System.Diagnostics;
using System.Xml;
using System.Xml.Schema;
using Jolt.Testing.Assertions;
 
void ValidateXml()
{
  using (XmlReader xml = LoadXml())
  {
    XmlSchemaSet schemas = LoadXmlSchemas();
    XmlValidityAssertion assert = new XmlValidityAssertion(schemas);
 
    IList<ValidationEventArgs> validationErrors = assert.Validate(xml);
    Debug.Assert(validationErrors.Count == 0, validationErrors[0].Message);
  }
}


You may also control the style of validation by initializing the XmlValidityAssertion class with an XmlSchemaValidationFlags enumeration as follows. The default validation style only enables XmlSchemaValidationFlags.ReportValidationWarnings.

using System.Collections.Generic;
using System.Diagnostics;
using System.Xml;
using System.Xml.Schema;
using Jolt.Testing.Assertions;
 
void ValidateXml()
{
  using (XmlReader xml = LoadXml())
  {
    XmlSchemaSet schemas = LoadXmlSchemas();
    XmlSchemaValidationFlags validationStyle = XmlSchemaValidationFlags.ProcessInlineSchema | XmlSchemaValidationFlags.ReportValidationWarnings;
    XmlValidityAssertion assert = new XmlValidityAssertion(schemas, validationStyle);
 
    IList<ValidationEventArgs> validationErrors = assert.Validate(xml);
    Debug.Assert(validationErrors.Count == 0, validationErrors[0].Message);
  }
}


The XmlEqualityAssertion determines if two XML elements are equal, as defined by the following constraints.
  • The names of both elements are equal
  • The namespaces of both elements are equal
  • All text node values contained in both elements are equal and are in the same order
  • The set of attributes name/value pairs in both elements are equal (attribute ordering does not matter)
  • The number of child elements both elements are equal
  • If i is the position of a child element, then for all i : child elements at position i are equal

The following example demonstrates how to use the XmlEqualityAssertion to assert the equality of two XML elements. Note: XmlEqualityAssertion reports only the first detected inequality in the given XML elements.

using System.Diagnostics;
using System.Xml;
using Jolt.Testing.Assertions;
 
void AreEqual()
{
  using (XmlReader expectedXml = LoadExpectedXml(),
                   actualXml = LoadActualXml())
  {
    XmlEqualityAssertion assert = new XmlEqualityAssert();
    XmlComparisonResult areEqual = assert.AreEqual(expectedXml, actualXml);
 
    Debug.Assert(areEqual.Result, areEqual.Message + areEqual.XPathHint);
  }
}


The XmlEquivalencyAssertion determines if two XML elements are equivalent by relaxing some of the constraints of the XmlEqualityAssertion assertion. Any constraint may be relaxed except the constraints dealing with element name and child element quantity equality.

The following example demonstrates how to use the XmlEquivalencyAssertion for asserting the strictest definition of equivalency for two XML elements. Note: using the strictest definition yields the same behavior as the XmlEqualityAssertion.

using System.Diagnostics;
using System.Xml;
using Jolt.Testing.Assertions;
 
void AreEqual()
{
  using (XmlReader expectedXml = LoadExpectedXml(),
                   actualXml = LoadActualXml())
  {
    XmlEquivalencyAssertion assert = new XmlEquivalencyAssertion(XmlComparisonFlags.Strict);
    XmlComparisonResult areEquivalent = assert.AreEquivalent(expectedXml, actualXml);
 
    Debug.Assert(areEqual.Result, areEqual.Message + areEqual.XPathHint);
  }
}


The level of equivalency is defined through the use of the XmlComparisonFlags enumeration, which is a required constructor argument of XmlEquivalencyAssertion.

[Flags]
public enum XmlComparisonFlags
{
  Strict = 0x00,
  IgnoreElementNamespaces = 0x01,
  IgnoreAttributeNamespaces = 0x02,
  IgnoreAttributes = 0x04,
  IgnoreElementValues = 0x08,
  IgnoreSequenceOrder = 0x10
}

The following example demonstrates how to use the XmlEquivalencyAssertion for asserting a relaxed definition of equivalency for two XML elements. In this case, element ordering and attributes are ignored. Note: XmlEquivalencyAssertion reports only the first detected inequivalency (1) in the given XML elements.

using System.Diagnostics;
using System.Xml;
using Jolt.Testing.Assertions;
 
void AreEqual()
{
  using (XmlReader expectedXml = LoadExpectedXml(),
                   actualXml = LoadActualXml())
  {
    XmlComparisonFlags strictness = XmlComparisonFlags.IgnoreAttributes | XmlComparisonFlags.IgnoreSequenceOrder;
    XmlEquivalencyAssertion assert = new XmlEquivalencyAssertion(strictness);
    XmlComparisonResult areEquivalent = assert.AreEquivalent(expectedXml, actualXml);
 
    Debug.Assert(areEquivalent.Result, areEquivalent.Message + areEquivalent.XPathHint);
  }
}


[1] Many XML-comparison libraries utilize tree-diff algorithms to report all differences between two XML documents. Jolt XML assertions use a simplified algorithm to report only the first occuring different. For an analysis of tree-diff algorithms and the Jolt equivalency algorithm, please refer to "Error Reporting for Xml Equivalency Assertion".

Last edited Sep 2, 2010 at 2:58 PM by SteveGuidi, version 4

Comments

No comments yet.