XML to Json

Converting XML to JSON without knowing the xsd schema is a really useful trick for storing data into mongo db.  I'm commenting this because although it is easy I don't want to forget!

Maven Dependency

    <dependency>
        <groupId>org.json</groupId>
        <artifactId>json</artifactId>
        <version>20151123</version>
    </dependency>

Conversion

    String data; // Xml as a string

    final JSONObject jsonObject = XML.toJSONObject(data);
    doc.put("data", BsonDocument.parse(jsonObject.toString()));
    doc.put("rawData", data);

JodaTime, XSD, XJC

For Java 1.7 and earlier JodaTime (http://www.joda.org/joda-time/) is a great replacement for the standard XML / XSD date time object that Java creates when using XJC.

From Java 1.8 onwards the date time support in Java is much better and is basically a JodaTime implementation (JSR310).

Specific Bindings

To use JodaTime create a bindings file which contains an XPath location to the correct element, such as,

    <jxb:bindings namespace="http://www.me.com/my/message" schemaLocation="my-message.xsd">

        <jxb:schemaBindings>

            <jxb:package name="com.me.my.message.generated" />

        </jxb:schemaBindings>

        

        <jxb:bindings node="//xs:complexType[@name='Message']/xs:sequence/xs:element[@name='PublishDateTime']">

            <xjc:javaType name="org.joda.time.DateTime" adapter="com.me.my.util.JodaDateTimeAdaptor" />    

        </jxb:bindings>

    </jxb:bindings>

Adaptor

The JodaDateTimeAdaptor will do the actual conversion and can be reused.

public class JodaDateTimeAdaptor

    extends XmlAdapter<String, DateTime>

{

    /**

     * Convert a DateTime to String.

     *

     * @param datetime DateTime to be converted.

     * @return The value of datetime formatted as per ISO8601, or the empty string if date is null.

     */

    @Override

    public String marshal(final DateTime datetime)

    {

        return datetime.toString();

    }

    /**

     * Convert a String representation of a date into a Date object.

     *

     * @param str String representation of a date/time, in the ISO8601. May be null or empty.

     * @return str converted to a Date object. If str is null or empty, null is returned.

     */

    @Override

    public DateTime unmarshal(final String str)

    {

        if (str == null || str.length() == 0)

        {

            return null;

        }

        return DateTime.parse(str);

    }

}

Global Bindings

Global bindings can also be used but these don't work when one XSD imports another and you want to use an adaptor on both.  You end up with a class name clash because the automatically generated Adaptors appear in a 

    org.w3._2001.xmlschema.Adaptor1.java

    org.w3._2001.xmlschema.Adaptor2.java

    ...

However, for a single xsd or single project this may work.  To use the global bindings instead of the specific ones use

    <!-- generateName allows the typed enums -->

    <jxb:globalBindings typesafeEnumMemberName="generateName" >

        <jxb:serializable uid="1" />

        <!-- use JODA-Time DateTime for xs:date -->

        <jxb:javaType name="org.joda.time.DateTime" xmlType="xs:date"

            parseMethod="com.me.my.util.JodaDateAdaptor.unmarshal"

            printMethod="com.me.my.util.JodaDateAdaptor.marshal"/>

        <jxb:javaType name="org.joda.time.DateTime" xmlType="xs:dateTime"

            parseMethod="com.me.my.util.JodaDateTimeAdaptor.unmarshal"

            printMethod="com.me.my.util.JodaDateTimeAdaptor.marshal"/>

    </jxb:globalBindings>

Spring WS Interceptor

Using the spring interceptor methods a SOAP message can be manipulated by adding or removing elements.

Create an interceptor and add it to the spring config
   
    <bean id="myInterceptor" class="com.me.MyInterceptor">
    </bean>
    
    <sws:interceptors>
    <ref bean="myInterceptor" />
    </sws:interceptors>

The interceptor class implements the EndpointInterceptor

    public class MyInterceptor implements EndpointInterceptor

This interface defines four methods,

    // Used to read and / or manipulate the request coming in the endpoint
    boolean handleRequest(MessageContext paramMessageContext, Object paramObject)

    // Used to read and / or manipulate the response coming out of the endpoint
    boolean handleResponse(MessageContext paramMessageContext, Object paramObject)

    // Used to read and / or manipulate any faults that have occurred
    boolean handleFault(MessageContext paramMessageContext, Object paramObject)

    // Called after the interceptor has finished.
    void afterCompletion(MessageContext paramMessageContext, Object paramObject, Exception paramException)

Java Soap Objects

The default java soap objects are from the java.xml.soap package,

    SOAPBody
    SOAPHeader
    SOAPMessage

These objects use the standard org.w3c.dom objects to manipulate the contents.  These objects can be obtained using the following,

    SaajSoapMessage message = (SaajSoapMessage) messageContext.getRequest();
    SOAPMessage soapMessage = message.getSaajMessage();
    SOAPHeader soapHeader = soapMessage.getSOAPHeader();
    SOAPBody soapBody = soapMessage.getSOAPBody();

Spring Soap Objects

The spring objects are in the package org.springframework.ws.soap,

    SoapBody

    SoapHeader
    SoapEnvelope
    SoapElement

and have only a few methods.  The Body and Header can be used to get a standard javax.xml.transform.Source or Result object.  These objects can be obtained using,

    SaajSoapMessage message = (SaajSoapMessage) messageContext.getRequest();

    SoapBody soapHeader = message.getSoapHeader();

    SoapBody soapBody = message.getSoapBody();

Reading using the Java Objects

Reading using the java objects is pretty easy. Using the java SOAPBody a node list can be obtained,

    SOAPBody soapBody = soapMessage.getSOAPBody();

    NodeList soapBodyNodeList = soapBody.getChildNodes();

    String soapBodyStr = null;

    for (int i = 0; i < soapBodyNodeList.getLength(); i++)

    {

        String localName = soapBodyNodeList.item(i).getLocalName();

    }

or other standard dom manipulation eg getElementByTagName("myTag").

Reading using the Spring Objects

Using the Spring soap header or body can really simply be read using a Transformer,

    SoapBody springSoapBody = message.getSoapBody();

    TransformerFactory factory = TransformerFactory.newInstance();

    Transformer transformer = factory.newTransformer();

    StringWriter stringWriter = new StringWriter();

    transformer.transform(springSoapBody.getPayloadSource(), new StreamResult(stringWriter));

The contents can then be obtained from the stringWriter using the toString().  Alternatively a DOMResult can be used,

    DOMResult result = new DOMResult();

    transformer.transform(springSoapBody.getPayloadSource(), result);

Adding using the Spring Objects

Added an object in easy using the transformer defined above,

    SoapHeader soapHeader = message.getSoapHeader();

    StringSource stringSource = new StringSource("<xmlToAdd>Hello</xmlToAdd>");

    transformer.transform(stringSource , soapHeader.getResult());

The header will now include the xml.  At the point above the xml is just a string so could be generated using JAXB to create complicated additional nested xml tags if necessary.

Soap UI

SoapUI is a brilliant tool for testing web services.  It can be downloaded from http://www.soapui.org/.

Property Transfers

SoapUI allows testing individual calls to a service but also allows multiple requests to be put together to form test cases.  For example, a test case creating and deleting would require the id generated by the first request to be passed to the delete request.  Soap UI's mechanism for doing this is with a Property Transfer.

    Right Click on the Test Steps -> Add Step -> Property Transfer

The transfer is done using XPath by default.  An example of a property transfer is below using the xml

CreateRequest
    <CreateRequest>
        <name>Andy</name>
        <data>123456</data>
    </CreateRequest>

CreateResponse
    <CreateResponse>
        <id>159</id>
        <name>Andy</name>
        <data>123456</data>
    </CreateResponse>

DeleteRequest
    <DeleteResponse>
        <id>159</id>
    </DeleteResponse>

Here the property transfer is between the CreateResponse and the DeleteRequest.  In SoapUI this will look like,

    Source: Create      Property: Response
    declare namespace myns='http://www.me.com/example';
    //myns:CreateResponse/id

    Target: Delete      Property: Request
    declare namespace myns='http://www.me.com/example';
    //myns:DeleteRequest/id

It is possible to include the whole <id>159</id> or to just transfer the value you can check the

    Transfer text content

Transfering a whole block of xml can be really powerful but the 'Transfer text content' does keep things simple and allows the transfer of the 'value' only.

Maven & xjc

Maven integration of xjc is available from a number of sources.  In this post I'll look at some examples of what is available from jvnet.

Documentation is available here: http://confluence.highsource.org/display/J2B/JAXB2+Basics+Plugins

Simple Example

    <plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <executions>
            <execution>    
                <goals>
                    <goal>generate</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <extension>true</extension>
           <schemaDirectory>src/main/resources</schemaDirectory>
           <schemaIncludes>
               <include>my_schema.xsd</include>
           </schemaIncludes>
            <generateDirectory>src/main/java</generateDirectory>
            <generatePackage>com.me.generated</generatePackage>
            <episode>false</episode>     
        </configuration>
    </plugin>

This simple example generates my_schema.xsd JAXB classes and puts them into src/main/java in the com.me.generated package.  This is pretty much the most simple example of this plugin.

Example With Catalogue File

    <plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <version>0.8.2</version>
        <executions>
            <execution>
                <goals>
                    <goal>generate</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <extension>true</extension>
            <generateDirectory>src/main/java</generateDirectory>

            <catalogs>
                <catalog>
                    <dependencyResource>            
                        <groupId>com.me.common</groupId>
                        <artifactId>xsd-project</artifactId>
                        <resource>common.catalog</resource>
                    </dependencyResource>
                </catalog>
            </catalogs>

            <episodes>
                <episode>
                    <groupId>com.me.common</groupId>
                    <artifactId>xsd-project</artifactId>
                </episode>
            </episodes>    

            <episode>true</episode> 

            <args>
                <arg>-XtoString</arg>
                <arg>-Xequals</arg>
                <arg>-XhashCode</arg>
            </args>
            <plugins>
                <plugin>
                    <groupId>org.jvnet.jaxb2_commons</groupId>
                    <artifactId>jaxb2-basics</artifactId>
                    <version>0.6.4</version>
                </plugin>
             </plugins>      
        </configuration>    
    </plugin>


This plugin will parse the available *.xsd files in the src/main/resources directory and create JAXB annotated classes in the src/main/java directory.  If an bindings.xjb is also present in the resources directory then this will be used.

generateDirectory - where to put the JAXB classes.
catalogs - These allow an xsd reference to be overridden and redirected locally or into a jar file.
episodes - This tells xjc to not create code for these xsd because it is already available.
episode - This tells xjc to generate an episode file for these xsds
args - optional arguments to add to the JAXB classes.  This requires the plugin that follows it.

Catalog File

A catalog file allows the xsd schema locations to be altered and redirected.  If a schema file has an import such as


    <xs:import namespace="http://www.me.com/common/config" />


Note there is no schemaLocation value.  This will fail the schema validation without a catalog file helping to tell xjc where the actual xsd source is.  A valid catalog file to do this would be

    PUBLIC "http://www.me.com/common/config" "maven:com.me.common:xsd-project!/config.xsd"

Here the first part of the catalog has the same namespace as the import namespace.  This is how they match. The second element can be a standard file location or, like here, a reference to an xsd in a jar which is a maven dependency.  It is only with the maven plugin that this maven:... reference will work.  Standard xjc does not recognise this notation. NOTE: there is a colon separating 'maven' from the groupId from the artifactId.  Putting a . instead will cause the catalogue redirect to not work and that can eat up time trying to find the reason why!!

Episode File

An episode file is just a standard bindings file.  It tells xjc not to create JAXB annotated classes for objects which may appear in an xsd (Not all the elements of the xsd need to be contained in an episode file although this would be a little unusual and require manually changing the episode file after generation).
This maven plugin can use episode files from other jars but can also be told to generate one using the

            <episode>true</episode> 

Common Errors

If you get a stack trace like this,

    com.sun.istack.SAXParseException2: SCD "x-schema::tns" didnt match any schema component

Then a schema is included in the episode file but never referenced.  If you can remove the episode file from the configuration then this will cure it.  Otherwise just create an empty schema which imports all the episode elements but does nothing with them.  I think this error is trying to be helpful and tell you when something is included and it doesn't need to be but the message isn't particularly clear.  Hopefully there may be an option to ignore this error in the future because an episode file may contain multiple schemas when you don't necessarily need them all.

XML Schemas / XSD

Here are a few simple memory joggers for how to create xml schema (xsd) files.

Namespaces

Namespaces are important parts of xsds.  They control the prefixes and help to remove ambiguity if there are two types which have the same name.  They are also import if you want to using binding files to create java classes in different packages for different files.

Normally a targetNamespace is defined for a schema along with a xmlns command which refers to the same targetNamespace.  The targetNamespace says that this xsd defines this namespace.  The xmlns is the same for all items but because it is the same as the targetNamespace it is satisfied by this file (it refers to itself so no import is needed).  The targetNamespace doesn't have to be included but makes it harder to extend or split up the schema later.


  <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" 
      xmlns:conf="http://www.mycompany.com/myapp/config" targetNamespace="http://www.mycompany.com/myapp/config">


Note that the conf and targetNamespace are the same.

ComplexType

ComplexType is very common.  It defines a link between one object that can have multiple children.  An example of a ComplexType is

    <xs:complexType name="person">
        <xs:annotation>
            <xs:documentation>
                The details about a person.
            </xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:element name="FirstName" type="xs:string" minOccurs="1" maxOccurs="1" />
            <xs:element name="Surname" type="xs:string" minOccurs="1" maxOccurs="1" />
            <xs:element name="Age" type="xs:int" minOccurs="0" maxOccurs="1" />
        </xs:sequence>
    </xs:complexType>


The complexType can then be referenced in another object

    <xs:element name="staff" type="person" minOccurs="0" maxOccurs="unbounded" />

and in xml the person would look like

    <Person>
        <FirstName>Fred</FirstName>
        <Surname>Bloggs</Surname>
        <Age>43</Age>
    </Person>

Enumeration

If an enumeration is being used the don't make it an anonymous one.  If this happens then the xjc parser cannot turn it into a Java enumeration but instead does some horrible string effort.  Therefore, always make enumerations proper types.


    <xs:simpleType name="YesNo">
        <xs:annotation>
            <xs:documentation>Yes or No enumeration</xs:documentation>
        </xs:annotation>
        <xs:restriction base="xs:string">
            <xs:enumeration value="YES" />
            <xs:enumeration value="NO" />
        </xs:restriction>
    </xs:simpleType>


Restrictions need to have a 'base' that is the type of all the enumerations that it includes.

Extension

One object can extend another to include extra values.  For example, the person complexType defined about in the person object can be extended,

    <xs:complexType name="ExtendedPerson" >

        <xs:annotation>

            <xs:documentation>More people details</xs:documentation>

        </xs:annotation>

        <xs:complexContent>

            <xs:extension base="person">

                <xs:sequence>

                    <xs:element name="Salutation" type="SalutationType" minOccurs="1" maxOccurs="1" />

                </xs:sequence>

            </xs:extension>

        </xs:complexContent>

    </xs:complexType>

Where SalutationType is another enumeration.  Once Java classes have been generated the ExtendedPerson class extends the Person class with salutationType being the only local variable.  So in xml

    <ExtendedPerson>
        <FirstName>Fred</FirstName>
        <Surname>Bloggs</Surname>
        <Age>43</Age>
        <SalutationType>MR</SalutationType>
    </ExtendedPerson>

Import

Sometimes it is useful to have an import from one schema into another.  This allows types to be used from different schemas.  Firstly include the xmlns for the right target into the schema header as follows,

    xmlns:com="http://www.mycompany.com/myapp/common" 

Then import a schemaLocation which tallies up to this namespace.  The import command is,

    <xs:import namespace="http://www.mycompany.com/myapp/common" schemaLocation="common.xsd"/>


Here the common.xsd must have a targetNamespace as the same as the namespace in the import statement, in this case 'http://www.mycompany.com/myapp/common'.

Binding Files

Binding files (.xjb) can be used to simplify the xjc command and create java objects in different target packages.  To use a binding file the different xsds must have different namespaces.  A binding file which allows different packages to be used could be,

    <jxb:bindings xmlns:jxb="http://java.sun.com/xml/ns/jaxb" version="2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" >

        <jxb:bindings namespace="http://www.mycompany.com/myapp/common" schemaLocation="common.xsd" >

            <jxb:schemaBindings>

                <jxb:package name="com.mycompany.myapp.common"/>

            </jxb:schemaBindings>

        </jxb:bindings>

    

        <jxb:bindings namespace="http://www.mycompany.com/myapp/config" schemaLocation="config.xsd" >

            <jxb:schemaBindings>

                <jxb:package name="com.mycompany.myapp.config"/>

            </jxb:schemaBindings>

        </jxb:bindings>

    </jxb:bindings>

The binding file can be referenced using the -b command in xjc.

GlobalBindings

It is possible to have a bespoke element that needs to be parsed in a particular way.  This can be done with a globalBinding such as,

    <jxb:globalBindings fixedAttributeAsConstantProperty="true">
        <jxb:javaType name="com.my.SpecialElement" xmlType="my:specialElement"
                      parseMethod="com.my.SpecialElementAdaptor.unmarshal"
                      printMethod="com.my.SpecialElementAdaptor.marshal">
        </jxb:javaType>
    </jxb:globalBindings>

This can also be used to format standard types such as dates, although this will be global across all the schemas and will result in xjc creating Adaptors in weird packages!

    <jxb:globalBindings fixedAttributeAsConstantProperty="true">
        <jxb:javaType name="java.util.Date" xmlType="xs:dateTime"
                      parseMethod="com.my.DateTimeAdaptor.unmarshal"
                      printMethod="com.my.DateTimeAdaptor.marshal">
        </jxb:javaType>
    </jxb:globalBindings>

SchemaBindings

In the situation where there are multiple schemas in a binding file this globalBinding is unlikely to be good enough (or even work at all).  Therefore, the binding can also be included in the schema part so that the binding only applies to one schema.

    <jxb:bindings namespace="http://www.my.com/common" schemaLocation="common.xsd" >
        <jxb:schemaBindings>
            <jxb:package name="com.my.common.generated"/>
        </jxb:schemaBindings>
        <jxb:bindings node="//xs:simpleType[@name='DeletionDateTime']">  
            <jxb:javaType name="java.util.Date"
  parseMethod="com.my.DateTimeAdaptor.unmarshal"
  printMethod="com.my.DateTimeAdaptor.marshal"/>  
        </jxb:bindings>
    </jxb:bindings>

This form uses an xPath to find the correct simpleType.

    node="//xs:simpleType[@name='DeletionDateTime']"

Note that // means search everywhere in the document for a xs:simpleType with name 'DeletionDateTime'.

Both the GlobalBindings and the SchemaBindings create Adaptors which are just named Adaptor#.  This can cause name clashes when you have a hierarchy of projects (jars) which means that you can end up not using the correct adaptor.

XJC Extension & Adaptors

A further option to allow a specific adaptor to be used is to specify the adaptor that you want to use.  This can be done using the xjc extension.  Firstly you need to include the namespace in the bindings file.
    <jxb:bindings  version="2.1" xmlns:jxb="http://java.sun.com/xml/ns/jaxb" 
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc">

Next you can then use the xjc options.

    <jxb:bindings namespace="http://com.my/data" schemaLocation="my-data.xsd">
        <jxb:schemaBindings>
            <jxb:package name="com.my.data.generated" />
        </jxb:schemaBindings>
        
        <jxb:bindings node="//xs:complexType[@name='_Name_']/xs:sequence/xs:element[@name='_Date_']">
            <xjc:javaType name="org.joda.time.DateTime" adapter="com.my.data.util.JodaDateAdaptor" />    
        </jxb:bindings>
    </jxb:bindings>

xjc

xjc is bundled with java and is used to parse xsd and xjb files to create JAXB annotated java classes for a particular xml schema.  Example commands are

To parse a single file

    xjc -d myapp/java -p com.mycompany.myapp config.xsd

To parse multiple files

    xjc -d myapp/java -p com.mycompany.myapp *.xsd

To parse multiple files using a binding

    xjc -d myapp/java -b myapp.xjb *.xsd

-d = The route directory to create the classes in

-p = The package from the route directory

-b = refer to a binding file

Dojo Topic

Dojo provides a really useful topic based messaging system that can be used to send messages between decoupled pieces of javascript.

require / define

To use the topic functionality just include

    "dojo/topic"

at the top of the javascript file in the require or define section

Subscribe / Publish

It is really easy to use this functionality.  Firstly subscribe to a topic where the topic has a particular string name and a function to call as a result of a message,


    topic.subscribe("/log/add", function(message) {
        console.log(message);
    });


Now everytime a publish is done to the /log/add topic the function will be called.

To publish to that topic

    topic.publish("/log/add", "the message");

The publish subscribe isn't limited to one variable and can be passed multiple values.

That's really all there is to it.  The functionality is incredibly useful for larger web apps and particularly where the app code has been split into different modules and not all modules know about each other.  It helps to keep the hierarchy correct.