Wednesday, February 21, 2018

Spring Integration and XSLT with parameters

I have had the opportunity to use Spring Integration to execute the XSLT on the projects I've been on. It is actually quite easy and very fast. It allows you to pass in properties to the XSLT, which can be very useful. For instance, maybe there is a database lookup that you need to perform to get a value, or perhaps there are values in the Headers that are needed in the transformed message.

When dealing with XML it is always a good idea to have an XSD the defines what the valid XML should look like. There should be an XSD for the incoming XML and another XSD for the transformed XML. It is good practice to perform XSD validation on both to ensure the messages that are being received and the messages that are being sent conform to the XSD specifications.

Let's start with the Spring Configuration. The following spring code was part of a chain, so it doesn't have the input and output channels explicitly defined. There are a few interesting parts that some may not be familiar with I'd like to point out. First, this code is making use of properties that get substituted at run time by a Spring PropertyPlaceholderConfigurer. Both of the property's below are paths, and the values are the relative path inside the class path.

The second interesting part is the parameter that is being passed in. It is accessing a value defined in a Java class, this could also just as easily call a Java method to return a value. Again the class name includes the relative path from the class path.

<int-xml:validating-filter schema-location="${my.relative.incoming.xsd.file.location}" throw-exception-on-rejection="true"/>
<int-xml:xslt-transformer          
    xsl-resource="${my.relative.xslt.file.location}" >
    <int-xml:xslt-param name="trackingId" 
        expression="headers.get(T(com.mycompany.package.MyClass).MESSAGE_ID)"/>
</int-xml:xslt-transformer>
<int-xml:validating-filter schema-location="${my.relative.outgoing.xsd.file.location}" throw-exception-on-rejection="true"/>

The above code will first validate the incoming XML against the incoming XSD. If it validates correctly it will then perform the XSLT transformation, and finally it will validate that the outgoing XML matches what is expected from the outgoing XSD. That's really all there is to it.

Some will wonder what happens if it fails either validation or the transformation. It will throw an error and the error handling that was defined for the flow will do what ever it was configured to do. Error handling is out of scope for this blog.

The below XSLT demonstrates how to receive the parameter into the XSLT, and how to use it in the XSLT. The line above the template match line, is where the variable is received by the XSLT. To use the variable you just access it using $trackingId.

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="no"/>
    <xsl:param name="trackingId" />
    <xsl:template match="/weatherDetail">
        <WeatherResponse>
            <Header>
                <Timestamp>
                    <xsl:value-of select="current-dateTime()" />
                </Timestamp>
            </Header>
            <Body>
                <externalId>
                    <xsl:value-of select="$trackingId" />
                </externalId>
            </Body>
        </WeatherResponse>
    </xsl:template>
</xsl:stylesheet> 

Tuesday, February 20, 2018

XSLT Templates and filtering based on elements

Sometimes when processing XML, there is a need to loop over a list. Templates are a very powerful way to accomplish this. Consider the following XML Response from a weather system. For, the sake of space am only including 3 of the hourly points, but you can easily imagine a whole days worth. There is also a second section that contains some daily weather totals with a type of CUM or AVG.

<weatherdetail>
   <icao>KJLN</icao>
   <startdatetime>2018-01-17-00.00.00</startdatetime>
   <enddatetime>2018-01-18-00.00.00</enddatetime>
   <weatherdata>
      <weatherdatalist>
         <hour>0</hour>
         <temp>20</temp>
         <relhumidity>0</relhumidity>
         <wind>5</wind>
         <precip>0.0</precip>
      </weatherdatalist>
      <weatherdatalist>
         <hour>1</hour>
         <temp>25</temp>
         <relhumidity>10</relhumidity>
         <wind>15</wind>
         <precip>0.1</precip>
      </weatherdatalist>
      <weatherdatalist>
         <hour>23</hour>
         <temp>10</temp>
         <relhumidity>0</relhumidity>
         <wind>5</wind>
         <precip>0.0</precip>
      </weatherdatalist>
   </weatherdata>
   <weathertotals>
      <dailytotal>
         <type>CUM</type>
         <parameter>precip</parameter>
         <value>0.50</value>
         <date>2018-01-17</date>
      </dailytotal>
      <dailytotal>
         <type>AVG</type>
         <parameter>precip</parameter>
         <value>0.05</value>
         <date>2018-01-17</date>
      </dailytotal>
      <dailytotal>
         <type>AVG</type>
         <parameter>temp</parameter>
         <value>19</value>
         <date>2018-01-17</date>
      </dailytotal>
      <dailytotal>
         <type>AVG</type>
         <parameter>relHumid</parameter>
         <value>5</value>
         <date>2018-01-17</date>
      </dailytotal>
      <dailytotal>
         <type>AVG</type>
         <parameter>Wind</parameter>
         <value>12</value>
         <date>2018-01-17</date>
      </dailytotal>
   </weathertotals>
</weatherdetail>


Suppose one customer has the need receive a response as follows for graphing purposes. You need an easy way to transform the above XML into this.

<weatherresponse>
   </weatherresponse><header>
      <timestamp>2018-02-20T20:09:48.546Z</timestamp>
      <icao>KJLN</icao>
      <startdatetime>2018-01-17T00:00:00.000</startdatetime>
      <enddatetime>2018-01-18T00:00:00.000</enddatetime>
   </header>
   <body>
   <data>
         <hour>0</hour>
         <temperature>20</temperature>
         <humidity>0</humidity>
         <wind>5</wind>
         <precip>0.0</precip>
      </data>
      <data>
         <hour>1</hour>
         <temperature>25</temperature>
         <humidity>10</humidity>
         <wind>15</wind>
         <precip>0.1</precip>
      </data>
      <data>
         <hour>23</hour>
         <temperature>10</temperature>
         <humidity>0</humidity>
         <wind>5</wind>
         <precip>0.0</precip>
      </data>
   </body>

Let's look at the following XSLT.

There are two different types of templates in this XSLT. One is a named template, formatMyDate. This is a slick way to create essentially a method to do something. In this case, it converts the date format from the incoming format to the format required for the outgoing XML. Multiple parameters can be passed in to this method, but in this example only one is being passed in.

The second type of template is the main one being covered today. This one is applying a template to a specific XPATH, if your XPATH was further nested you would just provide the additional nodes, like: <xsl:apply-templates select="weatherData/additionalNode1/additionalNode2">

As you can see you can also pass parameters into this template call. I added that purely for demonstration as I am not really using the parameter for anything.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:template match="/weatherDetail">
        <weatherresponse>
            <header>
                <timestamp>
                    <xsl:value-of select="current-dateTime()">
                </xsl:value-of></timestamp>
                <icao>
                    <xsl:value-of select="icao">
                </xsl:value-of></icao>
                <startdatetime>
                    <xsl:call-template name="formatMyDate">
                        <xsl:with-param name="dateValue">
                            <xsl:value-of select="startDateTime">
                        </xsl:value-of></xsl:with-param>
                    </xsl:call-template>
                </startdatetime>
                <enddatetime>
                    <xsl:call-template name="formatMyDate">
                        <xsl:with-param name="dateValue">
                            <xsl:value-of select="endDateTime">
                        </xsl:value-of></xsl:with-param>
                    </xsl:call-template>
                </enddatetime>
            </header>
            <body>
              <xsl:apply-templates select="weatherData">
                 <xsl:with-param name="icao"> 
                    <xsl:value-of select="icao">
                 </xsl:value-of></xsl:with-param>    
              </xsl:apply-templates> 
           </body>
        </weatherresponse>
    </xsl:template>

    <!-- This is the template for hourly Processing -->
    <xsl:template match="weatherDataList">
       <xsl:param name="icao">
       <data>
          <hour>
             <xsl:value-of select="hour">
          </xsl:value-of></hour>
          <temperature>
             <xsl:value-of select="temp">
          </xsl:value-of></temperature>
          <humidity>
             <xsl:value-of select="relHumidity">
          </xsl:value-of></humidity>
          <wind>
             <xsl:value-of select="wind">
          </xsl:value-of></wind>
          <precip>
             <xsl:value-of select="precip">
          </xsl:value-of></precip>
       </data>
    </xsl:param></xsl:template>
    
    <xsl:template name="formatMyDate">
        <xsl:param name="dateValue">
        <xsl:variable name="datePart" select="substring($dateValue, 1, 10)">
        <xsl:variable name="timePart" select="translate(substring($dateValue, 12), '.',  ':')">
        <xsl:value-of select="concat($datePart,'T', $timePart,'.000')">
    </xsl:value-of></xsl:variable></xsl:variable></xsl:param></xsl:template>

</xsl:stylesheet> 

That was pretty slick. Now let's suppose there is another customer who wants to process the daily values, but only ones that are of type AVG. How do we filter on an element in the XML we are trying to process? Check this out:


<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:template match="/weatherDetail">
        <weatherresponse>
            <header>
                <timestamp>
                    <xsl:value-of select="current-dateTime()">
                </xsl:value-of></timestamp>
                <icao>
                    <xsl:value-of select="icao">
                </xsl:value-of></icao>
                <startdatetime>
                    <xsl:call-template name="formatMyDate">
                        <xsl:with-param name="dateValue">
                            <xsl:value-of select="startDateTime">
                        </xsl:value-of></xsl:with-param>
                    </xsl:call-template>
                </startdatetime>
                <enddatetime>
                    <xsl:call-template name="formatMyDate">
                        <xsl:with-param name="dateValue">
                            <xsl:value-of select="endDateTime">
                        </xsl:value-of></xsl:with-param>
                    </xsl:call-template>
                </enddatetime>
            </header>
            <body>
                 <xsl:apply-templates select="weatherTotals">
                    <xsl:with-param name="icao"> 
                       <xsl:value-of select="icao">
                    </xsl:value-of></xsl:with-param>    
                 </xsl:apply-templates> 
           </body>
        </weatherresponse>
    </xsl:template>
  
    <!-- This is the template for AVG daily Processing -->
    <xsl:template match="dailyTotal[type='AVG']">
       <xsl:param name="icao">
       <data>
         <parameter>
            <xsl:value-of select="parameter">
         </xsl:value-of></parameter>
         <avg>
            <xsl:value-of select="value">
         </xsl:value-of></avg>
         <date>
            <xsl:value-of select="date">
         </xsl:value-of></date>
       </data>
   </xsl:param></xsl:template>
   
   <xsl:template match="dailyTotal[type!='AVG']">
   </xsl:template>

    
    <xsl:template name="formatMyDate">
        <xsl:param name="dateValue">
        <xsl:variable name="datePart" select="substring($dateValue, 1, 10)">
        <xsl:variable name="timePart" select="translate(substring($dateValue, 12), '.',  ':')">
        <xsl:value-of select="concat($datePart,'T', $timePart,'.000')">
    </xsl:value-of></xsl:variable></xsl:variable></xsl:param></xsl:template>

</xsl:stylesheet> 


The above XSLT generates the desired output:


<weatherresponse>
   <header>
      <timestamp>2018-02-20T20:26:53.832Z</timestamp>
      <icao>KJLN</icao>
      <startdatetime>2018-01-17T00:00:00.000</startdatetime>
      <enddatetime>2018-01-18T00:00:00.000</enddatetime>
   </header>
   <body>
      <data>
         <parameter>precip</parameter>
         <avg>0.05</avg>
         <date>2018-01-17</date>
      </data>
      <data>
         <parameter>temp</parameter>
         <avg>19</avg>
         <date>2018-01-17</date>
      </data>
      <data>
         <parameter>relHumid</parameter>
         <avg>5</avg>
         <date>2018-01-17</date>
      </data>
      <data>
         <parameter>Wind</parameter>
         <avg>12</avg>
         <date>2018-01-17</date>
      </data>
   </body>
</weatherresponse>

A easy way to validate your XSLT against some known XML is to use one of the free online XSLT testing tools.

Wednesday, September 26, 2012

Command Line SVN

I use SmartSVN for most of my SVN needs, such as update, checkout, checkin, diffing files, etc... It's a great tool with a lot of power. When it comes to creating branches though, the command line SVN comes in really handy. It is easy to install on Windows. Just go to the CollabNet site, and download the latest SVN command-line binaries. The installer is straight forward. Once it is done you can test it by opening a CMD window, and running svn ?. It should recognize SVN command and show a list of options that you can use.

Wednesday, April 25, 2012

Spring Integration Generic Request Routing

On a project I was on recently, I needed to create a test harness where a lot of requests would come in through a single URL and need to be handled appropriately. Spring Integration provides a very easy way to make this happen. First you need to make use of the generic marshaller I discussed in a previous blog. Next you need to define the payloadRootAnnotationMethodEndpointMapping Spring bean, this will enable Spring to route incoming requests based on their payload root method, based on annotations placed in the endpoint classes, which I will discuss more in a minute.


    <bean class="org.springframework.ws.server.endpoint.mapping.PayloadRootAnnotationMethodEndpointMapping" />

    <bean class="org.springframework.ws.server.endpoint.adapter.GenericMarshallingMethodEndpointAdapter">
        <property name="marshaller" ref="marshaller"/>
        <property name="unmarshaller" ref="marshaller"/>
    </bean>


It is amazingly simple with spring integration to use the endpoint annotations.

@Endpoint
public class MyExposedEndpoint {
    @PayloadRoot(localPart="MyEndpointMethod", namespace="http://www.company.com/mynamespace")
    public void processMyMethod(MyRequest myRequest) {
         // Insert code to process the request here.
    }
}

That's really all there is to it. The web.xml is set up the same as the first blog in this series.

Spring Integration Marshalling with JAXB

The project I was working on used SOAP based Web Services. The .wsdl and .xsd files that described the services we needed to integrate with were provided by the vendor. Often times I needed to access information in the SOAP responses, or I needed to easily build a SOAP request object. While you can work directly with the XML, it is much easier to use marshalling/unmarshalling techniques. Being a proof of concept I chose JAXB to do the marshalling/unmarshalling. There may be better choices depending on your needs and most likely spring integration supports it.

While it is possible to create by hand JAXB Java classes, it seems to make more sense to use the xjc tool and generate the classes automatically. I will not go into great detail on using this tool as there are other resources that do that. xjc was on my class path, so I was able to navigate to the directory containing the .xsd and .wsdl files. I created a directory there called temp and ran the following command on each of the files that I needed classes generated for, which would create all the classes I needed in the appropriate directory structure under temp.

xjc -d temp myVendorFile.xsd

Now that the files had been generated it was a matter of configuring spring to use them. There are several ways to go about this. You can create a generic marshaller that handles all classes, or you can create more targeted marshallers. If you are creating a generic one you might just set the id to marshaller, if you want more targeted the id name will reflect something more specific.

    <bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
        <property name="classesToBeBound">
            <list>
                <value>com.vendor.JAXBClass1</value>>
                <value>com.vendor.JAXBClass2</value>
                <value>com.vendor.JAXBClass3</value>
            </list>
        </property>
    </bean>

    <bean id="myServiceClient" class="com.company.project.ws.MyServiceClient">
        <property name="marshaller" ref="marshaller"/>
        <property name="unmarshaller" ref="marshaller"/>
    </bean>
the class MyServiceClient extends the spring WebServiceGatewaySupport class. This is an easy way to send requests and receive responses. Inside your class you can create the request object that you want sent using the JAXB ObjectFactory methods.
MyRequest myRequest = new ObjectFactory().createMyRequest(); 
Then you can set individual elements on that object, whatever they are. The request is sent by using a method from the Spring class. The SoapAction is not set automatically, so it needs to be set manually.
MyResponse myResponse = getWebServiceTemplate().marshallSendAndReceive(serviceUri, myRequest, new WebServiceMessageCallback() { public void doWithMessage(WebServiceMessage message) {
((SoapMessage)message).setSoapAction("MySoapAction");}});
That's really all there is to it. As you can see the above method automatically marshalls the request to XML and unmarshalls the response to the JAXB Java object. Also it is good to surround that method call with a try catch so that if there is a problem on the other end, your software can react appropriately.

Tuesday, April 24, 2012

Using Variable Substitution from Configuration Files in Spring

It doesn't take long in the development world to learn that hard coding property values is a bad idea.  Hard coded values will require the software to be rebuilt and redeployed if a change is required.   Moving values to configuration files so they can be modified in a central location is a first step.  A second step is moving the configuration file outside of the .war or .ear, so they can be modified at run time to avoid recompiling.  To read about how to externalize property files from your application see my earlier blog.

There are several places in the Spring configuration that can make use of variable substitution from a property file.  First things first. If you create a property file how do you access it from your spring configuration file?  Near the top of the configuration file add the following bean definition:


    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
         <property name="locations">
            <value>file:my-project.properties</value>
         </property>
    </bean>

If you need to have comments in the property file, start the line with a #.  Properties should be entered into the property file in the following format: property.name=property value, i.e.

my.favorite.search.engine=http://www.google.com

Properties can be referenced from value fields, as well as bean variables.

	<bean id="jmsQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
		<property name="jndiName" value="${my.project.testQueue}"/>
		<property name="jndiTemplate" ref="jndiTemplate" />
	</bean>
	<si-ws:outbound-gateway id="WebServiceGateway"
			 request-channel="jmsChannel"
			 reply-channel="replyChannel"
			 uri="${my.project.uri}"  />

It is also possible to access properties from spring annotations which can be very useful. The scheduled annotation is a good example.

@Scheduled(cron="${my.project.schedule.cron}")

Spring makes it very easy to leverage properties files for making systems easier to configure on the fly. There is no need to continue hard coding values that you or your client may want to change.

Wednesday, January 25, 2012

Configuring Web Logic 10.3.5 and log4j for your Java Application

I recently spent several days trying to track this down why Apache log4j for my Java application was working when deployed to Oracle WebLogic Server 10.3.5. I am posting my findings here in the hopes of saving others some time. I found several blogs that talked about this, but the end result was that log4j was still not working correctly. There was always one step missing. A co-worker and I finally figured it out for both Linux and Windows.

1. Copy wllog4j.jar from your WebLogic server/lib directory and place it in your domain_root/lib folder.
2. Copy your version of log4j.jar to your domain_root/lib folder.
3. Copy your log4j.xml to your domain_root folder.
4. Log in to the your WebLogic admin server console. Click on Servers -> Admin Server -> Logging. Click on advanced mode, and change the logging implementation from JDK to Log4J. Save your changes.

Most of the blogs had these items, but the one critical piece that was missing:
5. (Linux)Edit the setDomainEnv.sh file in the domain_root/bin directory. and find the following code:

if [ "${LOG4J_CONFIG_FILE}" != "" ] ; then
JAVA_PROPERTIES="${JAVA_PROPERTIES} -Dlog4j.configuration=file:${LOG4J_CONFIG_FILE}"
export JAVA_PROPERTIES
fi

Insert the following above those lines of code(replacing the path with the one for your system's domain_root directory:

LOG4J_CONFIG_FILE=”/u01/app/oracle/middleware/user_projects/domains/base_domain/log4j.xml”
Export LOG4J_CONFIG_FILE

5. (Windows)
Edit the setDomainEnv.cmd file in the domain_root/bin directory. and find the following code:

if NOT "%LOG4J_CONFIG_FILE%"=="" (
set JAVA_PROPERTIES=%JAVA_PROPERTIES% -Dlog4j.configuration=file:%LOG4J_CONFIG_FILE%
)

Insert the following above those lines of code(replacing the path with the one for your system's domain_root directory:

set LOG4J_CONFIG_FILE=”C:\Oracle\Middleware\user_projects\domains\base_domain\log4j.xml”


6. Activate the changes and restart the admin server.

Here is a sample log4j.xml file. Obviously, you will need to update it for a valid location for your file appender, and replace yourcompany and yourproject with valid values.


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="false">
<!--====================================================================-->
<!-- Appenders -->
<!-- %d{dd-MMM-yy HH:mm:ss} %-5p [%c{1}] %m%n -->
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[%p] - %m (%F:%M:%L)%n"/>
<!--param name="ConversionPattern" value="%d{ISO8601} %p [%c{1}] - %m (%F:%M:%L)%n"/-->
</layout>
</appender>

<appender name="FILE" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/temp/logs/yourproject.log"/>
<param name="Append" value="true"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %t %-5p %c{2} - %m%n"/>
</layout>
</appender>

<!--====================================================================-->
<!-- Logging Levels -->

<category name="com.yourcompany.yourproject">
<priority value="DEBUG"/>
</category>

<category name="org.apache">
<priority value="WARN"/>
</category>

<category name="org.springframework">
<priority value="WARN"/>
</category>

<!--===================================================================-->
<!-- Appender Assignments -->
<root>
<priority value="ERROR"/>
<appender-ref ref="FILE"/>
</root>
</log4j:configuration>


Again, I hope that this will save someone the pain I went through to track this down.