Configuring Web Logic 10.3.5 and log4j for your Java Application

I recently spent several days trying to track this down why Apache log4j for my Java application was working when deployed to Oracle WebLogic Server 10.3.5. I am posting my findings here in the hopes of saving others some time. I found several blogs that talked about this, but the end result was that log4j was still not working correctly. There was always one step missing. A co-worker and I finally figured it out for both Linux and Windows.

1. Copy wllog4j.jar from your WebLogic server/lib directory and place it in your domain_root/lib folder.
2. Copy your version of log4j.jar to your domain_root/lib folder.
3. Copy your log4j.xml to your domain_root folder.
4. Log in to the your WebLogic admin server console. Click on Servers -> Admin Server -> Logging. Click on advanced mode, and change the logging implementation from JDK to Log4J. Save your changes.

Most of the blogs had these items, but the one critical piece that was missing:
5. (Linux)Edit the setDomainEnv.sh file in the domain_root/bin directory. and find the following code:

if [ "${LOG4J_CONFIG_FILE}" != "" ] ; then
       JAVA_PROPERTIES="${JAVA_PROPERTIES} -Dlog4j.configuration=file:${LOG4J_CONFIG_FILE}"
       export JAVA_PROPERTIES
fi

Insert the following above those lines of code(replacing the path with the one for your system's domain_root directory:

LOG4J_CONFIG_FILE=”/u01/app/oracle/middleware/user_projects/domains/base_domain/log4j.xml”
Export LOG4J_CONFIG_FILE

5. (Windows)
Edit the setDomainEnv.cmd file in the domain_root/bin directory. and find the following code:

if NOT "%LOG4J_CONFIG_FILE%"=="" (                                                          
            set JAVA_PROPERTIES=%JAVA_PROPERTIES% -Dlog4j.configuration=file:%LOG4J_CONFIG_FILE%
)

Insert the following above those lines of code(replacing the path with the one for your system's domain_root directory:

set LOG4J_CONFIG_FILE=”C:\Oracle\Middleware\user_projects\domains\base_domain\log4j.xml”

6. Activate the changes and restart the admin server.

Here is a sample log4j.xml file. Obviously, you will need to update it for a valid location for your file appender, and replace yourcompany and yourproject with valid values.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="false">
   <!--====================================================================-->
   <!--    Appenders     -->
   <!-- %d{dd-MMM-yy HH:mm:ss} %-5p [%c{1}] %m%n  -->
    <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
        <param name="Target" value="System.out"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="[%p] - %m (%F:%M:%L)%n"/>
            <!--param name="ConversionPattern" value="%d{ISO8601} %p [%c{1}] - %m (%F:%M:%L)%n"/-->
        </layout>
    </appender>

    <appender name="FILE" class="org.apache.log4j.RollingFileAppender">
        <param name="File" value="/temp/logs/yourproject.log"/>
        <param name="Append" value="true"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d{ISO8601} %t %-5p %c{2} - %m%n"/>
        </layout>
    </appender>

    <!--====================================================================-->
    <!--    Logging Levels     -->

    <category name="com.yourcompany.yourproject">
      <priority value="DEBUG"/>
    </category>

    <category name="org.apache">
        <priority value="WARN"/>
    </category>

    <category name="org.springframework">
        <priority value="WARN"/>
    </category>

    <!--===================================================================-->
    <!--    Appender Assignments     -->
    <root>
        <priority value="ERROR"/>
        <appender-ref ref="FILE"/>
    </root>
</log4j:configuration>

Again, I hope that this will save someone the pain I went through to track this down.

Exploring Spring Integration & Spring Web Service, Part 1

I have been learning Spring Integration and Spring Web Services on the project I am on. The more I learn the more I like the framework, but I have to say it has not been easy to make progress. The number of resources available is lacking. I thought I would spend the next several blogs covering some of the information I have been learning.

I needed to do was expose four SOAP Web Services based on a set of WSDL and XSD files provided by the vendor. First thing that had to happen was setting up the web.xml file. From reading the documentation on the main Spring site, I determined to use MessageDispatcherServlet.

web.xml

<web-app>
   <display-name>My Application Name</display-name>
   <servlet>
       <servlet-name>si-ws-gateway</servlet-name>
       <servlet-class>org.springframework.ws.transport.http.MessageDispatcherServlet</servlet-class>
       <init-param>
           <param-name>contextConfigLocation</param-name>
           <param-value>classpath:si-ws-gateway-config.xml</param-value>
       </init-param>
       <load-on-startup>1</load-on-startup>
   </servlet>
   <servlet-mapping>
       <servlet-name>si-ws-gateway</servlet-name>
       <url-pattern>/*</url-pattern>    
   </servlet-mapping>
</web-app>

Next, let's take a look at the spring application context file.
si-ws-gateway-config.xml

    <bean class="org.springframework.ws.server.endpoint.mapping.UriEndpointMapping">
        <property name="mappings">
            <props>
                <prop key="${data.subscriber.url}">dataGateway</prop>
                <prop key="${request.status.url}">statusGateway</prop>
        </property>
    </bean>

    <si:channel id="dataArrivedChannel" />
    <si-ws:inbound-gateway id="dataGateway" request-channel="dataArrivedChannel" />
    <stream:stdout-channel-adapter id="stdoutAdaptorWithDefaultCharsetDataArrived" channel="dataArrivedChannel"/>

    <si:channel id="statusChangedChannel" />
    <si-ws:inbound-gateway id="statusChangedGateway"
                           request-channel="statusChangedChannel"
                           marshaller="marshaller"
                           unmarshaller="marshaller"/>
    <si:service-activator input-channel="statusChangedChannel"
                          ref="statusServiceEndpoint"
                          method="processStatusMessage"/>   

    <bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
        <property name="classesToBeBound">
            <list>
                <value>com.ourvendor.StatusUpdate</value>
            </list>
        </property>
    </bean>

    <bean id="StatusDao" class="com.company.StatusDao"/>

    <bean id="statusServiceEndpoint"
          class="com.company.StatusServiceEndpoint">
        <constructor-arg ref="statusManager"/>
    </bean>

    <bean id="statusManager"
          class="com.company.StatusManagerImpl">
        <constructor-arg ref="statusDao"/>
    </bean>

Well that seems like a lot of spring configuration, but it eliminates a bunch of Java code. Let's break down this configuration a bit. First, the UriEndpointMapping bean. You might notice that there are some values provided from a properties file. I will be covering properties in more detail in a future blog. For now, just know spring replaces the ${data.subscriber.url} and ${request.status.url} with a valid URL value from a properties file. This defines which URLs will be routed to the specified spring integration web services inbound-gateway. The gateway will place the SOAP message onto the specified channel as a DOMSource object.

As seen above the dataGateway does not specify a marshaller and unmarshaller which is why the DOMSource object is placed on the channel. The statusChangedGateway does specify the marshaller and unmarshaller, so when it places the object on the channel, it is not a DOMSource object, but a StatusUpdate object. As you can see it is very easy to leave the input in it's native format or convert it to a Java Object. In one case the object was only going to be routed(eventually) to a JMS queue, and did not need to be unmarshalled, in the other case it required unmarshalling so that a service activator endpoint could save the object to the database. As we will see in a future blog, it would have been nice if there was a way to specify the output to the channel as a string, instead of DOMSource. This would be a great enhancement to spring integration.

The service activator is an endpoint that allows a specific method to be called, and passes the object from the incoming channel to the method. It can also place the object, either unmodified or modified, onto an outgoing channel to be further processed by the system. The rest of the configuration is very straightforward spring beans with dependency injection. This works very well if each of the web services have a unique URL, but what happens if all calls come into a single URL. That will be the topic of the next blog in this series.

Accessing External Properties Files in WebLogic

It is not only common to do, but is also a very good practice to put values that may need to change into a properties file. This creates a central location to modify values. Especially for the configuration a system based on the installation location, etc. Having the properties file inside the .war is very inconvenient when you want to change it as it causes you to recompile your code, even if it hasn't changed. I was on a project that was able to pull the property file out of the .war and put it in the JBoss AS conf directory. The application could still access it. Unfortunately the project I am on now uses WebLogic. After lots of google searching and asking a few co-workers if they new how to do it, I did find the answer.

Simply place the properties file in $WEBLOGICHOME/user_projects/domains/mydomain where mydomain is the name of he domain your app uses.

Works great and now I can change properties all day long without recompiling.

Remote Desktop & Change Password

I ran into a problem where I needed to change the password on an account that was on a server half way around the world. All I could do was Remote Desktop in to that box, and when I did Ctrl + Alt + Delete, it was changing the password on the box I was physically at, not the one in the RD session. After some searching I found this solution: Ctrl + Alt + End.

Hope this helps someone.

Java JDBC Performance vs. iBatis

While working on a recent project, I worked with a database that contained 23 billion rows in one table, and several hundred million in another table. Performance retrieving information was critical for success. I have used Spring and iBatis for several years now and only had a couple of weeks to develop a prototype, so implemented the database access layer using iBatis and spring. It is quick to implement and easy to maintain. We had several use cases we wanted to test. Some use cases returned as few as one row, some returned as many as 20 million. Unfortunately, it was taking too much time to retrieve the information for most of the use cases. One downside of using iBatis, was that I couldn't determine if the time was spent in the database query, returning the results to Java, or in creating the java objects themselves. I decided to modify the code to use straight JDBC calls so that way I could time the retrieval separately from converting to Java Objects.

I did not expect to see much performance improvement converting to straight JDBC. However, just converting to straight JDBC was a substantial performance improvement over using iBatis(30-40% faster). Unfortunately this still wasn't enough. This was a web application being deployed to WebLogic. After some research a co-worker found that the default number of rows retrieved by the WebLogic data source was 10. Queries whose results were less than 10, performed very well, but most queries returned thousands or millions of rows, so performance was very poor. We initial bumped that value to 1000, and saw another 40+% performance improvement. This in turn prompted some investigation of JDBC, and I found that the default value for JDBC against an oracle database was 10. Inside the code I increased the fetch rows value to 1000, and saw another 30% performance improvement. I concluded that more was better on the fetch rows setting, so I increased it to 50,000 and found that I now had a new problem: running out of memory. After some additional modifications to the memory available to the web application, and reducing the fetch count to 5000, I was able to get a reliable result. Essentially, a higher fetch count, meant less database connections to retrieve the data.

Using iBatis is a great way to get a lot done in a short amount of time, and for smaller databases I still prefer to use it. In addition to being easy to implement, it is also easy to maintain. However, if you have a large database, then using Straight JDBC might be a good solution. Don't forget to take fetch size into account both in the application server and in the Java JDBC code when dealing with large databases. The larger the fetch size, the fewer database connections that will be used, but the more memory that will be required. I did not investigate setting the fetch size in iBatis, so do not know if that is possible.

In the Java JDBC code setting the fetch size is done as follows:

            String query = "SELECT * FROM my_table";
            Connection conn = _dataSource.getConnection();            
            PreparedStatement st = conn.prepareStatement(query);
            st.setFetchSize(5000);
            ResultSet rs = st.executeQuery();

Some other Java improvements that helped with performance, was reducing the number of java objects being created and destroyed. This includes making any strings that are reused static member variables so they are only created once and reducing the number of java objects that are being created. If you are outputting to a file or or a message, perform those operations inline to reduce creating intermediate objects.

Jersey and Spring Injection

On the project I am on we use iBatis, Spring, and Jersey. We have been using spring injection for our DAOs since the beginning, but our version of Jersey at the time did not support spring injection. To get around this we created factories allowing us to inject beans into the factories. Our REST classes then used the factory to get the beans they were interested in. As the project grew, so did the factory. When we upgraded to Jersey 1.0.3, we were able to take advantage of a Jersey-Spring jar, that would allow us to spring inject beans directly into our REST classes.

In our Maven pom.xml file we added the following dependency.

        <dependency>
            <groupId>com.sun.jersey.contribs</groupId>
            <artifactId>jersey-spring</artifactId>
            <version>1.0.3</version>
        </dependency>

Then in the constructor of the REST Classes we were able to do the following: (NOTE: as of Jersey 1.4, @Inject becomes @InjectParam)

public class CompanyWS {
    private ICompanyManager _companyManager;
    private ICompanyTypeManager _companyTypeManager;
    public CompanyWS (@InjectParam("companyManager") ICompanyManager companyManager,
                      @InjectParam("companyTypeManager") ICompanyTypeManager companyTypeManager) {
        _companyManager= companyManager;
        _companyTypeManager= companyTypeManager;
    }
...

The web.xml file also needs to be modified to call out the SpringServlet, instead of the ServletContainer:

    <servlet>
        <servlet-name>REST Application</servlet-name>
        <servlet-class>com.sun.jersey.spi.spring.container.servlet.SpringServlet</servlet-class>
        <init-param>
            <param-name>javax.ws.rs.Application</param-name>
            <param-value>com.accenture.netcds.api.rest.RegisterResources</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

One downside of the 1.0.3 Spring-Jersey library is that the automatic finding of the REST classes does not work in websphere 6.1. This has been reported as a bug to Jersey and is supposed to be fixed in the more recent versions. As a result the REST classes need to be registered in an Application class.

public class RegisterResources extends Application {    
    public Set<Class<?>> getClasses() {
        Set<Class<?>> s = new HashSet<Class<?>>();        
        s.add(CompanyWS.class);        
        return s;
    }
}

Spring successfully injected the beans into the REST classes, allowing us to delete the factories we had been using that had grown quite large. Another simplification to make it easier to maintain the software over time.

Nested iBatis Result Mappings

Although, this is not an earth shattering iBatis discovery, it is a time saver, both in development and maintenance so thought I would share it.

Let's assume we have a class called CompanyType that contains type_id and type name and another class called Company that among other properties has a CompanyType object companyType included.

In the iBatis xml file the result map for CompanyType would look something like

<sqlMap namespace="CompanyType">
    <resultMap id="companyTypeResultMap" class="com.sample.CompanyType">
        <result column="company_type_id" property="typeId"/>
        <result column="company_type_nm" property="typeName"/>
    </resultMap>
...

In the ibatis xml file for Company the result map would look like:

<sqlMap namespace="Company">
    <resultMap id="companyResultMap" class="com.sample.Company">
        <!-- other company properties here -->
        <result column="company_type_id" property="companyType.typeId"/>
        <result column="company_type_nm" property="companyType.typeName"/>
    </resultMap>

The problem with this is that if something changes in company type, you have to know to change it in two locations. Makes it harder to maintain. Instead though you could do the following:

<sqlMap namespace="Company">
    <resultMap id="companyResultMap" class="com.sample.Company">
        <!-- other company properties here -->
        <result property="companyType" resultMap="CompanyType.companyTypeResultMap"/>
    </resultMap>

As you can see, now if the result map is updated in CompanyType, the changes are automatically reflected in the Company ibatis file. Hope you find this helpful. I know I did.