Sunday, February 28, 2010

Bulk saving and updating using iBatis

Recently I had the need to save records to an association table. All the object values were the same except one. I was looping through the array list saving them one at a time, but kept getting unable to get database connection errors. I lamented that I couldn't do a bulk save. A co-worker thought that iBatis supported what I wanted to do, so after some investigation we were able to get both a bulk save and a bulk update to work.

For our example let us assume we have a table that contains a list of companies and another table that contains a list of Locations. From the client when you edit a company it allows you to multi-select a list of locations to associate with that company. On the back end you could loop through the locations and store each separately, but iBatis allows you to save them all at once. Let's examine how to do that.

Here are the Company and Location classes, minus getters/setters and imports;

package com.projectA;
public class Company {
private int companyId;
private String name;
private List locations;
}

package com.projectA;
public class Location {
private int locationId;
private String address;
private String city;
private String state;
private String zipCode;
private String phoneNumber;
}

The database contains a table containing all the companies, and a table containing all the locations. There is also a third table that contains the company - location associations. To do this we need to take advantage of the iBatis dynamic fields. Here is the iBaits XML file that contains the bulk save and update methods for the association table:

<sqlMap namespace="Company">

<typeAlias alias="company" type="com.projectA.Company"/>
<typeAlias alias="loation" type="com.projectA.Location"/>

<resultMap id="companyResultMap" class="company">
<result column="company_id" property="companyId" />
<result column="company_name" property="name" />
<result column="{companyId=company_id}" property="locations"
select="Company.getActiveLocations"/>
</resultMap>

<resultMap id="locationResultMap" class="location">
<result column="location_id" property="locationId" />
<result column="location_address" property="address" />
<result column="location_city" property="city" />
<result column="location_state" property="state" />
<result column="location_zip_code" property="zipCode" />
<result column="location_phone_number" property="phoneNumber" />
</resultMap>

<sql id="select_company_fragment">
SELECT company_id,
company_name,
FROM company
</sql>

<sql id="select_location_fragment">
SELECT cl.location_id,
loc.location_address,
loc.location_city,
loc.location_state,
loc.location_zip_code,
loc.location_phone_number
FROM company_location cl, location loc
WHERE cl.company_id=#companyId#
AND cl.location_id = loc.location_id
</sql>

<select id="getAllCompanies"
resultMap="companyResultMap">
<include refid="select_company_fragment"/>
ORDER BY company_name
</select>

<select id="getActiveLocations"
resultMap="int">
<include refid="select_location_fragment"/>
AND active_flag = 1
ORDER BY location_state, location_city
</select>

<insert id="bulkInsertCompanyLocation" parameterClass="map">
INSERT INTO company_location
(company_id,
location_id,
active_flag)
SELECT
#companyId#,
#activeFlag#,
loc.location_id,
FROM location loc LEFT JOIN company_location cl ON cl.company_id=#companyId#
AND cl.location_id = loc.location_id
WHERE cl.company_id IS NULL
<dynamic>
<isNotNull property="locations">
<iterate property="locations"
prepend="AND " open=" loc.location_id IN ("
close=")" conjunction=",">
#locations[]#
</iterate>
</isNotNull>
</dynamic>
</insert>

<update id="bulkUpdateCompanyLocation" parameterClass="map">
UPDATE company_location
SET
active_flag = #activeFlag#
WHERE company_id=#companyId#
<dynamic>
<isNotNull property="locations">
<iterate property="locations" prepend="AND " open=" location_id IN ("
close=")" conjunction=",">
#locations[]#
</iterate>
</isNotNull>
</dynamic>
</update>

</sqlMap>

The map that is passed into both the insert and update methods, contains the company_id, an active flag, and an array list of locationId's. These methods save or update multiple records in one shot. That's all there is to it. By changing to the bulk insert and update methods, the out of database connections errors went away.

The resulting SQL that iBatis generates looks like this:

INSERT INTO company_location
(company_id,
location_id,
active_flag)
SELECT
#companyId#,
#activeFlag#,
loc.location_id,
FROM location loc LEFT JOIN company_location cl ON cl.company_id=#companyId#
AND cl.location_id = loc.location_id
WHERE cl.company_id IS NULL
AND loc.location_id IN ("1","3","10")

UPDATE company_location
SET
active_flag = #activeFlag#
WHERE company_id=#companyId#
AND location_id IN ("4","6","12")

You can read more about the dynamic and iterate methods in the iBatis documentation.

12 comments:

Unknown said...

Hi Jeffrey,

Thanks for sharing such useful info about bulk updating in iBatis.

I am very new to iBatis, but i have worked on hibernate, an ORM framework.

Hence i would request you to help me in knowing few things about iBatis.

1. Can we print the final sql query
generated by iBatis at runtime? If so how?

2. Can we place a sql query between tags, and execute it as many times as no of objects in list being iterated?

for example:



UPDATE tableName SET column = #property# WHERE ID = #Id#



I am actually sending pojo objects in that list.

Any other approaches are also welcome.

Thanks in advance. I am keenly awaiting your response. :)

Unknown said...

Adding to the above comment, i had placed sql query inside iterate tags.

jlorenzen said...

@Priya
If you want to see the actual SQL that is ran, you could enable SQL logging. That is what I did in jboss. See http://stackoverflow.com/questions/2314667/log4j-different-category-priority-than-appender-threshold or here http://opensource.atlassian.com/confluence/oss/display/IBATIS/How+do+I+get+SqlMapClient+to+log+SQL+statements

Unknown said...

I am using ibatis for my current spring mvc project. And having an issue of accessing resultsets with multiple records using ibatis resultmap . It gives TooManyResultsException exception. So was wondering if your following xml snippet is actually working? As I also used the same synatx:
select id="getAllCompanies" resultMap="companyResultMap"> ORDER BY company_name select
Thanks,
Arpana

Unknown said...

Hi ,

I got the above issue resolved. The return type of mapper method should be a java.util.List. Earlier I had used java.util.Collection, and this was creating problem.

Regards,
Arpana

phillips4jc said...

Arpana, Glad you figured it out. I should probably change the example so that it is obvious. In the project code where I figured this out, we had multiple variables to pass in, so had to use a map. One of the parameters to that map was a java.util.List object.

Good luck.

Jeff

sriram said...

Hii Phillips,
I am new to Ibatis and I would be grateful if You can Answer my Question. I have two quick questions .
1) I have One complex object graph as following
Public class company{
private int companyId;
private List defaultAccounts;
}
public class Account
{
private int accountId;
private String accountName;
}
I need to insert company object into account table:(seqnum,companyid,accountid,accountname).So a company has 10 accounts ,10 records must be inserted. Whther this can be done in ibatis sqlmaps?

phillips4jc said...

Hi Sriram,

The best way to insert several of the items to the same table at once is using the batch capability of ibatis. I plan to blog on this, just haven't had a chance.

((SqlMapClientImpl) getSqlMapClient()).startBatch();
Then loop over your list doing
insert or update statements.

Then when you are done you can run
((SqlMapClientImpl) getSqlMapClient()).executeBatch();

This makes only one round trip to the database, instead of 10 in your example.

Hope that helps.

Jeff

Unknown said...

Hi Phillips,

What is the maximum number of records that can be inserted in batch insert in ibatis.

I get the following exception after around executing 1500 insert statements:

java.sql.SQLException: ORA-01000: maximum open cursors exceededjava.sql.SQLException: ORA-01000: maximum open cursors exceeded

Could you let me know why i am facing this issue?

phillips4jc said...

With out knowing more about your situation, I would suggest looking into your app server configuration. Perhaps the way it is configured is causing you to run out of resources.

7Legends said...

I have an array obect(eg. IList) in that i made same changes and i added new records now i want to save these to the database so how the xml statement descriptor of mine should be.

My Blog said...

Hi.....I need Dynamic Update Query Statement without entering column names..actually i passing column name and values in the list...Can u please help for this issue....