Tomcat Cluster Session Replication

Sample Image Tomcat Cluster Session Replication how to is a document that describes how we can setup a tomcat 5.5 cluster with apache web server frontend using in-memory session replication.
Tomcat 5.5.x has significant performance improvements than tomcat 5.0, mostly because of java 5 (from performance metrics taken with jmeter – http://jakarta.apache.org/jmeter/ about 50% faster in compilation of classes/jsp/servlets). In our example we will use 1 apache instance that load balance requests along 4 instances of tomcat servers in 2 different servers.

Our 2 test servers are server1, server2. Each one will have 2 instances of tomcat servers (tomcat1 and tomcat2) and server1 will have the apache web server instance.

Starting with the apache web server, we install / compile apache2.0.x in server1. I prefer to compile apache, for performance / security reasons, but an installation from packages (rpm / deb etc.) will do, too.

Then we compile the tomcat connector. We will use mod_jk (http://tomcat.apache.org/connectors-doc/ ). We can use a binary version, or compile from source. After we extract the source,

# cd jakarta-tomcat-connectors-version-src/jk/native
# CFLAGS="-DEAPI -O3" ./configure –with-apxs=/path_to_apache/bin/apxs –enable-DEAPI
# make
# make install

After successful compilation, in /apache_path/modules/ we should have the module compiled (mod_jk.so).

We need to add these lines in httpd.conf:

# Tomcat Configuration
LoadModule jk_module modules/mod_jk.so
JkWorkersFile "conf/workers.properties"
JkLogFile "logs/mod_jk.log"
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

# Tomcat Mapping
JKMount /*.jsp loadbalancer
JKMount /* loadbalancer
JkMount jkstatus

We also need to create a file named workers.properties in /apache_path/conf directory.

#
# workers.properties
#
# In Unix, we use forward slashes:
ps=/
# list the workers by name
worker.list=tomcat1, tomcat2, tomcat3, tomcat4, loadbalancer
# ————————
# 1st tomcat server
# ————————
worker.tomcat1.port=18009
worker.tomcat1.host=server1
worker.tomcat1.type=ajp13
# Specify the size of the open connection cache.
#worker.tomcat1.cachesize
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# —-> lbfactor must be > 0
# —-> Low lbfactor means less work done by the worker.
worker.tomcat1.lbfactor=100
# ————————
# 2nd tomcat server
# ————————
worker.tomcat2.port=28009
worker.tomcat2.host=server1
worker.tomcat2.type=ajp13
# Specify the size of the open connection cache.
#worker.tomcat2.cachesize
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# —-> lbfactor must be > 0
# —-> Low lbfactor means less work done by the worker.
worker.tomcat2.lbfactor=100

# ————————
# 3rd tomcat server
# ————————
worker.tomcat3.port=18009
worker.tomcat3.host=server2
worker.tomcat3.type=ajp13
# Specify the size of the open connection cache.
#worker.tomcat1.cachesize
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# —-> lbfactor must be > 0
# —-> Low lbfactor means less work done by the worker.
worker.tomcat3.lbfactor=100
# ————————
# 4th tomcat server
# ————————
worker.tomcat4.port=28009
worker.tomcat4.host=server2
worker.tomcat4.type=ajp13
# Specify the size of the open connection cache.
#worker.tomcat2.cachesize
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# —-> lbfactor must be > 0
# —-> Low lbfactor means less work done by the worker.
worker.tomcat4.lbfactor=100

# ————————
# Load Balancer worker
# ————————
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# —-> If a worker dies, the load balancer will check its state
# once in a while. Until then all work is redirected to peer
# worker.
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=tomcat1, tomcat2, tomcat3, tomcat4

# Add the status worker to the worker list
worker.list=jkstatus
# Define a 'jkstatus' worker using status
worker.jkstatus.type=status

#
# END workers.properties
#

In order to prepare tomcat installation, we install java5 jdk from http://java.sun.com/ and create a tomcat non-privileged user with the following options in user environment:

PATH="/java_path/bin:/java_path/jre/bin:$PATH"
JAVA_HOME="/java_path"
JAVA_OPTS=' -Xmx768m -Xms512m -Xss1024k -server'
export PATH JAVA_HOME JAVA_OPTS

We chose binary distributions for tomcat servers:
For both servers we do:

# cd /path_to install_tomcat
# tar zxvf tomcat-version.tar.gz
# mv tomcat-version tomcat1
# chown –R tomcat:root tomcat1
# cp -pr tomcat1 tomcat2

For tomcat1 in server1 configuration we change server.xml as follows:

<!– Example Server Configuration File –>
<Server port="18005" shutdown="SHUTDOWN">

</GlobalNamingResources>

</Host>

</Engine>

</Service>

</Server>

Now we can deploy our application in the path defined in server.xml (/var/www). As specified in http://tomcat.apache.org/tomcat-5.5-doc/cluster-howto.html :

All your session attributes must implement java.io.Serializable
Make sure your web.xml has the <distributable/> element or set at your <Context distributable="true" />
Make sure that all nodes have the same time and sync with NTP service!
Make sure that your loadbalancer is configured for sticky session mode.

You can use the following example session.jsp which shows the session id:

<!– session.jsp: example of session variable using Scriplets & HTML mixed –>

<%@page contentType="text/html"%>
<html>
<head><title>Session Example: session.jsp</title></head>
<body>

<!–
Get the session id and display it to the user
–>

<%– THIS IS ONE WAY TO WRITE THE CODE, MIXING SCRIPLETS WITH HTML –%>
This is the session id <%= session.getId()%>

<!– check to see if the session in new, if not display session information –>
<% if ( session.isNew() ) { %>
This is a new session ! 
<%} else { %>
This is an existing session 
<%}%>

</body>
</html>

Now start all tomcat instances and watch catalina.out log for messages like:

"…
org.apache.catalina.cluster.tcp.SimpleTcpCluster start
INFO: Cluster is about to start
org.apache.catalina.cluster.tcp.SimpleTcpCluster createDefaultClusterReceiver
INFO: Add Default ClusterReceiver at cluster server1
org.apache.catalina.cluster.tcp.SocketReplicationListener createServerSocket
org.apache.catalina.cluster.tcp.ReplicationTransmitter start
INFO: Start ClusterSender at cluster Catalina:type=Cluster,host=server1 with name Catalina:type=ClusterSender,host=server1
org.apache.catalina.cluster.mcast.McastService start
INFO: Sleeping for 2000 secs to establish cluster membership
org.apache.catalina.cluster.tcp.SimpleTcpCluster memberAdded
…"

If we open the url http://server1/session.jsp and try some restarts to tomcat instances we can watch session fail over from one instance to other without any interruption or session loss.

Now we have a robust 4-instance tomcat cluster with in-memory session replication. We can try some performance tests and see that we have full load balancing for our connections between apache and tomcat instances. We also have achieve session replication between those instances, for non-interruption in our services even if one instance or server fails.

Sources:

http://tomcat.apache.org/

SysLog.gr

Tomcat Cluster Session Replication

Categories

Archives