Chapter 17. JBossCache and JGroups Services - JBoss Application Server 4.0.3 release4 Guide 英文版指南文档

UDP is the preferred protocol for JGroups. UDP uses multicast or multiple unicasts to send and receive messages. If you choose UDP as the transport protocol for your cluster service, you need to configure it in the UDP sub-element in the JGroups Config element. Here is an example.

<UDP mcast_send_buf_size="32000"
    mcast_port="45566"
    ucast_recv_buf_size="64000"
    mcast_addr="228.8.8.8"
    bind_to_all_interfaces="true"
    loopback="true"
    mcast_recv_buf_size="64000"
    max_bundle_size="30000"
    max_bundle_timeout="30"
    use_incoming_packet_handler="false"
    use_outgoing_packet_handler="false"
    ucast_send_buf_size="32000"
    ip_ttl="32"
    enable_bundling="false"/>
                

The available attributes in the above JGroups configuration are listed below.

Alternatively, a JGroups-based cluster can also work over TCP connections. Compared with UDP, TCP generates more network traffic when the cluster size increases but TCP is more reliable. TCP is fundamentally a unicast protocol. To send multicast messages, JGroups uses multiple TCP unicasts. To use TCP as a transport protocol, you should define a TCP element in the JGroups Config element. Here is an example of the TCP element.

<TCP start_port="7800"
    bind_addr="192.168.5.1"
    loopback="true"/>
                

Below are the attributes available in the TCP element.

The TUNNEL protocol uses an external router to send messages. The external router is known as a GossipRouter. Each node has to register with the router. All messages are sent to the router and forwarded on to their destinations. The TUNNEL approach can be used to setup communication with nodes behind firewalls. A node can establish a TCP connection to the GossipRouter through the firewall (you can use port 80). The same connection is used by the router to send messages to nodes behind the firewall. The TUNNEL configuration is defined in the TUNNEL element in the JGroups Config element. Here is an example.

<TUNNEL router_port="12001"
    router_host="192.168.5.1"/>
                

The available attributes in the TUNNEL element are listed below.

The PING discovery protocol normally sits on top of the UDP transport protocol. Each node responds with a unicast UDP datagram back to the sender. Here is an example PING configuration under the JGroups Config element.

<PING timeout="2000"
    num_initial_members="2"/>
                

The available attributes in the PING element are listed below.

If both gossip_host and gossip_port are defined, the cluster uses the GossipRouter for the initial discovery. If the initial_hosts is specified, the cluster pings that static list of addresses for discovery. Otherwise, the cluster uses IP multicasting for discovery.

The TCPGOSSIP protocol only works with a GossipRouter. It works essentially the same way as the PING protocol configuration with valid gossip_host and gossip_port attributes. It works on top of both UDP and TCP transport protocols. Here is an example.

<PING timeout="2000"
    initial_hosts="192.168.5.1[12000],192.168.0.2[12000]"
    num_initial_members="3"/>
                

The available attributes in the TCPGOSSIP element are listed below.

The TCPPING protocol takes a set of known members and ping them for discovery. This is essentially a static configuration. It works on top of TCP. Here is an example of the TCPPING configuration element in the JGroups Config element.

<TCPPING timeout="2000"
    initial_hosts="192.168.5.1[7800],192.168.0.2[7800]"
    port_range="2"
    num_initial_members="3"/>
                

The available attributes in the TCPPING element are listed below.

The MPING protocol is a multicast ping over TCP. It works almost the same way as PING works on UDP. It does not require external processes (GossipRouter) or static configuration (initial host list). Here is an example of the MPING configuration element in the JGroups Config element.

<MPING timeout="2000"
    bind_to_all_interfaces="true"
    mcast_addr="228.8.8.8"
    mcast_port="7500"
    ip_ttl="8"
    num_initial_members="3"/>
                

The available attributes in the MPING element are listed below.

The FD discovery protocol requires each node periodically sends are-you-alive messages to its neighbor. If the neighbor fails to respond, the calling node sends a SUSPECT message to the cluster. The current group coordinator double checks that the suspect node is indeed dead and updates the cluster's view. Here is an example FD configuration.

<FD timeout="2000"
    max_tries="3"
    shun="true"/>
                

The available attributes in the FD element are listed below.

The are-you-alive messages in the FD protocol could increase the network load when there are many nodes. It could also produce false suspicions. For instance, if the network is too busy and the timeout is too short, nodes could be falsely suspected. Also, if one node is suspended in a debugger or profiler, it could also be suspected and shunned. The FD_SOCK protocol addresses the above issues by suspecting node failures only when a regular TCP connection to the node fails. However, the problem with such passive detection is that hung nodes will not be detected until it is accessed and the TCP timeouts after several minutes. FD_SOCK works best in high load networks where all nodes are frequently accessed. The simplest FD_SOCK configuration does not take any attribute. You can just declare an empty FD_SOCK element in JGroups's Config element.

<FD_SOCK/>
                

There is only one optional attribute in the FD_SOCK element.

The FD_SIMPLE protocol is a more tolerant (less false suspicions) protocol based on are-you-alive messages. Each node periodically sends are-you-alive messages to a randomly choosen node and wait for a response. If a response has not been received within a certain timeout time, a counter associated with that node will be incremented. If the counter exceeds a certain value, that node will be suspected. When a response to an are-you-alive message is received, the counter resets to zero. Here is an example configuration for the FD_SIMPLE protocol.

<FD_SIMPLE timeout="2000"
    max_missed_hbs="10"/>
                

The available attributes in the FD_SIMPLE element are listed below.

The UNICAST protocol is used for unicast messages. It uses ACK. It is configured as a sub-element under the JGroups Config element. Here is an example configuration for the UNICAST protocol.

<UNICAST timeout="100,200,400,800"/>
                

There is only one configurable attribute in the UNICAST element.

The NAKACK protocol is used for multicast messages. It uses NAK. Under this protocol, each message is tagged with a sequence number. The receiver keeps track of the sequence numbers and deliver the messages in order. When a gap in the sequence number is detected, the receiver asks the sender to retransmit the missing message. The NAKACK protocol is configured as the pbcast.NAKACK sub-element under the JGroups Config element. Here is an example configuration.

<pbcast.NAKACK
    max_xmit_size="8192"
    use_mcast_xmit="true" 
    retransmit_timeout="600,1200,2400,4800"/>
                

The configurable attributes in the pbcast.NAKACK element are as follows.

The group membership service in the JGroups stack maintains a list of active nodes. It handles the requests to join and leave the cluster. It also handles the SUSPECT messages sent by failure detection protocols. All nodes in the cluster, as well as the load balancer and client side interceptors, are notified if the group membership changes. The group membership service is configured in the pbcast.GMS sub-element under the JGroups Config element. Here is an example configuration.

<pbcast.GMS print_local_addr="true"
    join_timeout="3000"
    down_thread="false" 
    join_retry_timeout="2000"
    shun="true"/>
                

The configurable attributes in the pbcast.GMS element are as follows.

The flow control service tries to adapt the sending data rate and the receiving data among nodes. If a sender node is too fast, it might overwhelm the receiver node and result in dropped packets that have to be retransmitted. In JGroups, the flow control is implemented via a credit-based system. The sender and receiver nodes have the same number of credits (bytes) to start with. The sender subtracts credits by the number of bytes in messages it sends. The receiver accumulates credits for the bytes in the messages it receives. When the sender's credit drops to a threshold, the receivers sends some credit to the sender. If the sender's credit is used up, the sender blocks until it receives credits from the receiver. The flow control service is configured in the FC sub-element under the JGroups Config element. Here is an example configuration.

<FC max_credits="1000000"
    down_thread="false" 
    min_threshold="0.10"/>
                

The configurable attributes in the FC element are as follows.

The state transfer service transfers the state from an existing node (i.e., the cluster coordinator) to a newly joining node. It is configured in the pbcast.STATE_TRANSFER sub-element under the JGroups Config element. It does not have any configurable attribute. Here is an example configuration.

<pbcast.STATE_TRANSFER 
    down_thread="false"
    up_thread="false"/>
                

In a JGroups cluster, all nodes have to store all messages received for potential retransmission in case of a failure. However, if we store all messages forever, we will run out of memory. So, the distributed garbage collection service in JGroups periodically purges messages that have seen by all nodes from the memory in each node. The distributed garbage collection service is configured in the pbcast.STABLE sub-element under the JGroups Config element. Here is an example configuration.

<pbcast.STABLE stability_delay="1000"
    desired_avg_gossip="5000" 
    down_thread="false"
    max_bytes="250000"/>
                

The configurable attributes in the pbcast.STABLE element are as follows.

When a network error occurs, the cluster might be partitioned into several different partitions. JGroups has a MERGE service that allows the coordinators in partitions to communicate with each other and form a single cluster back again. The flow control service is configured in the MERGE2 sub-element under the JGroups Config element. Here is an example configuration.

<MERGE2 max_interval="10000"
    min_interval="2000"/>
                

The configurable attributes in the FC element are as follows.

JGroups chooses a random value between min_interval and max_interval to send out the MERGE message.