Eclipse or SUNrise...

Eclipse or SUNrise...
...JAVA for sure

Thursday, September 23, 2010

Integrating WebSphere DataPower Appliance with Custom J2EE Applications and WebSphere MQ

In this post, I'll deal with something a little bit different - but still tightly connected with Java environment. Lately I had a challenge and another opportunity to work with a very powerful machine - DataPower Appliance. To be precise, the XB60 and XI50 models.

To get started I have to say, that I will cover only the DataPower part, the whole solution is a topic of its own. Maybe a field for future posts. Oh, and because I was using WebSphere Application Servers as hosts of test applications I noted them on the drawing, but you can use any app server instead (any which support at least J2EE 1.3). The important thing is the MQ protocol which is specific for this case (MQMD and MQRFH2 headers).

The scenario

The scenario implemented in this article demonstrates an example of the DataPower appliance usability in a SOA environment pattern where we have a Service Provider and Service Consumer. In our case, the Service Consumer is simulated by a Java EE component (MDB) which runs on WebSphere Application Server and invokes an external service for particular information. The simulated, external system is also a JEE application (a web application). Each component is deployed in a separate environment and uses WebSphere MQ or HTTP protocol to communicate. Intentially I used two popular but different type of technologies (MDB and web based application) to demonstrate the feasibility and capabilities of Java and WebSphere environment.



Detailed steps of the scenario are as follows:

1. A valid XML message with RFH2 header is placed in a DP.INIT.IN queue using RFHUtil tool (for example, it could come from a business process - i.e. WebSphere Process Server).
2. DataPower gets the message from the DP.INIT.IN queue using Front Side Handler.
3. The message is processed by DataPower and then is being sent to web application.
4. The web application (servlet in this case) receives the message.
5. The servlet process the message and returns a HTTP result.
6. The DataPower picks up the HTTP response.
7. DataPower process the message and puts it in queue MDB.IN
8. The MDB component, which listens on MDB.IN queue picks it up and process it.
9. In the end the MDB puts it in the MDB.OUT queue.

Setting up the environment

Creating MQ Resources
Configuration of Application Server

- Connection factory for the listener
- Listener (or activation specification if you want to use EJB3.0 MDB)
- Connection factory for the MDB sender
- Input queue for the listener connection factory
- Output queue for the MDB sender

I explained above configuration in this post.

Configuration of DataPower appliance

Configuration of the DataPower Appliance will be explained in certain order. On this screen I show each of the steps of its configuration:



- Create new Multiprotocol Gateway

First of all create a new Multiprotocol Gateway using a wizard from the welcome page. In my case, I named it My_MQ_HTTP_MPGW (1)

- Configure the static backend

Be sure to switch the static-backend option (which is the default one) and in Backend URL (3) type the http URL of your service. In ma case it was

http://localhost:9081/SimpleWeb/testService but remember to type the real IP address of the server because DataPower appliance will not resolve the localhost host alias.

- Configure the Front Side Handler

Click the ‘+’ sign (4) to add a new Front Side Handler and select the MQ Handler and fill the wizard fields. On this screen I show the most important settings. Note that I switched on Rules and Formatting Header for MQRFH and MQRFH2. This is very important because it will allow DataPower to understand the messages putted using JMS into WebSphere MQ.




- Configure the message types

In my case I require the messages in the flow to be a valid XML messages so select both response and request type to XML (5)

- Configure the Policy for the Gateway

The last but very important step of the configuration is to set up the policy – which is what is going on inside the DataPower with the message. What we want to achieve is to receive a XML message, remember its MQ MessageId retrieved from MQMD header and sent it to backend. Then we want to receive response, add a new MQRFH2 header and copy the MessageId to CorrelationId (because we want the MDB or other listener to mach the response message with the sent request). To achieve this click the ‘+’ sign and create a following policy.



On the picture above you can see that I created a Policy with a name My_MQ_HTTP_Policy. In the policy I created two, one-way rules. First (Client to Server) rule maches all URLs and has got a Transform step. In the transform I used a following XSL style sheet to copy the MessageId and save it in the var://context/MQMD/messageId context variable:


<xsl:stylesheet version="1.0" xsl="http://www.w3.org/1999/XSL/Transform" dp="http://www.datapower.com/extensions" prefixes="dp">
<xsl:template match="/">
<!-- Retrieve old MQMD if it exists -->
<xsl:variable name="oldMQMD">
<xsl:value-of select="dp:request-header('MQMD')" escaping="yes">
</xsl:value-of>

<!-- copy the Message Id for later use (correlation) -->
<xsl:variable name="mqmd" select="dp:parse(dp:request-header('MQMD'))">
<dp:set-variable name="'var://context/MQMD/messageId'" value="normalize-space($mqmd//MsgId/text())">
<!-- copy all elements and their attributes-->
<xsl:copy-of select="node()">
</xsl:copy-of>
</dp:set-variable>
</xsl:variable></xsl:variable></xsl:template></xsl:stylesheet>


We will inspect this variable while testing the scenario.

In the second rule, I have used another XSL style sheet which makes a little more with the headers. This is my XSL template that I have used:

<xsl:stylesheet version="1.0" xsl="http://www.w3.org/1999/XSL/Transform" dp="http://www.datapower.com/extensions" prefixes="dp">

<xsl:template match="/">

<xsl:variable name="MQMDStr">
<mqmd>
<format>MQHRF2</format>
<correlid>
<xsl:value-of select="dp:variable('var://context/MQMD/messageId')">
</xsl:value-of>
</correlid>
</mqmd>

<xsl:variable name="MQMDStr2">
<dp:serialize select="$MQMDStr" decl="yes">
</dp:serialize>
<dp:set-response-header name="'MQMD'" value="$MQMDStr2">

<xsl:variable name="RFH2">
<mqrfh2>
<strucid>RFH</strucid>
<version>2</version>
<encoding>546</encoding>
<format>MQSTR</format>
<flags>0</flags>
<namevalueccsid>1208</namevalueccsid>
<namevaluedata>
<namevalue>
<mcd>
<msd>jms_test</msd>
</mcd>
</namevalue>
<namevalue>
<jms>
<dst>queue:///</dst>
<rto>queue:///</rto>
</jms>
</namevalue>
</namevaluedata>
</mqrfh2>

</xsl:variable>

<!-- SERIALIZED MQRFH2 HEADER -->
<xsl:variable name="rfh2Str">
<dp:serialize select="$RFH2" decl="yes">
</dp:serialize>
<dp:set-response-header name="'MQRFH2'" value="$rfh2Str">

<xsl:message priority="info">
<xsl:copy-of select="concat('New RFH2 header : ', $rfh2Str)">
</xsl:copy-of>

<!-- copy all elements and their attributes-->
<xsl:copy-of select="node()">

</xsl:copy-of>

</dp:set-variable>


Note that in the firs section I prepare a brand new MQMD header with two properties – Format and CorrelId. They are a minimum required by clients to understand the message. First one informs that this is a MQRFH2 message with additional information buried in the body and the other is the correlation identifier which is used to mach incoming responses with requests.

The second part of the transformation generates the MQRFH2 header with its attributes. Not all of them are required but I left them to give you a clue on how the MQRFH2 header might look like. We will inspect it in the RFHUtil tool as well.

Now, if everything went OK we have prepared environment to start unit testing.

Testing the end to end scenario

In order to test the scenario you will need a valid XML message with a RFH2 header. You can use my Initial_rfh2_request.zip file and load it into RFHUtil tool or generate your own (to do this you can simply put a text message in MDB.IN queue and get the MDB.OUT queue – it will contain a proper message). And of course you have to install two applications – your own or you can use mine bundled with all required resources for this article.

1. Check if both applications (MDB and web) are running and if the EmdebListener is up and running
2. Log on to DataPower console and turn on the Probe tool to investigate its internal traffic
3. Load the initialization message to the RFHUtil tool and check its properties
before sending

This is my sample message shown in XML format:



Message contains MQRFH2 header:



Message also contains a proper MQMD header - note that there is a proper (for MQRFH2) Message Format - MQHRF2 - and yes, it is typed correctly ;-):



4. Send it to DP.INIT.IN queue using send it button
5. Refresh the Probe in the DataPower console to see the results

There should be a successfully processed request and response in the Probe window:



Click the request to see its details:



As we can see, DataPower received our message. Click on the magnifying glass after the transform step. In order to see if the MessageId was properly saved in context variable click on ContextVariables tab:



That was exactly what we needed; the long alpha-numeric value is the message Id that we will use later on. Now close this window and return to the probe and click on the response:



As we can see the text is changed by the web application. Note also that the MQMD and MQRFH2 disappeared. It is because we received a HTTP request which doesn’t contain such information. Because we ware prepared for this case in the next, transform step click the magnifying glass icon next to it:




6. Open MQ Explorer or RFHUtil to check if the result message is placed inside MDB.OUT queue and if it contains proper headers
7. Optionally you can investigate the WebSphere Application Server and the J2EE application logs on both servers

Conclusion

This article demonstrates a valuable ability of DataPower to connect separate systems using different protocols (in this case HTTP and MQ), which use different types of headers and are technologically different.

In the article I showed how to build a valid RFH2 header which can be then processed by a JMS ready application (a common real life example) and also how to use the correlation id to inform the backend system about its reply (another common problem).

This is only a small part of the DataPower capabilities but shows its potential and power of customization and usability and also a clear pattern of how to use it.

No comments: