Eclipse or SUNrise...

Eclipse or SUNrise...
...JAVA for sure

Wednesday, November 16, 2011

WebSphere Application Server 8 Administration Guide by Packt book review

I just finished reading a new book from Packt's Publishing, the WebSphere Application Server 8 Administration Guide. I'm sure there will be many others reviews of this book, but I would like to look at it from perspective of the IBM's official WAS8 RedBook. Both books are about administration so they should contain similar content. Similar but it can't be the same.

To start with, this book is the first one from the Packt series that I had chance to read so I didn't have any clue of what to expect. My first comparison with the redbook was clearly mathematical - the Packts book is about 500 pages, where the redbook is over 1000. I never liked long books, but trust me, when it came to write about WAS it is really hard not to mention about so many things. Steve Robinson, the author of this book, had to sacrifice something to be able to limit its size. So is it worth to pay extra for this publication?

The Packts book consist of 11 chapters where you will learn from the basics: installation process to typical administrative tasks. I won't write about the content of each chapter, you can check it out by your self here. I think that chapters 3, 5, 7 and 8 deserve a special credit. This is because they contain very valuable information and many superb hints. It is clear that the author has worked with the WAS for many years which led him to make his own "best practices" and it is always good idea to learn from others.

From the other hand there are couple of things that could be improved. The biggest gap in my perspective is the chapter 11 - Product Maintenance, which is valid but only for WAS version 7. I think that the author couldn't test the fixpack update functionality with the WAS version 8 since there were no fixpack by the time when he wrote the book, but the way of updating WAS V8 has changed just as the installation of the WAS itself (the Update Installer for example is deprecated for V8 product). For this information you will have to checkout the redbook.

I also didn't like the part about working with the HTTP server in chapter 9. For me it was a little awkward and it also talked about the older IHS V7, not the current V8. I also miss that author didn't present the capabilities of new WebSphere Customization Toolbox which helps when working with IHS V8.

The last thing I miss is that you wont learn about the powerful WebSphere Network Deployment capabilities like clustering, high availability or new cool V8 concept like the improved Centralized Installation Manager (CIM), which is covered in the redbook. This is why the Packt's book is half the size of the redbook.

So then, to sum it up...

The strong sides of the book

  • Easy and fast to read
  • Excellent for people new to WAS
  • great hints from a person who professionally works with WAS from many years
  • Nice examples with sample code and scripts available to download

The book weaknesses

  • Administration of only standalone WAS installations (Network Deployment is not covered)
  • Unfortunately the part about applying fixpacks in chapter 11 is not valid for WAS version 8
  • Part about HTTP server also could be updated to version 8 in chapter 9

The current eBook (PDF only) version price is £25 (The printed version is £42). I think this is a good price, but if you are new to WAS - you will get a great value for the money. It will guide you from the beginning to all important aspects. Because it is well written and quite short you will have a good feel of how to work with WAS. However, after this introduction you will need more information to jump into the advanced ND topology.

My grade is 4/5.

You can download the book from www.packtpub.com/ibm-websphere-application-server-8-0-administration-guide/book

Thursday, November 3, 2011

WebSphere Application Server 8 Administration Guide by Packt

Couple of days ago I came across a book about WebSphere Application Server version 8 - considering the official IBM redbook is already out in a draft version I am very curious of what this book will be. WebSphere Application Server 8 Administartion Guide is a comercial publication by Packt Publishing. You can checkout the book here. I will write more about it soon.

Thursday, October 13, 2011

Father of C and UNIX died...

UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity.
These were the words from Dennis Ritchie, who passed away last weekend. Its a shame that there is no Nobel prize for someone who gave birth to a C language and UNIX system... Rest in peace Dennis!

Friday, October 7, 2011

Draft WebSphere Application Server 8 RedBook

Finally, yesterday IBM published a draft version of a long awaited WebSphere Application Server V8 Administration and Configuration Guide. You can download it and review it here.
There is also first fixpack 8.0.0.1 available for WAS8 (http://www-01.ibm.com/support/docview.wss?uid=swg24030916). It introduces a Centralized installation manager sudo support which is not covered in the redbook. This feature facilitate installations using CIM and non-root accounts. This is the first fixpack, which will need Installation Manager to be installed in the WebSphere Application Server software.

Thursday, July 21, 2011

WebSphere Application Server 8 Red Book

IBM has released the WebSphere Application Server 8.0 a month ago, and there will be a first RedBook "WebSphere Application Server V8 Concepts, Planning and Design Guide" coming out veeeery soon.

I had a pleasure to work on another publication, the "WebSphere Application Server V8 Administration and Configuration Guide", but it should be availabe not sooner than in a month. I would however recommend to read the concepts to get a felling of what have changed. And to be honest there were couple of changes - one of the biggest is that WAS8 will use the Installation Manager and Update Installer and Installation Factory has been fired... There is a lot that changed in the CIM also - it is smarter then before. And of course with JEE6 there are couple of new technologies like support for Servlet 3.0, EJB3.1 or bundled support for JPA2.0, OSGi 4.2.

I also have a personal feeling after working with WAS8 that you can feel some speed improvements - but that's only my impression after couple of weeks working with it.

You can read more about the new features in a great Developer Works article.

There is also new Rational Application Developer 8 with support for developing OSGi or SCA application with plenty of examples (1900 pages!!!!!!!!!!!! - ITSO record) "Rational Application Developer for WebSphere Software V8 Programming Guide" that you can get here.

Enjoy your book :-)

Tuesday, May 17, 2011

Bitcoin!


Yes, you probably hear about this stuff. If not, you have to see http://bitcoin.org. This project is spreading amazingly fast! It's not only about money, but a philosophy of using money.

Bitcoin means also computing - mining for coins and this is something, that I will look closer to. I really like that they are using GPU for the calculations (which can speed up something like 10 times when comparing to a standard desktop CPU).

I'll write more about bitcoin and its technology later...

And in the meanwhile, if you found information on my blog helpfull pleas drop some bitcoins :-)
Thanks!

127YiYGxFgtWzeSdhpXtxNnvMdjGQfpcMM

Monday, May 9, 2011

WebSphere logs archiving solution

WebSphere Application Server just like others app servers have a nice feature to keep its logs - the SystemOuts and Errs and also traces and JVM natives logs. It allows you to keep logs in given number of files of given maximum size, so it lets you to have a nice rolling log out of the box.

But what if you require to keep history of more than 99 files which is the maximum file number for WAS? There is also a maximum file size of 50MB (personally I would recommend much smaller, 20MB size of logs). So there are two problems - number of files and its size. First lets deal with the number of files. To extend it we can simply use a cron mechanism, which will move the files to another directory every 5 minutes. This simple script will do the trick:


cd /ibm/logs/Srv01/
mv trace_* archive


It assumes that you have a separate directory /ibm/logs/Srv01/archive - it would be nice if it was another file system. So WAS will produce new logs in trace_ files (example file name trace_11.05.09_11.15.33.log). Remember that the current logging activity always goes to a trace.log with no additional postfix so it is important not to move this file. If the trace.log will be full (it reaches the maximum size), WAS renames the file and adds it a postfix time stamp (as in the example above) and creates a new, blank trace.log file.

OK, so we managed to have more than 99 files, now what with its size? If we want to handle more than 99 files of size about 20 MB a day it will give us more than 2GB of space per application server per every single day! We have to compress it. It would also be nice to keep it in one archive per day so if we would like to inspect what happed on given day we would just copy only one bundle. A good information is that text files compress very nicely.

To do this we will also use a script, but a bit more complex one. Here's what I did:


# This script will compress all trace logs from yesterday in its working directory
cd /ibm/logs/Srv01/archive

echo '########## Started on ' $(date +%Y%m%d) >> logger.log
YESTERDAY=`TZ=aaa24 date +%y.%m.%d`
echo $YESTERDAY

YESTERDAY=trace_$YESTERDAY

YESTERDAY=$YESTERDAY*
echo $YESTERDAY

if ls $YESTERDAY
then
echo 'Logs sucessfuly compresed, removing those files:' >> logger.log
tar cvf - $YESTERDAY | gzip > trace_`TZ=aaa24 date +%Y%m%d`.tar.gz

rm $YESTERDAY >> logger.log
else
echo 'Failed compresing the logs!' >> logger.log
fi


OK, so a quick explanation for this script - it enters the archive directory, then it prepares a $YESTERDAY variable which forms an yesterday date (the script will be triggered every day for yesterday logs). Note that I also added a * sign, so this simple regular expression will enable me to list all files on given day (trace_11.05.09*). You can shorten my script but I created it this way only to give you a step by step guide of what it does. OK, so now if we know what files we want to archive, we just try to list them via ls command, if the script succeed, that means that there are files to be archived so it continue to tar them and also zip them to a single file. The last part is the deletion of compressed files. You can also add another if instruction to the tar command to be sure to remove the plain log files only when the zipping command will succeed.

The result archive will look like this trace_20110507.tar.gz

The script also adds a log to a logger.log file to keep a history of what happed in case you would want to check it out - remember this will run automatically!

The last task would be to add two cron lines, first one will trigger the first script and it triggers every 5 minutes. The second one triggers every 2 AM (the night time is usually the best time to utilize the servers).

5 * * * * /first_script.sh
0 2 * * * /second_script.sh

Hope you'll find it useful. I created those scripts on AIX system but they should also run on most of Linux whith little or no change.

Monday, March 28, 2011

Integrating with Rational Team Concert

In this post I'll show you an example of automation of tasks with Rational Team Concert using Java. Couple of days ago I was asked to "somehow" automate a process of generating and publishing a bunch of reports to RTC. The process contained couple of steps - first a cron job which fired a PL/SQL procedure which generated a proper report, then it had to be processed, zipped and uploaded to an existing RTC custom item called report (similar to a task).

So lets focus on the interesting part - integration with RTC. Because RTC gives an JAZZ API I decided to upload the files using a simple, JAVA SE application. Unfortunately as I found out on the JAZZ official pages there was very little information about the API. Yes, there were couple of examples, but no real documentation or at least a proper Javadoc... I found some helpful info on the Internet but again, it still is not a rich source of documentation for this.

Oh, and to gain access to this so called documentation you have to register on JAZZ pages. Although you won't find a complete and proper documentation there, you will have an access to a few handy examples and a forum. At the end of this article, you can see a list of useful links.

To run the application, you'll need the client libraries to be able to connect to the RTC. I downloaded a JazzPlainJavaClient-2.0 package. It is more then 20MB of jars, but you don't require all of them to create your application. You can download it here. Look for 'Plain Java Client Libraries'. There is a lot more to download, but if you plan to develop something small don't really bother with anything else.

Now when you have your copy of libraries wire them to your project and you can run my code.

The following code is just a main body, which initialize the libraries and executes the real logic of the program - the run() function passing it all the attributes. You will find a very similar code on the examples on jazz pages.


public static void main(String[] args) {

boolean result;
TeamPlatform.startup();
try {
result = run(args);
} catch (TeamRepositoryException x) {
x.printStackTrace();
result = false;
} finally {
TeamPlatform.shutdown();
}

if (!result)
System.exit(1);

}


The run() function simply parse the arguments - there should be 6 of them and one of them should be integer. Of course you can improve the parsing, my is just as good as it have to be. Then, if they are are ok, the program authenticates provided user in the RTC repository (so be sure you gave a proper user with a valid password).

Then it looks for a project area to connect to (it will be needed to upload the attachment, note also that your user must have access there). If the program will find it, it searches for a given item (for which it will upload the report).

Now the interesting part - using WorkItemWorkingCopy object to get the item data - the one that we really want is the the item reference, which we pass later to upload the attachment (itemCopy.getReferences()). Generally we can use it also to update the task itself - for example to add additional comment or change the priority.

Now we invoke the addAttachment() function passing it the stream to a file that we want to upload, its name visible in RTC, RTC item reference to which we will attach the file, the client binding that we will use to persist the attachment and finally the RTC project area.


private static boolean run(String[] args) throws TeamRepositoryException {

if (args.length != 6) {
System.out
.println("Usage: UpdateWorkItem ");
return false;
}

int rtcDocumentId = -1; // 6666

if (args[0] != null && args[0].length() > 0) {

try {
rtcDocumentId = Integer.valueOf(args[0]);
} catch (NumberFormatException nfe) {
System.out.println("The workitem id have to be a valid number");
return false;
}
}

SimpleDateFormat dateFormat = new SimpleDateFormat("M/dd/yyyy hh:mm");
System.out.println("Starting program at " + dateFormat.format(new Date(System.currentTimeMillis())));

final String repositoryURI = args[1];
final String userId = args[2];
final String password = args[3];
final String projectAreaName = args[4];
final String fileName = args[5];

// loging in to the RTC
ITeamRepository teamRepository = TeamPlatform.getTeamRepositoryService().getTeamRepository(repositoryURI);
teamRepository.registerLoginHandler(new LoginHandler(userId, password));
teamRepository.login(null);

URI uri = URI.create(projectAreaName.replaceAll(" ", "%20"));
IProcessClientService processClient = (IProcessClientService) teamRepository
.getClientLibrary(IProcessClientService.class);
IProjectArea projectArea = (IProjectArea) processClient.findProcessArea(uri, null, null);
if (projectArea == null) {
System.out.println("Project area not found, check if you gave the right name");
return false;
}

System.out.println("Querying workitems with id " + rtcDocumentId);

IWorkItemClient workItemClient = (IWorkItemClient) teamRepository.getClientLibrary(IWorkItemClient.class);
IWorkItemWorkingCopyManager copyManager = workItemClient.getWorkItemWorkingCopyManager();
IWorkItemHandle handle = workItemClient.findWorkItemById(rtcDocumentId, IWorkItem.FULL_PROFILE, null);
try {
copyManager.connect(handle, IWorkItem.FULL_PROFILE, null);
WorkItemWorkingCopy itemCopy = copyManager.getWorkingCopy(handle);
IWorkItem item = itemCopy.getWorkItem();

System.out.println("Found work item: " + item.getHTMLSummary());
System.out.println("Adding new attachment...");

InputStream fileInputStream = null;
try {
if (fileName != null)
fileInputStream = new FileInputStream(fileName);

InputStream inputStream = fileInputStream != null ? fileInputStream : System.in;

addAttachment(inputStream, fileName, itemCopy.getReferences(), workItemClient, projectArea);

} catch (FileNotFoundException e) {
System.out.println("###\n Error accured, file " + fileName + " not found\n###");
return false;
} finally {
if (fileInputStream != null) {
try {
fileInputStream.close();
} catch (IOException e) {
System.out.println("Couldn't close the fileInputStream, probably you can skip this error :-) ");
}
}
}

System.out.println("Commiting the new attachment to RTC workitem..");
itemCopy.save(null);
System.out.println("Attachement added succesfully!");

} finally {
copyManager.disconnect(handle);
}

System.out.println("Disconnecting from RTC...");
teamRepository.logout();
System.out.println("Disconnected! Finishing the program. Thanks for using!");

return true;
}


Finally the addAttachment function - it simply creates a new attachment and saves it. Notice that I used a text "UTF-8" as the coding for this file. Unfortunately, I haven't found much about this interface and with other parameter values it didn't worked - so I left it with UTF-8, maybe it isn't the best option, but it works pretty well. Again, more documentation would be nice...


private static void addAttachment(InputStream inputStream, String fileName, IWorkItemReferences workItemReferences,
IWorkItemClient workItemClient, IProjectArea projectArea) throws TeamRepositoryException {
try {

IAttachment newAttachment = workItemClient.createAttachment(projectArea, fileName, "", "text", "UTF-8",
inputStream, null);

newAttachment = (IAttachment) newAttachment.getWorkingCopy();
newAttachment = workItemClient.saveAttachment(newAttachment, null);
IItemReference reference = WorkItemLinkTypes.createAttachmentReference(newAttachment);
workItemReferences.add(WorkItemEndPoints.ATTACHMENT, reference);

} catch (Exception e) {
System.out.println("Exception while adding attachement: " + e.getMessage());
throw new TeamRepositoryException(e);
}
}


Now the last thing to do is to run the program - remember to pass it a proper set of arguments and link it with the client libraries. You can use the following command to run it:


java -Djava.ext.dirs="/usr/lib/JazzPlainJavaClient-2.0:/usr/jre1.6.0_24/lib/ext" -jar UpdateWorkItem.jar 6666 https://10.10.10.10:9443/jazz account@account.com password "MyProject" "/home/xxx/report_2011_03_20.zip" >> sendReportToRTC.log


And that's just enough to start playing with RTC. I just regret that they won't supply more documentation...

Here couple of handy links that I used:

https://jazz.net/wiki/bin/view/Main/QueryDevGuide
https://jazz.net/wiki/bin/view/Main/ProgrammaticWorkItemCreation
https://jazz.net/wiki/bin/view/Main/WorkItemExampleSource

Tuesday, March 8, 2011

Hacking WebSphere Application Server console

I guess everyone had a situation where he or she forgot his or her password and had to renew it or ask someone to do it for him. This comes more complex where you as an administrator of your server forgets the password or maybe you have to work with a server that you don't know credentials for.

When it comes for WAS, there is a way to change the administrative user account password even if you can't get in the console in the first place! You can do it if you have access to the operating system where the WAS resides and have at least permissions for the user which runs the WAS (so it needn't be a root).

When you log in as that user go to the WAS install directory, then to the profile config subdirectory and find the cells subdirectory. Let's assume we installed WAS here:

/ibm/WebSphere/AppServer/profiles/WPSprofile01/config/cells/XXX_Cell/


In there you will find a nice
security.xml
file. Edit it. The first lines are:


<?xml version="1.0" encoding="UTF-8"?>
<security:Security xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:or
b.securityprotocol="http://www.ibm.com/websphere/appserver/schemas/5.0/orb.secur
ityprotocol.xmi" xmlns:security="http://www.ibm.com/websphere/appserver/schemas/
5.0/security.xmi" xmi:id="Security_1" useLocalSecurityServer="true" useDomainQua
lifiedUserNames="false" enabled="true" cacheTimeout="598200" issuePermissionWarn
ing="true" activeProtocol="BOTH" enforceJava2Security="false" enforceFineGrained
JCASecurity="false" appEnabled="true" dynamicallyUpdateSSLConfig="true" activeAu
thMechanism="LTPA_1" activeUserRegistry="WIMUserRegistry_1" defaultSSLSettings="
SSLConfig_1">


Now, notice the attribute enabled="true". This property says that administrative security is enabled for the console. So if you change the value to false - you disable it.

Easy? Not really - in order to use the new settings the server has to be restarted and WAS will require a password for that to make it happen. So unless you don't have a valid password you will not be able to log in. Fortunately you can deal with it by using kill command for the WAS process. Just find the process using ps and grepping the WAS profile name.

Now, start the server and log in using the standard url without https (the standard URL is http://washost:9060/ibm/console), just click ok button, you don't have to specify the user and you will see the console.

So you hacked it! Now let's change the password and enable the security. We won't do it with a file, just go to the security tab. Now it depends of your security realm settings of how you can change the password. I'll write more about security and managing users on WAS in another post - basically you can do pretty much in here.

In the end don't forget to switch the security on and restart the server again. This scenario is easy for a standalone server, it gets more complicated with clustered, network managed environments but it also works with them. It just requires additional node synchronizing and a correct order of your operations to be effective.

Friday, March 4, 2011

JDK security alert for CVE-2010-4476

Lately IBM have announced a fix for its JDK virtual machine bug which might lead to a DoS (Denial of Service) attack. It is an important security issue. You can read about it here. WAS versions 6, 6.1 and 7.0 are affected so the bug stayed in Java up from version 1.4... quite some time, nice :-).

This is a result from a similar info from Sun... oh, Oralce that is. And you can read it here. IBM JDK is modified Oracle's one.

But what made me wonder was the reason that cosed that behavior. I quote:

This Security Alert addresses security issue CVE-2010-4476 (Java Runtime Environment hangs when converting "2.2250738585072012e-308" to a binary floating-point number), which is a vulnerability in the Java Runtime Environment component of the Oracle Java SE and Java for Business products. This vulnerability allows unauthenticated network attacks ( i.e. it may be exploited over a network without the need for a username and password). Successful attack of this vulnerability can result in unauthorized ability to cause a hang or frequently repeatable crash (complete Denial of Service) of the Java Runtime Environment. Java based application and web servers are especially at risk from this vulnerability.


Digging in the problem, that means that an instruction like this:

Double thisWillHurt = Double.parseDouble("2.2250738585072012e-308")


Will crash the app server... nice. I'm just curious how do they come up with this particular COMPLEX in fact number. I wonder if there is any other magical number...

Anyway, I advise to install the new fixpack (if possible) on your environments, or at least only an APAR for this:

WAS v7.0
WAS v6.1
WAS v6.0

Monday, February 28, 2011

Getting Real with 37signals - book review

Lately I came up with this book while browsing on the net. The book costs $19 for a PDF version and $25 for a printed one, but you can read it free on the web here - just like I did :-).



37signals is a small company who made its business developing smart and handy web applications - like Basecamp or Ta-Da list. They focus on open source (i.e. heavy usage of Ruby on Rails) and an agile method of developing its products. I would say that they are the essence of what a KISS rule in programming is.

The book itself is something like their business card (I suppose that's why you can have a free access for this publication), a presentation of their point of view and their methods. I especially liked the light and direct approach of it. It is not like the majority of books with a stream of text and charts. It is build as a bunch of assays grouped in chapters. Each chapter is about some part of a development process - from the start, how to prepare to a project, how to plan it, even how to build a team and how to choose application features, how to say no and stay lean, how to launch it and keep it successful.

It is all about so much in so few words, each chapter consists of only couple of pages and on each page you will find no more than few sentences (plus extra quotes of external sources given as examples). This gives us only the essence - the really important stuff that matters.

I think that this short-form of the book gives you one thing more - it leaves you with your own thoughts, it just doesn't want to point everything for you - it leaves it to yourself. I really enjoyed reading it in this form.

I worked with both, small open source and big corporations and I have to agree that what they write about is very good for web based projects. Especially for small companies there is a lot that you can adopt and use and there is also plenty of room for improvement for big companies too, but it requires a major change in thinking and planning.

At the end 37signals put an interesting example of using thieir way not only in web apps market. I qoute:

Special ops forces, like the Green Berets or Navy Seals, use small teams and rapid deployment to accomplish tasks that other units are too big or too slow to get done


They are true, the power of these special units rely on their small size - only in this way they can accomplish their tasks like infiltration of enemy territory.

I have to say though, that these units can make huge impact on the overall, lets say war scenario, they will not win the whole war. Yes, they can bring the end of the war much closer to the end but imagine what would happen if one side of conflict wouldn't have big army. Ones who would bet only on small sized units wouldn't have a chance (they would defend heroically but would fail in the end). What I try to say is that rapid, extreme way presented by 37signals is a great way with many, many advantages but it won't fit for every occasion. I doubt if it would work for lets say NASA project of sending people to moon.

But from the other hand, big companies which work in a complex IT environments and using complex way of driving projects, force to adopt this heavy way even for small or medium scale projects - which is IMHO wrong. They are blind for new approaches which doesn't integrate with their current procedures...

This book tells you how you can use 100% of people potential. But it's only an interface - the implementation is up to you.

Thursday, February 3, 2011

Tuning HTTP Server part 1

Finally I got some time to write something (useful I hope). Today I want to point out one of important HTTP server parameters - the keepAlive settings. This property is used for allowing persistent connections to IHS. By default this option is on and it has got couple of others parameters which tune the timeout of keepAlive or maximum keep alive requests.

The keepAlive options are good for example for apps where users stay logged in and often communicate with the server because IHS wont have to initialize the whole connection procedure from scratch so this acts something like a connection cache. In many other cases though, this is not a good option to use.

Couple of days ago I had a case of a client who had an ESB with IHS servers in the front and its performance was very poor, there were a lot of timeouts and the environment could work stable just for a couple of hours. One of important changes that solved the problem was turning off the keepAlive option.

On this chart you can see the traffic on IHS before the change.



You can notice, that the IHS used all threads for processing, they kept the threads instead of releasing them which cosed the degrading performance - the incoming requests had to wait longer and longer to process.

On the next chart you can see the traffic after the change (with the historic view of previous behavior).




IHS are now stable and the original Threads pool are big enough to process the request in the real time. Another interesting thing is, that the KA parameter doesn't really show that it causes the problem, because this value is quite low - but mainly the reason for that is that during big load all available connections are immediately occupied.

Documentation of IHS says that setting keepAlive will result in higher CPU utilization (which is logic because servers will have more work with creating new connections), but the effect that I measured was the opposite. NMON showed an average CPU utilization to be lower by about 5 to 10%. I can explain this because before the change IHS were heavily loaded with new requests which the server couldn't process - in the keepAlive off scenario IHS managed to offload the incoming traffic in real time and there were no repeating calls to it that it would have to handle.

All in all - the Apache HTTP Server on which IHS is based, has a real potential and though it runs very fast even with default settings, we have to remember that it will influence the rest of the system performance since it is one of the first components in the system architecture (in most cases) and the bigger gap is there, the slower whole system will run.

Monday, January 24, 2011

Analyzing IBM HTTP Server performance

In order to tune the performance of HTTP server (in this case IBM HTTP Server which basically is an Apache) you have to get some informations about its current configuration, update configuration accordingly to the retrieved data and then again recheck if there is any performance gain (and repeat these steps until you'll get what you wanted).

One of possibilities of capturing data in IHS is using the mod_mpmstatus.so plugin. It is very similar to standard mod_status.so of the Apache but it has not the down side of having to call the /server-status page manually or having some automate which would do it and grab the results. The mod_mpmstatus is just appending the results to standard error_log file. Additionally the format in which it does it is very easly modifiable as I'll further present.

To use this plugin, you will have to download the IBM diagnostic pack - the current version of it is ihsdiag V1.4.14 and you can download it freely from here (there multiple versions for different operating systems):



After unpacking it, you'll find the module file in

ihsdiag-1.4.14/2.0/aix/mod_pmpstatus.so

Copy this file to /{IHS_ROOT}/modules/ directory

Now edit httpd.conf (a good practice is to create a backup of this file after editing it) and add two lines


LoadModule mpmstats_module modules/mod_mpmstats.so
ReportInterval 120


The first one will load the new plugin to the IHS runtime configuration - this also mean that you will have to restart the IHS in order to run the plugin logic. The second line configures the period of reporting - how often (in seconds) it will log the current stats on the server (in error_log only).

After restarting you should see a following:


[Mon Jan 24 10:45:09 2011] [notice] mpmstats: rdy 40 bsy 10 rd 0 wr 7 ka 3 log 0 dns 0 cls 0
[Mon Jan 24 10:47:09 2011] [notice] mpmstats: rdy 47 bsy 3 rd 0 wr 1 ka 2 log 0 dns 0 cls 0
[Mon Jan 24 10:49:09 2011] [notice] mpmstats: rdy 46 bsy 4 rd 0 wr 1 ka 3 log 0 dns 0 cls 0
[Mon Jan 24 10:51:09 2011] [notice] mpmstats: rdy 47 bsy 3 rd 0 wr 1 ka 1 log 0 dns 0 cls 1
[Mon Jan 24 10:53:09 2011] [notice] mpmstats: rdy 47 bsy 3 rd 0 wr 0 ka 3 log 0 dns 0 cls 0
[Mon Jan 24 10:55:09 2011] [notice] mpmstats: rdy 45 bsy 5 rd 0 wr 2 ka 3 log 0 dns 0 cls 0
[Mon Jan 24 10:57:09 2011] [notice] mpmstats: rdy 46 bsy 4 rd 0 wr 0 ka 2 log 0 dns 0 cls 2
[Mon Jan 24 10:59:10 2011] [notice] mpmstats: rdy 47 bsy 3 rd 0 wr 1 ka 2 log 0 dns 0 cls 0


Notice that there is a special tag [notice] for each statistic and it is being logged every 2 minutes. The small lag is because the IHS tries to lunch the plugin when there is possibly low utilization on the server.

Each of the short description next to the number means:


  • rdy (ready) the number of web server threads started and ready to process new client connections


  • bsy (busy) the number of web server threads already processing a client connection


  • rd (reading) the number of busy web server threads currently reading the request from the client


  • wr (writing) the number of busy web server threads that have read the request from the client but are either processing the request (e.g., waiting on a response from WebSphere Application Server) or are writing the response back to the client
    ka (keepalive) the number of busy web server threads that are not processing a request but instead are waiting to see if the client will send another request on the same connection; refer to the KeepAliveTimeout directive to decrease the amount of time that a web server thread remains in this state


  • log (logging) the number of busy web server threads that are writing to the access log


  • dns (dns lookup) the number of busy web server threads that are performing a dns lookup


  • cls (closing) the number of busy web server threads that are waiting for the client to acknowledge that the entire response has been received so that the connection can be closed



Now, to create a more readable graph of what is happening you can simply use grep or any other editor. I will use Open Office Spreadsheet to create a nice chart. Here is a sample of what you can produce:



In next articles I'll cover the other part about analyzing these logs and use them to tune the performance of the server.