Eclipse or SUNrise...

Eclipse or SUNrise...
...JAVA for sure

Monday, February 28, 2011

Getting Real with 37signals - book review

Lately I came up with this book while browsing on the net. The book costs $19 for a PDF version and $25 for a printed one, but you can read it free on the web here - just like I did :-).



37signals is a small company who made its business developing smart and handy web applications - like Basecamp or Ta-Da list. They focus on open source (i.e. heavy usage of Ruby on Rails) and an agile method of developing its products. I would say that they are the essence of what a KISS rule in programming is.

The book itself is something like their business card (I suppose that's why you can have a free access for this publication), a presentation of their point of view and their methods. I especially liked the light and direct approach of it. It is not like the majority of books with a stream of text and charts. It is build as a bunch of assays grouped in chapters. Each chapter is about some part of a development process - from the start, how to prepare to a project, how to plan it, even how to build a team and how to choose application features, how to say no and stay lean, how to launch it and keep it successful.

It is all about so much in so few words, each chapter consists of only couple of pages and on each page you will find no more than few sentences (plus extra quotes of external sources given as examples). This gives us only the essence - the really important stuff that matters.

I think that this short-form of the book gives you one thing more - it leaves you with your own thoughts, it just doesn't want to point everything for you - it leaves it to yourself. I really enjoyed reading it in this form.

I worked with both, small open source and big corporations and I have to agree that what they write about is very good for web based projects. Especially for small companies there is a lot that you can adopt and use and there is also plenty of room for improvement for big companies too, but it requires a major change in thinking and planning.

At the end 37signals put an interesting example of using thieir way not only in web apps market. I qoute:

Special ops forces, like the Green Berets or Navy Seals, use small teams and rapid deployment to accomplish tasks that other units are too big or too slow to get done


They are true, the power of these special units rely on their small size - only in this way they can accomplish their tasks like infiltration of enemy territory.

I have to say though, that these units can make huge impact on the overall, lets say war scenario, they will not win the whole war. Yes, they can bring the end of the war much closer to the end but imagine what would happen if one side of conflict wouldn't have big army. Ones who would bet only on small sized units wouldn't have a chance (they would defend heroically but would fail in the end). What I try to say is that rapid, extreme way presented by 37signals is a great way with many, many advantages but it won't fit for every occasion. I doubt if it would work for lets say NASA project of sending people to moon.

But from the other hand, big companies which work in a complex IT environments and using complex way of driving projects, force to adopt this heavy way even for small or medium scale projects - which is IMHO wrong. They are blind for new approaches which doesn't integrate with their current procedures...

This book tells you how you can use 100% of people potential. But it's only an interface - the implementation is up to you.

Thursday, February 3, 2011

Tuning HTTP Server part 1

Finally I got some time to write something (useful I hope). Today I want to point out one of important HTTP server parameters - the keepAlive settings. This property is used for allowing persistent connections to IHS. By default this option is on and it has got couple of others parameters which tune the timeout of keepAlive or maximum keep alive requests.

The keepAlive options are good for example for apps where users stay logged in and often communicate with the server because IHS wont have to initialize the whole connection procedure from scratch so this acts something like a connection cache. In many other cases though, this is not a good option to use.

Couple of days ago I had a case of a client who had an ESB with IHS servers in the front and its performance was very poor, there were a lot of timeouts and the environment could work stable just for a couple of hours. One of important changes that solved the problem was turning off the keepAlive option.

On this chart you can see the traffic on IHS before the change.



You can notice, that the IHS used all threads for processing, they kept the threads instead of releasing them which cosed the degrading performance - the incoming requests had to wait longer and longer to process.

On the next chart you can see the traffic after the change (with the historic view of previous behavior).




IHS are now stable and the original Threads pool are big enough to process the request in the real time. Another interesting thing is, that the KA parameter doesn't really show that it causes the problem, because this value is quite low - but mainly the reason for that is that during big load all available connections are immediately occupied.

Documentation of IHS says that setting keepAlive will result in higher CPU utilization (which is logic because servers will have more work with creating new connections), but the effect that I measured was the opposite. NMON showed an average CPU utilization to be lower by about 5 to 10%. I can explain this because before the change IHS were heavily loaded with new requests which the server couldn't process - in the keepAlive off scenario IHS managed to offload the incoming traffic in real time and there were no repeating calls to it that it would have to handle.

All in all - the Apache HTTP Server on which IHS is based, has a real potential and though it runs very fast even with default settings, we have to remember that it will influence the rest of the system performance since it is one of the first components in the system architecture (in most cases) and the bigger gap is there, the slower whole system will run.