Appreciating garbage

I once was working with a company that complained that my code was not optimized because it used 100% of the CPU. So I asked if the user interface was responsive. And it was. So I asked if they experienced any noticeable delays in responses. They didn't. So I asked for how long the CPU was at 100%. And the answer was...

...the CPU was spiking at 100% for a second or two a few times per minute. So I spent an hour educating the customer on how computers work and explained why it is good that the program (it was a client program) used 100% CPU when it needed. And that it actually meant the code was in pretty good shape given that there was no impact to the user interface when this happened.

Similar to that story I once in a while encounter people who are afraid of allocating memory in their .Net program. And they do all kinds of things to optimize memory allocation. And most of the time that optimization only causes more problems and work than the actual gain. Creating and collecting short lived objects is actually extremely cheap in .Net. Yes, there is a cost compared to not allocating memory but the potential added complexity of the program may not be worth the performance gain.

Does that mean you never need to think about how you allocate memory in .Net? No, in especially services that handle a lot of requests you might actually benefit from reusing objects over creating new ones even if short lived. I once ran into this problem where the garbage collector was running 20-25 times per second (according to windows performance counters) meaning that even very short lived objects were around for multiple GC cycles causing a lot of fragmentation and in general bad performance.

In that case on of the most created objects was a buffer used to receive data from the network. By allocating a much smaller buffer (the default size was ten times larger than needed) the number of GC cycles per second went down to about once per second... Look! I didn't reuse buffers - I just allocated the right size buffer and it helped. Late on in that project buffers were actually reused from a pool of buffers but to be honest the performance gain from that was not even close to the one of using the right size of buffer.

So the lesson I learned was that lots of short lived objects is just fine. And lots of objects that are around for a very long time is also fine. It is the objects that live for a while (surviving one or two GC cycles) that will bite you.

I came to think about this topic when I was viewing this excellent talk on garbage collection. A lot of the stuff is applicable to .Net too even though it focuses on the JVM garbage collector.