The Moth

« Internet Explorer 8 using Restart APIs t... | Home | Best source code comments »

Measuring performance under Visual Studio

Mon, April 6, 2009, 07:58 PM under dotNET

When I created my Visual Studio C# demos for showing off Tasks versus Threads, I of course created the project in Release mode, but there were 2 additional things that I had to do.

One had little impact, but is important to remember: set the power plan on Windows to "High Performance". Otherwise CPU usage can be throttled by the OS, which is not what you want when you are demonstrating parallelism and 100% CPU usage ;-)

The other had a profound impact and had me puzzled for a bit: use Ctrl+F5 instead of F5. The difference in the time the threads scenario took was humongous. That is because the naïve threads demo creates around 1023 threads and it turns out creating threads under the debugger (F5 case, even in release mode) is measurably slower than not. I should have known that, but I didn't, so sharing it here in case you didn't either: when measuring perf, always use Ctrl+F5 in release mode.

For a lot more on the performance difference between running under the debugger and not, I recommend Joc's blog post: Why is my app so much slower under the debugger.

Comments [3] |

Permalink

Tuesday, April 07, 2009 8:44:00 AM (Pacific Daylight Time, UTC-07:00)

Hey Daniel just noticed you've touched the perf thing :)

Yonks ago, Vance Morrison implemented his Code Timers for perf measurement: http://blogs.msdn.com/vancem/archive/2006/09/21/765648.aspx

It's great as it incorporates the basic statistics, all those mean/median/standard deviation things that give you a much richer understanding of a measurement.

(And you can see how they work in real code: Mocking Frameworks Compare project uses it - see Performance Comparison at http://code.google.com/p/mocking-frameworks-compare/source/browse/trunk/Tests/CS/Performance/Program.cs)

Andrew

Thursday, April 09, 2009 5:02:00 AM (Pacific Daylight Time, UTC-07:00)

Daniel,

That reminds me of the difference of behaviour in the GC between debug and release modes that Jeffrey Ritcher mentions in his CLR via C# book.

http://stupiddumbguy.blogspot.com/2008/05/net-garbage-collection-behavior-for.html discusses it in more detail for those that don't have access to the book.

Daniel did you ever buy/read that book- I remember lending you the first edition way back in 2002? :-)

Richard OD

Saturday, May 09, 2009 1:45:57 PM (Pacific Daylight Time, UTC-07:00)

Andrew: thanks for sharing

Richard: Yes I read it… wow, that was a long time ago :-)

Daniel Moth

All comments require the approval of the site owner before being displayed.