The Moth

Slides and code for MPI Cluster Debugger

2010-03-02T15:01:00.000Z

I've blogged before about the MPI Cluster Debugger in VS2010 that facilitates launching the application on the cluster and attaching the debugger (btw, a shorter version of the screencast I link to there, is here).

There have been requests for the code I use in the screencast, so please find a ZIP with that code.

There have also been requests for a PowerPoint deck to use when showing this feature to others. Feel free to download some slides I threw together the other day.

DirectCompute

2010-02-20T00:00:00.000Z

In my previous blog post I introduced the concept of GPGPU ending with:

On Windows, there is already a cross-GPU-vendor way of programming GPUs and that is the Direct X API. Specifically, on Windows Vista and Windows 7, the DirectX 11 API offers a dedicated subset of the API for GPGPU programming: DirectCompute. You use this API on the CPU side, to set up and execute the kernels on the GPU. The kernels are written in a language called HLSL (High Level Shader Language). You can use DirectCompute with HLSL to write a "compute shader", which is the term DirectX uses for what I've been referring to in this post as "kernel".

In this post I want to share some links to get you started with DirectCompute and HLSL.

1. Watch the recording of the PDC 09 session: DirectX11 DirectCompute.

2. If session recordings is your thing there are two more on DirectCompute from nvidia's GTC09 conference 1015 (pdf, mp4) and 1411 (mp4 plus the presenter's written version of the session).

3. Over at gamedev there is an old Compute Shader tutorial. At the same site, there is a 3-part blog post on Compute Shader: Introduction, Resources and Addressing.

4. From PDC, you can also download the DirectCompute Hands On Lab.

5. When you are ready to get your hands even dirtier, download the latest Windows DirectX SDK (at the time of writing the latest is dated Feb 2010).

6. Within the SDK you'll find a Compute Shader Overview and samples such as: Basic, Sort, OIT, NBodyGravity, HDR Tone Mapping.

7. Talking of DX11/DirectCompute samples, there are also a couple of good ones on this URL.

8. The documentation of the various APIs is available online. Here are just some good (but far from complete) taster entry points into that: numthreads, SV_DispatchThreadID, SV_GroupThreadID, SV_GroupID, SV_GroupIndex, D3D11CreateDevice, D3DX11CompileFromFile, CreateComputeShader, Dispatch, D3D11_BIND_FLAG, GSSetShader.

GPGPU

2010-02-19T23:58:00.000Z

What
GPU obviously stands for Graphics Processing Unit (the silicon powering the display you are using to read this blog post). The extra GP in front of that stands for General Purpose computing.

So, altogether GPGPU refers to computing we can perform on GPU for purposes beyond just drawing on the screen. In effect, we can use a GPGPU a bit like we already use a CPU: to perform some calculation (that doesn’t have to have any visual element to it). The attraction is that a GPGPU can be orders of magnitude faster than a CPU.

Why
When I was at the SuperComputing conference in Portland last November, GPGPUs were all the rage. A quick online search reveals many articles introducing the GPGPU topic. I'll just share 3 here: pcper (ignoring all pages except the first, it is a good consumer perspective), gizmodo (nice take using mostly layman terms) and vizworld (answering the question on "what's the big deal").

The GPGPU programming paradigm (from a high level) is simple: in your CPU program you define functions (aka kernels) that take some input, can perform the costly operation and return the output. The kernels are the things that execute on the GPGPU leveraging its power (and hence execute faster than what they could on the CPU) while the host CPU program waits for the results or asynchronously performs other tasks.

However, GPGPUs have different characteristics to CPUs which means they are suitable only for certain classes of problem (i.e. data parallel algorithms) and not for others (e.g. algorithms with branching or recursion or other complex flow control). You also pay a high cost for transferring the input data from the CPU to the GPU (and vice versa the results back to the CPU), so the computation itself has to be long enough to justify the overhead transfer costs. If your problem space fits the criteria then you probably want to check out this technology.

How
So where can you get a graphics card to start playing with all this? At the time of writing, the two main vendors ATI (owned by AMD) and NVIDIA are the obvious players in this industry. You can read about GPGPU on this AMD page and also on this NVIDIA page. NVIDIA's website also has a free chapter on the topic from the "GPU Gems" book: A Toolkit for Computation on GPUs.

If you followed the links above, then you've already come across some of the choices of programming models that are available today. Essentially, AMD is offering their ATI Stream technology accessible via a language they call Brook+; NVIDIA offers their CUDA platform which is accessible from CUDA C. Choosing either of those locks you into the GPU vendor and hence your code cannot run on systems with cards from the other vendor (e.g. imagine if your CPU code would run on Intel chips but not AMD chips). Having said that, both vendors plan to support a new emerging standard called OpenCL, which theoretically means your kernels can execute on any GPU that supports it. To learn more about all of these there is a website: gpgpu.org. The caveat about that site is that (currently) it completely ignores the Microsoft approach, which I touch on next.

On Windows, there is already a cross-GPU-vendor way of programming GPUs and that is the DirectX API. Specifically, on Windows Vista and Windows 7, the DirectX 11 API offers a dedicated subset of the API for GPGPU programming: DirectCompute. You use this API on the CPU side, to set up and execute the kernels that run on the GPU. The kernels are written in a language called HLSL (High Level Shader Language). You can use DirectCompute with HLSL to write a "compute shader", which is the term DirectX uses for what I've been referring to in this post as a "kernel". For a comprehensive collection of links about this (including tutorials, videos and samples) please see my blog post: DirectCompute.

Note that there are many efforts to build even higher level languages on top of DirectX that aim to expose GPGPU programming to a wider audience by making it as easy as today's mainstream programming models. I'll mention here just two of those efforts: Accelerator from MSR and Brahma by Ananth.

Code for Parallelism Features Tour

2010-02-06T06:54:00.000Z

Last year I linked to a screencast that shows off many VS2010 features delivered by the Parallel Computing team.

There have been requests for the code used to demonstrate the features. Like with all my screencasts, you can see all the code in action, so you could simply type it in. To save you doing that though, you may download the two files with the demo code here: MM.cs and Program.cs. HTH.

Managed code and the Shell – Do?

2010-02-01T02:24:00.000Z

Back in 2006 I wrote a blog post titled: Managed code and the Shell – Don't!. Please visit that post to see why that advice was given.

The crux of the issue has been addressed in the latest CLR via In-Process Side-by-Side Execution. In addition to the MSDN documentation I just linked, there is also an MSDN article on the topic: In-Process Side-by-Side.

Now, even though the major technical impediment seems to be removed, I don’t know if Microsoft is now officially supporting managed extensions to the shell. Either way, I noticed a CodePlex project that is marching ahead to enable exactly that: Managed Mini Shell Extension Framework. Not much activity there, but maybe it will grow once .NET 4 is released...

Dev Lead Job opening on my team

2010-01-04T07:00:00.000Z

My product unit (Parallel Developer Tools) is hiring a developer lead here in Redmond. This position is specifically on the debugger feature team that I "Program Manage".

So, if you have what it takes and don't mind working with me every single day, click on the link below to read more and apply. You can also send me your resume and I'll make sure it gets to the right place and that you get a prompt response.

There is a very long job description on the Microsoft careers site under job id 707388.

Here is an excerpt from the middle (emphasis mine):

"...
We are in search of a talented and innovative senior lead software design engineer to own development of the debugging tools for data parallelism (including GP-GPU) and HPC Clusters being built by our team.

To be successful, you need to be able to guide careers, design and architect well, communicate and share the best development practices, collaborate with your peers, contribute to the vision, and code significant portions of the solution. We want to hear from you if you're passionate about making your mark in the parallel development space, improving people, and building world-class tools."

Responsibilities include:
Managing a team of senior and junior developers
Design and coding high-quality software
..."

For the full background story, requirements, qualifications and responsibilities please visit the official page.

Best of "The Moth" 2009

2010-01-01T09:01:00.000Z

Not wanting to break the tradition (2004, 2005, 2006, 2007, 2008) below are some blog posts I picked from my blogging last year. As you can see by comparing with the links above, 2009 marks my lowest output yet with only 64 posts, but hopefully the quality has not been lowered ;-)

1. Parallel Computing was a strong focus of course. You can find links to most of that content aggregated in the post where I shared my entire parallelism session. Related to that was the link to the screencast I shared of the Parallel Computing Features Tour.

2. Parallel Debugging is obviously part of the parallel computing links above, but I created more in depth content around that area of Visual Studio 2010 since it is the one I directly own. I aggregated all the links to that content in my post: Parallel Debugging.

3. High Performance Computing through clusters is an area I'll be focusing more next year (besides parallelism on a single node on the client captured above) and I started introducing the topic on my blog this year. Read the (currently) 6 posts bottom up from my category on HPC.

4. Windows 7 Task Manager. In April I shared a screenshot which was the most "borrowed" item from my blog (I should have watermarked it ;-)

5. Windows Phone non-support in VS2010. Did my bit to spread clarification of the story.

6. Window positions in Visual Studio is a long post, but one that I strongly advise all VS users to read and benefit from.

7. Bug Triage gives you a glimpse into one thing all (Microsoft) product teams do.

If you haven't yet, you can subscribe via one of the options on the left. Either way, thank you for staying tuned… Happy New Year!

Bug Triage

2009-12-07T03:56:00.000Z

In this ~~blog post~~ brain dump, I'll attempt to describe the process my team tries to follow when dealing with new bug reports (specifically, code defect reports). This is not official Microsoft policy, just the way we do things… if you do things differently and want to share, you can do so at the bottom in the comments (or on your blog).

Feature Triage Team
A subset of the feature crew, the triage team (which has representations from the PM, Dev and QA disciplines), looks at all unassigned bugs at regular intervals. This can be weekly or daily (or other frequency) dependent on which part of the product cycle we are in and what the untriaged bug load looks like. They discuss each bug considering the evidence and make a decision of whether the bug goes from Not Yet Assigned to Assigned (plus the name of the DEV to fix this) or whether it goes from Active to Resolved (which means it gets assigned back to the requestor for closure or further debate if they were not present at the triage meeting). Close to critical milestones, the feature triage team needs to further justify bugs they take to additional higher-level triage teams.

Bug Opened = Not Yet Assigned
Someone (typically an SDET from the QA team) creates the bug item (e.g. in TFS), ensuring they populate all the relevant fields including: Title, Description, Repro Steps (including the Actual Result at the end of the steps), attachments of code and/or screenshots, Build number that they observed the issue in, regression details if applicable, how it was found, if a test case exists or needs to be created etc. They also indicate their opinion on the Priority and Severity. The bug status is left as Not Yet Assigned.

"Issue" versus "Fix for issue"
The solution to some bugs is easy to determine, e.g. "bug: the column name is misspelled". Obviously the fix is to correct the spelling – still, the triage team should be explicit and enter the correct spelling in the bug's Description. Note that a bad bug name here would be "bug: fix the spelling of the column" (it describes the solution, rather than the problem).

Other solutions are trickier to establish, e.g. "bug: the column header is not accessible (can only be clicked on with the mouse, not reached via keyboard)". What is the correct solution here? The last thing to do is leave this undetermined and just assign it to a developer. The solution has to be entered in the description. Behind this type of a bug usually hides a spec defect or a new feature request.

The person opening the bug should focus on describing the issue, rather than the solution. The person indicates what the fix is in their opinion by stating the Expected Result (immediately after stating the Actual Result). If they have a complex suggested solution, that should be split out in a separate part, but the triage team has the final say before assigning it. If the solution is lengthy/complicated to describe, the bug can be assigned to the PM. Note: the strict interpretation suggests that any bug with no clear, obvious solution is always a hole in the spec and should always go to the PM. This also ensures the spec gets updated.

Not Yet Assigned -> Not Yet Assigned (on someone else's plate)
If the bug is observed in our feature, but the cause is actually another team, we change the Area Path (which is the way we identify teams in TFS) and leave it as Not Yet Assigned. The triage team may add more comments as appropriate including potentially changing the repro steps. In some cases, we may even resolve the bug in our area path and open a new bug in the area path of the other team.

Even though there is no action on a dev on the team, the bug still needs to be tracked. One way of doing this is to implement some notification system that informs the team when the tracked bug changed status; another way is to occasionally run a global query (against all area paths) for bugs that have been opened by a member of the team and follow up with the current owners for stale bugs.

Not Yet Assigned -> Resolved
This state transition can only be made by the Feature Triage Team.

0. Sometimes the bug description is not clear and in that case it gets Resolved as More Information Needed, so the original requestor can provide it.
After understanding what the bug item is about, the first decision is to determine whether it needs to go to a dev.

1. If it is a known bug, it gets resolved as "Duplicate" and linked to the existing bug.

2. If it is "By Design" it gets resolved as such, indicating that the triage team does not think this is a bug.

3. If the bug does not repro on latest bits, it is resolved as "No Repro"

4. The most painful: If it is decided that we cannot fix it for this release it gets resolved as "Postponed" or "Won't Fix". The former is typically due to resources and time constraints, while the latter is due to deciding that it is not important enough to consume our resources in any release (yes, not all bugs must be fixed!). For both cases, there are other factors that contribute to the decision such as: existence of a reasonable workaround, frequency we expect users to encounter the issue, dependencies on other team to offer a solution, whether it breaks a core scenario, whether it prohibits customer feedback on a major feature, is it a regression from a previous release, impact of the fix on other partner teams (e.g. User Education, User Experience, Localization/Globalization), whether this is the right fix, does the fix impact performance goals, and last but not least, severity of bug (e.g. loss of customer data, security threat, crash, hang). The bar for fixing a bug goes up as the release date approaches. The triage team becomes hardnosed about which bugs to take, while the developers are busy resolving assigned bugs thus everyone drives for Zero Bug Bounce (ZBB). ZBB is when you have 0 active bugs older than 48 hours.

Not Yet Assigned -> Assigned
If the bug is something we decide to fix in this release and the solution is known, then it is assigned to a DEV. This is either the developer that will do the work, or a Lead that can further assign it to one of his developer team based on a load balancing algorithm of their choosing.

Sometimes, the triage team needs the dev to do some investigation work before deciding whether to take the fix; similarly, the checkin for the fix may be gated on code review by the triage team. In these cases, these instructions are provided in the comments section of the bug and when the developer is done they notify the triage team for final decision.

Additionally, a Priority and Severity (from 0 to 4) has to be entered, e.g. a P0 means "drop anything you are doing and fix this now" whereas a P4 is something you get to after all P0,1,2,3 bugs are fixed.

From a testing perspective, if the bug was found through ad-hoc testing or an external team, the decision is made whether test cases should be added to avoid future regressions. This is communicated to the QA team.

Assigned -> Resolved
When the developer receives the bug (they should be checking daily for new bugs on their plate looking at bugs in order of priority and from older to newer) they can send it back to triage if the information is not clear. Otherwise, they investigate the bug, setting the Sub Status to "Investigating"; if they cannot make progress, they set the Sub Status to "Blocked" and discuss this with triage or whoever else can help them get unblocked. Once they are unblocked, they set the Sub Status to "Working on Solution"; once they are code complete they send a code review request, setting the Sub Status to "Fix Available". After the iterative code review process is over and everyone is happy with the fix, the developer checks it in and changes the state of the bug from Active (and Assigned to them) to Resolved (and Assigned to someone else).

The developer needs to ensure that when the status is changed to Resolved that it is assigned to a QA person. For example, maybe the PM opened the bug, but it should be a QA person that will verify the fix - the developer needs to manually change the assignee in that case. Typically the QA person will send an email to the original requestor notifying them that the fix is verified.

Resolved -> ??
In all cases above, note that the final state was Resolved. What happens after that? The final step should be Closed. The bug is closed once the QA person verifying the fix is happy with it. If the person is not happy, then they change the state from Resolved to Active, thus sending it back to the developer. If the developer and QA person cannot reach agreement, then triage can be brought into it. An easy way to do that is change the status back to Not Yet Assigned with appropriate comments so the triage team can re-review.

It is important to note that only QA can close a bug. That means that if the opener of the bug was a PM, when the bug gets resolved by the dev it may land on the PM's plate and after a quick review, the PM would re-assign to an SDET, which is the only role that can close bugs. One exception to this is if the person that filed the bug is external: in that case, we leave it Resolved and assigned to them and also send them a notification that they need to verify the fix. Another exception is if specialized developer knowledge is needed for verifying the bug fix (e.g. it was a refactoring suggestion bug typically not observable by the user) in which case it is fine to have a developer verify the fix, and ideally a different developer to the one that opened the bug.

Other links on bug triage
A quick search reveals that others have talked about this subject, e.g. here, here, here, here and here.

Your take?
If you have other best practices your team uses to deal with incoming bug reports, feel free to share in the comments below or on your blog.

Parallel Computing Features Tour in VS2010

2009-11-17T23:26:00.001Z

Just realized that I have not linked from here to a screencast I recorded a couple weeks ago that shows the API, parallel debugger and concurrency visualizer in VS2010. Take a few minutes to watch the VS2010 Parallel Computing Features Tour.

MPI Cluster Debugger launch integration in VS2010

2009-11-15T07:55:00.000Z

Let's assume that you have all the HPC bits installed and that you have existing MPI code (or you created a "Hello World" project using the MPI project template). Of course, you create a single MPI application and at runtime it will correspond to multiple processes (of the same app) launched on multiple nodes (i.e. machines) on the cluster. So how do you debug such a situation by simply hitting the familiar "F5" keystroke (i.e. Debug -> Start Debugging)?

WATCH IT INSTEAD OF READING ABOUT IT
If you can't bear to read through all the details below, just watch this 19-minute screencast explaining this VS2010 feature. Alternatively, or even additionally, keep on reading.

REQUIREMENT
When you debug an MPI application, you would want the copying of resources from your client machine (where Visual Studio is installed) to each compute node (where Windows HPC Server is installed) to take place automatically for you. 'Resources' in the previous sentence includes your application binary, plus any binary or data dependencies it may have, plus PDBs if needed, plus the debug CRT of the correct bitness, plus msvsmon for remote debugging to work. You would also want, after copying is complete, to have your app and msvsmon launched and attached so that you can hit breakpoints back in Visual Studio on your client machine. All these thing that you would want are delivered in VS2010.

STEPS TO F5
1. In your MPI project where you have placed a breakpoint go to Project Properties -> Configuration Properties -> Debugging. Ensure the "Debugger to launch" combo box value is set to MPI Cluster Debugger.

2. There are a whole bunch of properties here and typically you can ignore all of them except one: Run Environment. By default it is set to run 1 process on your local machine and if you change the number after that to, for example, 4 it will launch 4 processes of your app on your local machine.

You want this to run on your cluster though, so go to the dropdown arrow at the end of the Run Environment cell and open it to expose the "Edit Hpc node" menu which opens the Node Selector dialog:

In this dialog you can enter (or pick from a list) the cluster head node name and then the number of processes you want to execute on the cluster and then hit OK and… you are done.

3. Press F5 and watch your breakpoint get hit (after giving it some time for copying, remote execution, attachment and symbol resolution to take place).

GOING DEEPER
In the MPI Cluster Debugger project properties above, you can see many additional properties to the Run Environment. They are all optional, but you may want to understand them in order to fine tune your cluster debugging. Read all about each one of these on the MSDN page Configuration Properties for the MPI Cluster Debugger.

In the Node Selector dialog above you can see more options than just the Head Node name and Number of Process to run. They should be self-explanatory but I also cover them in depth in my screencast showing you an example of why you would choose to schedule processes per core versus per node. You can also read about these options on MSDN as part of the page How to: Configure and Launch the MPI Cluster Debugger.

To read through an example that touches on MPI project creation, project properties, node selector, and also usage of MPI with OpenMP plus MPI with PPL, read the MSDN page Walkthrough: Launching the MPI Cluster Debugger in Visual Studio 2010.

Happy MPI debugging!

Parallel Debugging

2009-11-13T05:06:00.000Z

Using Visual Studio 2010 parallel debugging is easy. Two new debugging windows provide a total view of the internals of your PPL and TPL applications with hints on where to start investigations. These are not mere extensions to VS, but tightly integrated with the rest of the debugger experience, so you don't need to learn many new techniques. Use them in your program to eclipse bugs from existence!

One of the most FAQ I receive is links to VS2010 parallel debugging content and rather than keep sending many, I decided to gather them all under one permalink, hence this multi link blog post.

- MSDN Magazine article on Parallel Debugging.
- Screencast of sample code from the article.

- MSDN Walkthrough: Debugging a Parallel Application (VB, C++, C#).
- Screencast of walkthrough for Parallel Stacks.
- Screencast of walkthrough for Parallel Tasks.

- MSDN "How To" on Parallel Tasks.
- MSDN "How To" on Parallel Stacks.

- Detailed blog post on Parallel Tasks.
- Detailed blog post on Parallel Stacks.
- Detailed blog post on Parallel Stacks - Tasks View.
- Detailed blog post on Parallel Stacks - Method View.

- Download slides on Parallel Tasks and Parallel Stacks (pptx).

If you have questions on these, please post to any of the parallel computing forums or the debugging forum (your question will be routed to me if nobody else can answer it).

"Parallel Programming Talk" show

2009-11-12T04:09:00.000Z

Over at the Intel Software Network Aaron Tersteeg runs a "Parallel Programming Talk" audio show on which I was invited as a guest (for the 55th episode) to talk about Microsoft's parallelism offerings in Visual Studio 2010. The call started at 7:45AM, so if my voice sounds croaky to you, now you know why ;)

Check out the 20-minute chat (and related hyperlinks) on Aaron's blog.

Message Passing Interface (MPI)

2009-11-12T00:01:00.000Z

So you have installed your cluster and you are done with introductory material on Windows HPC. Now you want to develop an application with the most common programming model: Message Passing Interface.

The MPI programming model is a standard with implementations from many vendors. For newbies (like myself!), I have aggregated below links for getting started.

Non-Microsoft MPI resources (useful even if you are not on the Windows platform)
1. Message Passing Interface on wikipedia.

2. The MPI standard.
3. MPICH2 - an MPI implementation.
4. Tutorial on MPI by William Gropp.
5. MPI patterns presented as a tutorial with sample code.

6. THE official MPI Forum (maintains the standard) including the wiki discussing the MPI future.

7. Great MPI tutorial including at the end the MPI Exercise.

8. C++ MPI Exercises by John Burkardt.

9. Book online: MPI The Complete Reference.

MS-MPI
10. Windows HPC Server 2008 - Using MS-MPI whitepaper (15 page doc).

11. Tracing MPI applications (27 page doc).

12. Using Microsoft MPI (TechNet section).

13. Windows HPC Server MPI forum (for posting questions).

MPI.NET
14. MPI.NET Home Page (not owned by Microsoft).
15. MPI.NET Tutorial.

16. HPC Development using F# using MPI.NET (38 page doc).

Next time I'll post resources for the Microsoft Cluster SOA programming model - happy coding...

Windows HPC Server links

2009-11-11T03:27:00.000Z

I've already described how to setup a Windows HPC Server for development. Before you dive into developing for the cluster, if you are new to this it is probably a good idea to learn the basics by reading some overview material. Below is a list of links.

Direct Links to Windows HPC content
1. Windows HPC Server 2008 Overview Datasheet (4 page pdf).

2. Windows HPC Server 2008 Technical Overview (32 page doc).

3. Windows HPC Server 2008 Getting Started Guide (26 page doc) which actually is available online as part of the TechNet technical library section on Windows HPC Server 2008, which includes much more useful data.

4. Windows HPC Server 2008 Job Scheduler (38 page doc).
5. Windows HPC Server 2008 Job Templates (56 page doc).

6. Developing for the Windows HPC Server 2008 Platform (16 page doc or pdf version).

Windows HPC sites
7. Windows HPC Forums.

8. HPC Developer Resources.

9. Windows HPC Server 2008 Resource Kit - Developer.

10. Windows HPC Server 2008 - TechNet.

11. The Windows HPC Team Blog.

HPC Course
12. High-Performance Computing Fundamentals Course (pluralisight)
13. Classic HPC Development using Visual C++ (course slides and materials in a ZIP). Author's blog post.
14. From sequential to parallel code (course slides and materials in a ZIP). Author's blog post.

Next time I will post resources specific to the most popular programming models for the cluster today: MPI and Cluster SOA - until then, happy reading!

Extension Manager in Visual Studio 2010

2009-11-10T18:36:00.000Z

One of the powerful aspect of Visual Studio is its ability to be extended and many people do that. You can find numerous extensions at the Visual Studio Gallery. The VSX team links to a 4-part blog series on how to create and share templates. You can also look find extension examples on the vsx code gallery.

With Visual Studio 2010, you can search for items and install them directly from within Visual Studio's new Extension Manager. You launch it from the Tools menu:

When the dialog comes up, be sure to explore the various actionable areas on the left and also note the search on the right. For example, I typed "MP" and it quickly filtered the list to show me the MPI Project Template:

Others have written about this before me, just bing Extension Manager (and note that Beta2 introduced changes, some of which you can witness in the screenshot above).

Core debugger enhancements in VS2010

2009-11-10T03:15:00.001Z

Since my team offers "parallel debugging", we refer to the team delivering all the other debugging features as the "core debugger" team. They have published a video of new VS2010 debugger features that I encourage you to watch to find out about enhancements with DataTips, breakpoints, dump debugging (inc. IL interpreter) and Threads window.

The raw list of features with short description is also here.

Dump debugging with Parallel Stacks

2009-11-08T21:57:00.000Z

One of the areas where we fixed many bugs for Beta2 in our parallel debugging windows is with regards to managed dump debugging. So it was really cool to see Tess use the Parallel Stacks window in that scenario in her video demo with Scott.

Other than the neat ability to open managed dumps in VS2010, Parallel Stacks was the only debugging feature she needed for diagnosing the issue! Check out the video, definitely worth 10 minutes of your time.

Slides for Parallel Debugging windows

2009-11-08T03:50:00.000Z

Recently I gave a talk at our Microsoft Shanghai offices on Parallel Programming so I had to update my existing Beta1 deck to Beta2 content. Specifically for Parallel Tasks and Parallel Stacks, I used 5 slides to accompany the demo.

In case you are giving talks on parallelism within Visual Studio 2010, please feel free to download and use the updated parallel debugger slides (pptx).

TIP: The slides have animations so be sure to F5 the deck for the full benefit and they also have text in the Comments section so be sure to see them at design time too.

MPI Project Template for VS2010

2009-11-07T04:54:00.000Z

If you are developing MS MPI applications with Visual Studio 2010, you are probably tired of following some tedious steps for every new C++ project that you create, similar to the following:

1. In Solution Explorer, right-click YourProjectName, then click Properties to open the Property Pages dialog box.

2. Expand Configuration Properties and then under VC++ Directories place the cursor at the beginning of the list that appears in the Include Directories text box and then specify the location of the MS MPI C header files, followed by a semicolon, e.g.
C:\Program Files\Microsoft HPC Pack 2008 SDK\Include;

3. Still under Configuration Properties and under VC++ Directories place the cursor at the beginning of the list that appears in the Library Directories text box and then specify the location of the Microsoft HPC Pack 2008 SDK library file, followed by a semicolon, e.g.
if you want to build/debug 32bit application:
C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\i386;
if you want to build/debug 64bit application:
C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64;

4. Under Configuration Properties and then under Linker, select Input and place the cursor at the beginning of the list that appears in the Additional Dependencies text box and then type the name of the MS MPI library, i.e.
msmpi.lib;

5. In the code file
#include "mpi.h"

6. To debug the MPI project you have just setup, under Configuration Properties select Debugging and then switch the Debugger to launch combo value from Local Windows Debugger to MPI Cluster Debugger.

Wouldn't it be great if at C++ project creation time you could choose an MPI Project Template that included the steps/configurations above? If you answered "yes", I have good news for you courtesy of a developer on our team (Qing).

Feel free to download from Visual Studio gallery the MPI Project Template.

Instructions for collecting dumps in VS2010

2009-10-30T03:11:00.000Z

Came across this on an internal email thread and thought it would be useful to share, in case you run into a VS crash and wish to collect the information needed by Microsoft support.

1. Start 2 instances of Visual Studio 2010 (devenv.exe).
2. In the first one select Tools -> Attach to Process.
3. Click the Select button for “Attach to:” and select Managed and Native.
4. Select the other devenv from the list and click Attach.
5. Switch to the other devenv.
6. Load your solution and try to reproduce the crash.
7. When you hit the crash the debugger should automatically break.
8. Select Debug -> Save Dump As, then set the type to minidump with heap and save the file to a location with lot’s of space.

Note: If you are running the VB profile you will not see the Save Dump As menu item. To add this menu item:
a. Select Tools -> Customize
b. Select the Commands tab
c. Select Debug from the “Menu bar” dropdown
d. Click “Add Command…”
e. Select Debug from the Categories list.
f. Find the “Save Dump As” entry in the Commands window.
g. Click OK (the “Save Dump As…” command is added to the top of the Debug menu).
h. Click “Close”

Positioning the Parallel Stacks window

2009-10-23T04:22:00.000+01:00

When we developed the Parallel Debugging windows, we had to choose their default position in Visual Studio. For Parallel Tasks it was easy: we chose wherever the Threads window appears by default, since the two windows share characteristics in usage and appearance. Parallel Stacks may be similar to the basic concept of the Call Stack window, but it has slightly different usage patterns and very different UI. Here I'll describe our rational for the current choice for the Parallel Stacks window, offering tips and request feedback.

It was more challenging that just picking a position at random, especially due to the window having high screen real estate demands for typical applications – after all, it is a graphical view of all the call stacks of all the threads/tasks in your application. So, we quickly dismissed having it docked at the bottom or top since those are typically narrow in height windows. Some of us believed that the document editor area would be a good place for it since it has more screen real estate to offer. That approach breaks down when you realize that the Parallel Stacks window is not just presenting data, but it also allows you to quickly switch to any stack frame of any thread (with a double click). When that user action takes place, the code editor navigates to the method context and thus steals the focus from the Parallel Stacks window which gets pushed to the back (this also happens when you simply toggle the "Show External Code" option). So if you want to navigate to various places in the code via a series of examinations, that position quickly becomes irritating (just like if you docked Solution Explorer in the code editor area).

So we decided the default position to be floating! The idea is that you can resize it based on your preferences and move it in and out of the way if necessary: you get to see as much of it as you want and you keep it on top of other windows (typically to the right side). The real expectation is that you will drag the floating window to your second monitor and maximize it there.

Two other options came close to be chosen and I'll offer them here as tips in case they suit you better (in particular if you don't have a second monitor or when debugging on the go on your laptop). One is to dock it within the document editor, but in a New Vertical Tab Group. This way it takes up the space it typically needs, while keeping your code editor in view when you switch stack frames.

The other option is to dock the Parallel Stacks window where the Solution Explorer window lives. That way you can resize that group of windows horizontally to see more content (and use for navigation) without taking too much away from your code view (assuming your lines of code are not extremely long). When you are done using it, you can quickly resize it back to normal.

It is amazing how much thought/discussion goes into, what many people would consider, an insignificant detail, but that is the kind of thing I enjoy, which I guess is why I do what I do. In any case, I am eager to hear how we got this decision wrong, so please let me know.

Window Positions in Visual Studio

2009-10-21T01:04:00.000+01:00

There are many toolwindows in Visual Studio and each release keeps adding new ones. This is great for offering more features and increasing productivity, but it also adds to the number of windows we need to manage and it makes it even more important to understand how the positioning works in VS (and most of this applies to toolbars as well).

The first important thing to understand is that there are 3 different window configurations persisted per user on the machine.
1. The standard configuration: the default when you open a solution in Visual Studio.

2. The debugging configuration: the one when you hit F5 (and a little window earthquake takes place while everything readjusts).

3. The often forgotten, full screen configuration when you hit Shift+Alt+Enter, in any of the previous configurations

Once you understand the above, you should never utter the words: "didn't I just close that window, why did it come back", which we typically hear from presenters on stage after hiding, for example, Solution Explorer and then when they hit F5 it comes back. You need to hide (or otherwise position) each window for every one of the 3 configurations if you really want it to be like that all the time. Another example is if you decide to move the Breakpoints window in standard configuration, don't be surprised if it pops up somewhere else when you debug: to make sure it appears at the same place always, you need to move it to where you want in all configurations. Understanding the above also means that you won’t be looking for the Parallel Tasks window when in standard configuration since it is a debugging window and hence applicable only when in debugging mode. I hope the point is made.

The second most important thing to understand is that each window has a default position for each one of the configurations. Those default positions are different per profile that you chose the first time you run Visual Studio. Just because you have some defaults, it doesn't mean you need to stick with them, so choose what is best for you.

So what are the two basic choices we make per window?
A. Is it hidden or visible. If it is hidden, we should at least know where it lives on the menus so we can bring it up when we need it (and thus avoid the "out of sight, out of mind" pitfall). So go ahead and explore all the menus under the View menu and don't forget the ones under Architecture, Test and Analyze:

Don't forget that menu options change when you are in debugging mode, so explore the Debug menu once you've hit a breakpoint:

TIP: If you are not familiar with all of these windows above, maybe you are not the VS power user you thought you were ;-)

B. If you've opened a window, it has a default position that each profile team chose. You need to decide where it lives: floating (this is usually a temporary state or used on the secondary monitor, combined with MAX options), docked to the left/right/top/bottom (further decision if it is pinned or autohides controlled via the pushpin) or even docked to the center as a tabbed document (see the drop down options in the screenshot of the link).

With regards to moving a window around for docking I often observe people fighting the IDE, especially since each docking target can have multiple side-by-side groups. It is quite simple. Drag the window towards your desired area and look out for blue shaded region to appear, it represents where your window will go: if you like that position then you can release the mouse button; if you don't, keep moving the window. Along with the blue shadowed region, you also get some arrow targets appearing that you can drag the mouse pointer over (while still keeping the mouse button down so as not to release the window). Here is a screenshot of dragging the Parallel Stacks window towards the top and moving the mouse over the left arrow box that appeared.

After this screenshot was taken I released the mouse button and a new window group was created next to the Parallel Tasks window that was occupied by the Parallel Stacks window. If I wanted it to join the same group as the Parallel Tasks window, I would have dragged towards the middle box and released then.

TIP: if you want to drag a window out of its group to floating, don't drag it by the title bar, but instead drag it from its tab. I.e. in the following screenshot, dragging from the title bar (red circle) will float the entire window group whereas dragging from the tab (blue circle) will float just the Parallel Tasks window.

Finally, you can backup and restore the window layouts via the Tools-> Import and Export Settings menu. FYI, they are stored on your machine in this folder: "%userprofile%\AppData\Roaming\Microsoft\VisualStudio\9.0". Look for the windows.prf and windowsidx.prf files. I found that via this msdn link.

Visual Studio 2010 Beta2 (build 21006)

2009-10-19T17:11:00.000+01:00

MSDN Subscribers can download Visual Studio 2010 Beta2 today with general public availability to follow in a couple of days. Official blog posts by Soma.

I have been using this since October 8th (2 days after it was built as you can tell from the build number) and it is amazingly better than the Beta1 in terms of performance, stability and new features too. It is worthy of its "go-live" license.

More posts to follow from me on VS2010; for now, I just updated the VS Profile screenshot and the Parallel Debugging content.

Parallel Tasks and Parallel Stacks content updated for VS2010 Beta2

2009-10-19T16:59:00.000+01:00

For VS2010 Beta2, rather than write new introductory and overview posts on my two favorite debugger windows (Parallel Tasks and Parallel Stacks), I thought I’d update 4 of the existing blog posts on those topics.

Please read the updated text and enjoy the updated screenshots
- Parallel Tasks
- Parallel Stacks
- Parallel Stacks – Tasks View
- Parallel Stacks – Method View

Also, I have updated the recordings behind the screencast URLs on channel9:
- Parallel Stacks
- Parallel Tasks

Each one of the links above has a comment section, feel free to use it for feedback and questions!

Installing HPC Server 2008

2009-10-19T04:47:00.000+01:00

Recently I decided to play around with developing for a cluster on the Microsoft platform and below are the steps I had to take with regards to installation.

First I gathered a number of old machines at the office, in my case that is 3 dual-core boxes, but you could have done it with just 2 PCs (or many more of course). On each one of those you must install: Windows HPC Server 2008 (link to trial)

a. This includes a trial of the Windows Server 2008 RTM 64-bit operating system (SRVHPC_EN.iso). I already had Windows Server on a separate disc, so I installed what I had. I did not try R2, which is the same code base as Win7, but that should work too. Note that this is just the vanilla operating system (even Standard Edition is good enough) and the important part is the 64-bit. I will assume that you are all capable of installing an OS, so no more instruction or special consideration needed here (window update, join domain, add users etc).

b. The second part of the link above is an add-on to the operating system, namely the HPC Pack 2008 (HPCEval.iso). It is a wizard with a set of steps to complete which are all fairly explanatory of the "click Next" flavor. After installation of the HPC Pack, the HPC Cluster Manager application runs up automatically (or else you can run it through the Start menu) for further step-by-step configuration in an easy to follow outofthebox To-do List.

Here are some tips for step b from above, for your developer cluster setup:

i) In the HPC Pack wizard, you will create one of the machines as the Head Node.

ii) In the HPC Pack wizard, the remainder machines will be created as the Compute Nodes.

iii) In Cluster Manager, click on "Configure your network" and choose topology 5 "All Nodes only on an Enterprise Network".

iv) In Cluster Manager, click on "Add compute nodes" and then select "Add compute nodes that have already been configured".

v) In Cluster Manager, click "Change the role of the head node" and also make it a WCF Broker Node.

vi) In Cluster Manager, "Validate your cluster" under Diagnostics.

It may be useful to cross-reference the text above with the following screenshot of HPC Cluster Manager

Now that your cluster is installed, you need to set up your development machine (x86 or x64) for developing on this cluster. Here are the things you need to install:

1. Visual Studio 2010

2. HPC Pack 2008. This is essentially the same msi you installed on the server (step b above), but here you will choose the 3rd option Client Utilities (which will probably be your only enabled option as per my screenshot further above).

3. HPC Pack 2008 SDK. This allows writing client applications that interact with the Job Scheduler running on the Head Node. MSDN has a dedicated page to the HPC SDK.

4. (optional) Install the separate MPI Project Template, for VS2010.

You are now ready to develop your cluster applications on your development machine and to debug/deploy them on your cluster. More on that in future blog posts.