Friday, April 23, 2010

Interviews of Thought Leaders - Alan Page

As promised in my post, Interviews of Thought Leaders in Software Testing, here is the first interview. This interview is with Alan Page. Alan is the Principal SDET at Microsoft Corporation. In the past, he has also played the role of Director, Test Excellence at Microsoft. Along with being a frequent conference speaker and author, Alan is a prominent thought leader in software testing. Alan also writes on his blog at

Though I had been reading about Alan and his thoughts before, my interaction with him began in the year 2008. I respect his vast knowledge, professional responses and helpfulness. In this interview, you can read Alan's thoughtful and detailed responses to what I hope are pertinent questions from all of us in the software testing space. Who know, some of Alan's replies may subtly lead you to even greater testing successes.

1. We know that you are a very senior person in the area of software testing. What roles have you played in your software testing career so far and what other roles would you like to play in the future?

[Alan] I don’t know that I’m senior – perhaps just highly exposed. I started testing before I knew what testing was – I was hired to perform product support for a music software company and was notified the first day that I was also the (only) software tester. When I joined Microsoft a few years later I tested configurations and user interface for the Windows 95 networking components. My role on the windows 98 team was primarily API testing – with a slant towards compatibility testing. Later on, I ended up working a lot with the kernel and drivers and ended up owning those areas of Windows CE as a test lead for just over a year. I’ve always felt I was a stronger performer as an Individual Contributor than as a lead or manager, so when I stopped my stint as a lead I moved into a Test Architect role (which was really just a fancy title for a senior non-manager tester at Microsoft). As a TA, I functioned as a technical leader for the CE test team – acting as a go to person, a reviewer, and a strategist for the team. I spent about five years in Microsoft’s Engineering Excellence group – the first two as a technical instructor and pseudo-consultant, and the last two and a half as the Director of Test Excellence where I was sort of the overseer for Testers at Microsoft – in charge of technical training, long term strategy, etc. In my new role on the Office Communicator team I don’t test a specific part of the product – instead I try to identify everything I can that will help make the test team more effective and improve overall software quality.

In the future – at least for the next few years, I see more of the same – perhaps with larger teams or tackling even harder problems.

2. What are the different kinds of tests that you have done or led/ managed/ directed?

[Alan] You name it, and I’ve probably done it (although not always successfully).

3. Could you tell us about a particularly tough challenge that you faced in your testing career? How did you resolve it?

[Alan] My toughest challenge is always my next challenge – I’m always looking for new challenges. However, I’ve probably had a few particularly interesting challenges. The first that comes to mind was when I was first asked to join the Windows ME team.. The debug version had a ton of great error checking that enabled application developers to find a slew of programming errors early. The problem was that the debug version of Windows had been ignored for completely a year and wouldn’t even boot. In addition to kernel and driver testing, I was asked to “fix” debug windows (and own debugger development as well). I had no idea what I was doing, yet somehow happened to make it all work. It was one of those times when I felt I was completely over my head, but somehow managed to figure it out. I’ve always been able to learn quickly and to hold a bunch of disparate information in my head and somehow put it together in the right order.

4. What is the one achievement in your testing career that you are most proud of?

[Alan] It’s probably writing (and finishing) How We Test Software at Microsoft. Sometimes I still can’t believe I took on that project, but I’m glad I did.

There are plenty of things I wish I would have done differently in that book, but I’m still proud of getting it done and that people like what we came up with.

5. How do you think the software testing space is transforming now? What specific software testing roles/ jobs would increase in the future? What skills and knowledge do you think will be required by software testers in the coming years?

[Alan] Software is getting more interconnected and complex rapidly – the challenge of testing this interconnected and complex software is growing rapidly.

It’s really not enough to find surface level functionality bugs anymore – the important bugs are lurking within the connection points where functions, modules, and features connect. Testers are learning that they need to have a strategy for determining where to target testing, and what types of testing to perform to get the maximum value from their time.

Testers spend a lot of time looking at data – today, it’s most often bug data and pass rates, but many testers also look at coverage reports, code churn or complexity metrics. The problem is that most testers aren’t data analysts or statisticians, and aren’t really very good at figuring out what action to take from data. The result is that the data is either ignored, or misinterpreted. I think the business intelligence role will become more important in the future of test – without meaningful analysis of the massive amount of test collateral generated by software teams, test will never be able to catch up with the growing complexity of future applications.

6. What qualities and abilities do you like in the younger software testing professionals? Which qualities you do not like in the younger crowd?

[Alan] You’re making me feel old :}!. Beyond anything else, I think a passion for learning is the most important attribute of a tester. I also think it’s important to point out that “passion” has to be demonstrated rather than stated. In other words, I admire testers who seek new knowledge and actively look for ways to understand how to apply that knowledge – rather than those testers who state random bits of new information and look for ways to dismiss that knowledge without taking the time to understand it.

Well, this was the first part of our interview. Answers to the following questions are still to come:

* What skills and qualities would you look for if you interview a software test engineer for the purpose of hiring?

* What advice would you give to the software test leads/ test managers working in various organizations? What should they do if they want to progress in their testing career?

* Many software testing professionals find effort estimation a bit challenging. What advice can you give to arrive at a fairly accurate estimate, especially in automation projects?

* We know that you handle sizable responsibilities. In addition, you are a frequent conference speaker and author. Could you give us some tips on how do you manage your time and energy so successfully?

* Do you think that you made the right choice in selecting software testing as a career?

* Would you like to share any other advice/ wisdom with us readers?

Be sure to check out Alan Page's replies to these questions shortly. Is there any other question that you would like to ask Alan?

[Update on 29 Apr 2010] You can now go to the second and final part of this interview.


  1. Interview was brief concentrating on how Alan's test career had developed, high level view about testing,its future but its very brief , i guess because of time constraints.

  2. Here are a couple of questions from another reader. I wonder about Alan's response to these.

    1. What is that we software testers do not test for in the SDLC life cycle and that has major impact on later stages?

    2. Whats the value add that testers bring to the organization ? Finding bugs /testing functionality etc is part of job responsibility, can we call this as value add?

  3. 1. depends - but I'll explain why. Every team and product is different, so the things that testers do not test for can vary from team to team. I advise teams to look at the types of issues they are finding late stages and spend a bit of time brainstorming about types of testing (or quality improvements) that could have found the problem earlier.

    This is a just lightweight form of root cause analysis (rca), but can be pretty effective. Note, that there may be multiple methods for finding a bug earlier, and that's ok. Nobody ever said there had to be one, and only one, root cause.

    That said, I do see some things always come up. I know that many testers don't have an opportunity to see the source code for the apps they test, but one are where testers can help is with code reviews. Code reviews are very effective in finding bugs early. Programmers, of course, usually perform code reviews, but they don't often ask the right questions. By that I mean they review code to see if it will work as written. Testers ask better questions - like "in what situations could this condition fail", or "how would we test for this code path".

    Another approach that I like is to create full end-to-end scenarios covering a broad part of the application as early as possible. Do it before the application is available if possible, otherwise just do it early. Then run your scenarios. Early on, most, if not all, will probably fail - but that's ok. Keep doing whatever other testing you're doing "most of the time", but run the scenarios every week or so (timing depends on product size) and report results. Over time, these scenarios will move from "red" to "green". Also encourage testers to vary their approach and explore the scenario - have them pair up or trade off to remove any facets of fatigue.
    I've observed that many issues discovered late are integration issues - flaws where all of the moving pieces fit together. By focusing on end-to-end scenarios early, these integration issues are often discovered much earlier.

    (I have blog entries going up soon on both of these topics if you'd like to know more).

    AND - I'll answer the value-add question in a new comment .

  4. 2. Whats the value add that testers bring to the organization ? Finding bugs /testing functionality etc is part of job responsibility, can we call this as value add?

    I hate the role of testing as "verifying functionality" - I prefer software to at least function when I start testing it - that way I can look for more interesting issues. As for bugs, I don't see bug finding as part of the role either - they are just a side effect of what we do.

    Our value add is to uncover product issues and risk. We do that by analyzing, questioning and exercising the product with a critical eye. Testers uncover issues and unearth information about the product that would not be found otherwise - at least by anyone other than customers. As a tester, rather than discover issues that make the developer say "oops" I love to discover things that make the developer say "that's incredible - how in the world did you discover that?" I find that a portfolio of these "discoveries" provides plenty of value-add for the stakeholders.

  5. Very well put by Alan! I especially like the two suggestions for testers:
    1. Code reviews
    2. End to end scenarios

    Would love to learn more of Alan's thoughts on his upcoming blog posts on these topics.

    And yes, the testers add value by discovering product issues and identifying risks. We are able to do this because we consider the product from different perspectives (including the perspective of the customer/ end-user).

  6. The discussion is informative as a whole. Two things took my attentiom most are;
    1.Business Intelligence Role
    2.Passion for Learning

  7. Fantastic insights.

  8. Such insights help a lot.
    Manas( )