Interviewing a Testing Expert - Jonathan Kohl from Kohl Concepts [Part-2]

Friends, I am back with the final part of the interview with Jonathan Kohl, co founder of Kohl Concepts and a world-renowned expert in Software Testing. In case, you missed out the first part, I would strongly suggest you to read it first before proceeding to read this one. So here we go:

Debasis: Tell me of any situation when you had wished you were NOT a tester!

Jonathan: Testing can be political. The wishes of the technical staff and executives sometimes don't map to the reality of what they are delivering. That means you can be the least popular person on the team, and it can be discouraging in the moment.

When you challenge someone's personal faith or political views, they get upset. If you challenge their favorite technology or methodology, they behave similarly. Confronting that requires a lot of courage, integrity, ethics and a thick skin. They respect you for it in the long run, but it’s tough to take in the short run. I've learned over the years that if I am tempted not to say something, that means I should say it, but with tact, diplomacy and empathy. It's served my career well, but it's tough in the moment to challenge something that is distracting us from providing value. Anytime someone has said: “don’t raise that important issue, it will be a career-limiting move” I have listened to my conscience, spoken up, faced initial resistance, but over the long term, they were always career-catapulting moves.

Debasis: Has the profession (testing) ever affected your personal life? If yes, how?

Jonathan: I think about testing everything. Sometimes it's hard to take time off and not analyze all sorts of systems and look for weak points. Now that we can work from home, or work on side projects and things like that much more easily with the Internet, it's hard to turn work off and relax sometimes. The '55 Pontiac and car club are good outlets, and when I have the time for it, so is performing music.

Sometimes my friends don’t want a thorough analysis on their ideas, they just want me to empathize, so once in a while they ask me to turn off my tester brain.

Debasis: What do you think as the most essential skills that make a great tester?

Jonathan: Mostly, skills related to investigation and communication. You can develop those just about anywhere. Most testers have something in their background that they learned some basic investigative skills from, such as scientific experiments at school, or reading detective novels. Sometimes making that connection to your testing work can be a real eye opener. Good investigators see things that others miss, and are able to find problems and report them clearly. There are many different skills testers draw on to do that, and that depends on their background and training. It doesn’t hurt to learn some technical skills so you can help track down issues more quickly and communicate with programmers and other technical people in terms they understand.

Good investigators have solid general knowledge. They know what new technologies are up and coming, and they have a deeper knowledge of the systems they are working with. What are the inherent problems with the technology? They have a catalog of problem areas, and remember things from their own experience to draw on as they look for potential problem areas. They also have a deep knowledge of the users of the software. Who are our users? What problems are they trying to solve with our software? What are their goals with our software? They also have a good understanding of the team, and the equipment available for testing. They also have an understanding of a lot of different testing techniques, and how to identify risks or opportunities, and recommend techniques to look into those areas more. They have at least a basic technical understanding of the software and the environment in which it operates. All of that helps them strategize, plan and generate a lot of testing ideas that leads to important information about the product they are testing.

It's not a skill, but having ethics and integrity is also important as a tester. Stakeholders need to be able to trust us to tell them the truth. When we violate that trust (even by giving in to their pressure in the short term) we lose our effectiveness.

You also need an interest in discovering problems. Sometimes testing can be tedious, and if you are creative, you can turn any situation into something interesting for you. Without that interest in finding and reporting problems, it’s going to be difficult to enjoy testing and want to develop skills.

Debasis: Tell me about the most fascinating bug that you have encountered in your entire testing career.

Jonathan: Several intermittent bugs have been fascinating. One of the most difficult to track down was due to a social issue. Every release, one of our biggest clients would file a bug report. We would try to repeat it, and just couldn't get a repeatable case. Our software could dial home with a stack trace on error, so we asked the clients to send it in. The programmers looked at the code and tried to make it as robust as possible, then we'd release it, still get the same bug report from the client. The team sent me to spend the day with one of the client's users to get more information, and I was shocked at what I learned. It turned out that the software would crash on startup, but the user blamed herself for that error because they had learned a workaround (and hadn't told us about it) and she forgot to go through those workaround steps. After she went through the workaround, a few minutes later the software crashed and she showed me the familiar stack trace. I got her to give us the information from the first crash, which is really where the bug was located. The stack trace after that was a red herring because the system was in an unstable state, and could fail any number of ways. I learned to pay a lot more attention to the users and the social context they use our software in, and to incorporate that into my testing. That led me to look into usability testing and user scenarios instead of just looking at the requirements and functional aspects of the application.

Debasis: What single thing would you want to tell every newbie who is struggling in the early stage of building software testing career?

Jonathan: Concentrate on developing your skills as a software tester. There are a lot of distractions from skill development, whether it is to pass some multiple choice certification exam, learning about the latest fad software development methodology. We also talk a lot about analysis and planning more than test execution. (There is value to all of those activities, but not at the expense of test execution skill development.)

Certifications, tools, methodologies and practices change over time, but demand for good testing skills remains constant. For example, if you are testing on an Extreme Programming team, the novelty of the unique environment and context wears off, and you need to deliver skilled testing. Learning the methods, values, shared language and rituals only gets you so far. Your team is going to expect you to add value as a tester. It’s kind of like extreme ironing. Once they get over the novelty of you ironing in an unusual context, your customers will expect nicely pressed shirts. If you focus on improving your test execution and reporting skills, you will find more interesting information, and will become more valuable to the organizations you serve. Good testing and communication skills transcend methodologies and tools, and good testers can provide value anywhere if they have the skills.

Some things to consider: Develop your investigative and reporting skills. Be a software investigator, and practice your test idea generation. Use your background to leverage your investigative side. If you have a programming background, use that. If you don’t look at other areas you developed skills as an investigator. I’ve worked with wonderful testers with backgrounds from accounting, writing, studying history, programming, systems administration, investigative journalism and others. Most people have some sort of investigative experience in their background. Find yours, and apply it to software testing. Failing that, read detective stories and watch investigative programs on TV and try to apply some of the techniques to testing.

Here is how to practice being a software investigator: Look at simple systems around you, pinpoint their weaknesses and imagine how they might fail. Apply that to software testing. Bounce the ideas off of colleagues, and take their criticism of the ideas seriously. Be self-critical – are you finding important problems? Is your feedback helping the team prevent defects? Do the problems you log get fixed? If not, why? Also, work on your bug and status reports. Ask for help from programmers on how to write better reports, and ask them what kinds of bugs they appreciate you finding. Work with them to learn more diagnostic and investigative reporting skills.

I thank Jonathan for taking time to answer these questions and for sharing his thoughts and insights with my readers and me. It was great to know him in a better way via these questions. I hope that you (my readers) enjoyed this interview as much as I did. Let me know how you felt about the interview, so that I will plan more such interviews on Software Testing Zone in future.

Happy Testing…

Interviewing a Testing Expert - Jonathan Kohl from Kohl Concepts [Part-1]

I am back with the second installment of the “Interviewing a Testing Expert” series. This time I am going to present an interview with Jonathan Kohl, co-founder of Kohl Concepts, who was kind enough to spare me some time for this interview. Based in Calgary, Alberta, Canada, Jonathan is a software testing consultant, author, and speaker in the software industry. Jonathan writes about and speaks on software testing. He draws on technical, philosophical and business concepts in his work. Jonathan's pragmatic approach focuses on getting down to the nuts and bolts of problem solving, encouraging organization-wide collaboration. Since this interview grew bit lengthier, for the sake of easy readability I have decided to present it in 2 parts. So, here is what Jonathan has to say:

Debasis: What led you to become a software tester? And what was the topmost reason that attracted you to the field of testing?

Jonathan: I started in testing when a small software company offered me an intern position in a QA department. They tended to start people with no professional software experience in QA. That way you could get an overall view of the organization, products, tools and methodologies and learn the basics. If you wanted to move on to programming, support, marketing, etc. then you had a good basic understanding from your QA experience. I enjoyed testing because I saw it as "philosophy of software". I had taken a lot of philosophy and logic courses as an undergrad, and I enjoyed the investigative, experimental approach of problem solving in testing, making a good case for bug fixes in bug reports, and having to make my test strategy, approach and decisions defensible to the rest of the team.

I later moved on to technical writing and then became a full time programmer, but I missed the variation of skills and activities in testing, so I went back into a testing role. My topmost reason for staying is the software investigation angle of my work. It’s challenging and fun to help track down and solve important problems that help teams succeed. I am still attracted to the variety of skills I use: investigative, analysis, observation, programming, writing, psychology… the list goes on. I can make testing into whatever I need to help discover and report important information that stakeholders need to make decisions.

Debasis: Did you try testing anything other than software before diving into software testing?

Jonathan: I grew up in a family where critical thinking and coming up for different explanations of why things are the way they are were rewarded. My father was a teacher, and is a brilliant critical thinker. He is able to quickly work through a massive amount of data, synthesize it, come to conclusions and make his thinking process defensible. We have debated many issues over the years, and it took me a long time to be able to spar with him. That background was really valuable as a tester, because he taught me to look for things other people might miss, would point out inconsistencies or flaws in my arguments, and he also taught me a lot about strategic thinking. From that, I went on to study a lot of logic and philosophy in university, and the critical thinking and argumentation analysis was vital to transitioning to thinking about software systems from a testing perspective.

Other than that, I used to take apart everything I owned and tried to put it back together. I needed to know why things worked the way they did so I could understand them. As a software tester, I decompose or reverse engineer products all the time so that I can find areas of potential weakness and I exploit those with tests.

Also, as an undergrad, I took part in experiment design (qualitative research), as well as a participant (helping with quantitative research.) In Biology, my experiments always went horribly awry and I frequently skewed the results in a sample set, much to the chagrin of the grad student who was overseeing us. They didn't like my results because they didn't map to their expectations (a problem with the hypothetical deductive model that scientific research hinges on) so they discouraged me from doing any more work for them. In testing, having experiments go horribly awry leads to fabulous information that stakeholders value, so I get paid to have failed experiments and my audience likes that information. I suppose my failed Biology experiments contributed to my career in testing.

Debasis: Tell me 5 unknown/least-known facts about you.

Jonathan: 1. I own, drive and work on a 1955 Pontiac custom. It is done in the style that car customizers and hot rodders used in the late '50s and early '60s. I belong to a car club with a local group of car and motorcycle builders who all drive and work on hot rods or customs. Some are original hot rodders from the '50s, right up to me, the youngest and least experienced.

2. I was involved heavily in music for most of my life, until quite recently. From the age of 4 when my parents put me on stage to sing a solo, to saxophone in junior high band, singing classical music in a high school chamber choir that traveled around to interesting places, playing guitar or bass and singing in alternative bands through university, to dabbling in my spare time after graduating. Music has always been a big part of my life and I have learned that a lot of that experience has shaped my approach to software projects.

3. I have an entrepreneurial streak. As a kid, I started money-making ventures from roadside lemon-aid stands to a tiny museum. None of them made much money, but they were a lot of fun. As I got older, I worked in sales part-time. I sold anything from shoes, to clothes, to house wares, and later, I even sold web-based software to help pay for university. I also helped start a software company in university, and now as an independent consultant I work with startup companies and advise founders and executives and help them with their business plans, strategic vision, product or service vision, and as they develop pitches to investors. It takes an enormous amount of effort to start something from nothing, and I have huge respect for software entrepreneurs. It irritates me when we techies make fun of management, or sale. They are hard jobs to do well and can come with enormous pressure and responsibility.

4. I'm a trained manager with experience from corporate training and undergrad work while in university. I started management training on the job as an assistant at the age of 18, during a recession a few years ago. Many employees reporting to me were former entrepreneurs or former senior managers from firms that had laid them off, and they were working part-time to help make ends meet. To lead people who were old enough to be my parents took a special combination of skill and diplomacy. I had to earn their respect by being willing to do any job I asked them to do, to work hard, and to not be a dictator. I learned something important from them - they were willing to work and do whatever it took to do right by their ethics and their families. They weren't distracted by titles or status; they just did whatever needed to be done. They were successful prior to the last big recession, survived it, and were even more successful afterwards. Those early lessons had a huge impact on my career.

5. I've been writing for most of my life, and I love it. Now I write on technical topics, but I plan on branching out into fiction eventually.

Debasis: What was the hardest challenge that you found in your career as a tester?

Jonathan: Overcoming narrow thinking about testing. Early in my career, I worked for small startups, and they demanded that all employees provide value. They didn't really care how we did that as testers, as long as we provided them with the information they needed. Some of them were using iterative, incremental lifecycles and practices back in the '90s, and we had to be creative and solve problems differently than traditional testing texts recommended. As I moved on, I found that some testing groups were less concerned about results than they were in some adhering to some process ideal. Back then, people were more concerned about heavyweight processes. More recently, being Agile-compliant is often more important than providing good results. (By the way, I’ve been involved on a lot of Agile projects over the past several years, some successful, others not.)

I don't understand why we have so many extreme views in software that have so much emotion and blind faith wrapped up in them. I value sustainable results, and so do most stakeholders I work with. On most real-world projects, there is a mix of practices: Agile and traditional, exploratory and scripted testing, manual and automation. It bothers me that we have to be one thing or another, or be at one extreme and denigrate other ideas, tools and practices. Why not broaden thinking and try for a mix of tools and practices in testing? Thankfully, there has been a softening of views and more people are open to trying whatever it is that can help them create value, rather than being pure adherents to some methodology or tool ideal.

That has been interesting and rewarding to see. Even five years ago exploratory testing was often misunderstood and discouraged. Now, people are interested in trying out practices to learn how to do it better and add it to their process and tool mix. The same thing goes for open source testing tools, and different software development methodologies. The fusion of ideas, and the mashups of tools, processes, practices and methodologies different teams use to create value over time is fun to watch.

Debasis: Tell me about the most satisfying moment in your testing career.

Jonathan: When I first realized my work mapped directly to our company being able to provide more value to our customers and end users. Now, I get satisfaction when I help track down high impact intermittent bugs that not only help the team create value for their users, but also it takes a weight of pressure off the team. As a trainer of software testers, I get enormous satisfaction when I see a tester learn and grow as I help them develop their skills, and learn how to learn the craft of software testing. It's fun to have a manager ask me what on earth I did to help this junior tester rocket to a bug finding and bug-preventing star in such a short time. Of course, I did very little but encourage an interested tester figure out how to tap into their own skill set and potential.

I hope all of you enjoyed the interview as much as I did. I will be posting the final part of this interview soon. So stay tuned.

Happy Testing...