I won't name my ISP, but this story unfortunately can apply to most of them.
Last Saturday, my Internet connection was working just fine. I went out for a few hours, came back, and went to look something up. Pages were taking a minute or two to load. I ran a speed test. It took awhile to get loaded, but once it loaded, it was fine. Speeds were good.
My first thought was DNS. So, I switched DNS servers. The problem was still there. I restarted my router. It was still there. So, I thought of latency and started doing some pings. That's when the problem became clear. Pings were low, but I was losing about 1 out of every 10 pings. I tried a few servers, including my ISP's page. They all showed packet loss. A quick test at http://pingtest.net confirmed a packet loss of 8%.
I started thinking my router might be having problems. So, I booted off a Linux live CD, cloned my MAC address to that of the router, and plugged my laptop directly into the ONT in my garage. I got an IP address and repeated the ping test. It showed 10% loss. I pinged my gateway, and it showed loss as well.
So, at this point, I figured I had done my homework. The problem was either my ONT, the ISP's gateway, or something in between. I tried resetting my ONT, but it didn't help. I decided it was time to call tech support. The problem was, I didn't have the number. Luckily, my connection was still somewhat working, and I managed to find it, after about 30 minutes of waiting for pages to load.
As soon as the support agent answered, I had a feeling I was doomed. It was Saturday night, and the call was obviously outsourced. Still, I had some faith in the system. The call went something like this:
Agent: How can I help you?
Me: I'm experiencing 10% packet loss between the ONT and the gateway. This is causing web pages to take several minutes to load, if they even load at all. I've already ruled out my router by connecting my laptop directly up to the ONT. I reset the ONT, but it didn't help. I'm going to need somebody to come out here and fix it.
Agent: I'm sorry you are having problems. I have a few things you can try.
At this point, I know I'm going to have to play along for a few minutes. After all, he doesn't know that I actually know what I'm talking about.
Agent: Could you try rebooting your router. Go to the router, disconnect power for 30 seconds, then reconnect it.
I know this won't work, but I play along anyway. "Ok. I'm doing that." My router was upstairs, and I was downstairs, so I figured about 90 seconds was enough time to have waited to tell him it was done "rebooting."
Agent: Can you try going to [link for the ISP's speed test]?
Me: Sure.
I waited for the page to load. While I'm waiting, I ran a few pings. Packet loss is up to 30%.
Me: It's still a not loading. It's a blank page. This isn't a speed problem, though. I'm seeing packet loss, and it's getting worse.
Agent: How long has it been like this?
Me: It was fine this afternoon. Three hours later, I saw some problems with 10% loss. I called you, and in the last 15 minutes, it went up to 30%.
Agent: Okay. Let me check with a network technician.
After 5 minutes on hold:
Agent: He wants you to connect your router back to the ONT, not your laptop.
Me: I already did that.
Agent: Okay. Try resetting your router. Power it off, hold down the reset button, and bring it back up.
I have plenty of port forwarding rules, firewall rules, custom DNS entries, and static DHCP entries. I'm not about to wipe all those out for something that's not even the router's problem. So:
Me: Okay, but I need to back up my settings first.
Agent: Oh, that would be a good idea.
Again, I figured a couple of minutes is enough time to have "reset" the router.
Me: Okay, it's reset. I'm trying the speed test page, and it's not loading at all now.
Agent: Let me check with the technician.
While on hold, I connected the laptop back up to see how much worse it had gotten. I didn't get an IP address. I rebooted my router. It couldn't get an IP address. I've reach 100% packet loss.
Agent: He's going to reprogram your ONT.
This sounded promising. I was wrong, but I did get my hopes up for a little bit.
Me: Okay. By the way, it's gotten worse. My router won't even get an IP address now. I'm seeing 100% loss and have a completely dead connection.
Agent: Okay. Please hold.
15 minutes later, he came back.
Agent: Okay. He wants you to reset the ONT. Unplug the power to the ONT for 1 minute.
Me: Sure. By the way, can you just connect me to the network technician so I don't have to keep waiting on hold.
Agent: Sorry, we don't have a way to do that.
Me (rolling my eyes at this point and getting frustrated): I'm resetting it. Do I have to disconnect the battery?
I knew the answer, but just had to ask.
Agent: No. Just the AC
BUZZ! Sorry, wrong answer. Unplugging the AC from a device with a battery backup inside it isn't going to do much.
Me: I unplugged it. The lights are still on.
Agent: One second.
On hold again.
Agent: Okay. Sorry. Unplug the battery too.
Me: Okay.
I unplugged the battery, and the ONT went dead. I waited 60 seconds, and plugged everything back in.
Me: Okay. It's coming up. The fail light is flashing, just like it should while it tries to connect....okay....it's still flashing. It should've connected by now...still nothing.
Agent: Okay. Please hold.
......
Agent: Hmm...can you try going to the speed test site?
Me: The ONT's not connecting. I won't be able to.
Agent: Can you try restarting the router?
Me: The ONT's not connecting. The router has nothing to do with it. My router's not even getting a link now. My connection is dead.
Agent: Please wait....
About 15 minutes later
Agent: The technician can't even reach your ONT. We're going to have to send somebody out.
FINALLY!
Me: Thank you.
Agent: Unfortunately, our ticketing system is down, so I can't create the ticket. Somebody will call you.
Me: When will that be?
Agent: I don't know. Our system is down.
Me: Well, it's Saturday night. If your system comes back up as soon as you hang up, will somebody be calling me tonight, or will it have to wait until Monday at the earliest?
Agent: Our system is down.
Me: Yes, I know, but will I get a call, say, tomorrow, if the system comes back up, or do you only call back during regular business hours?
Agent: During regular business hours.
Me: Okay. Thank you.
Agent: Is there anything else I can do for you?
I look at my phone. It's been an hour and 43 minutes since I called.
Me: No. It's been almost two hours as it is, so I don't think there's much else you can do.
Agent: Thank your for choosing [ISP]
So, at that point, after spending over 1 1/2 hours on the phone, it looked like my connection was going to be down awhile. They were going to send somebody out, which was what I asked for 2 minutes into the call.
Sunday morning, I turned on my cell phone. I had a voice mail from the ISP. They were sending somebody out at 2:00 and I needed to call back if that wouldn't work. Had I known they might call, I would have left the phone on. But, at least it was going to get fixed.
I went to the store. At 1:30, I got a call from the dispatcher saying that the technician was on his way. I started heading home. At 1:35, I got a call from the technician saying he was at my house. I told him I was 5 minutes away.
I rushed home, let him into the garage, and explained what I had been seeing. At this point, the ONT was dead. The first thing he did was replace it. It took him five minutes. I came back out in the garage and saw him staring at the ONT. The fail light was flashing. Apparently, the ONT wasn't the problem.
He stood there for a good five minutes, just staring at the pretty blinking light. Then, he started to talk to somebody on his phone. After he hung up, he said he was going to try re-authorizing the ONT and went back to his truck. He came back and said his connection from the truck was down and would have to call it in for somebody to do manually.
Irony.
A few minutes later, he rang the bell. He said the problem wasn't the ONT or in the box in the front yard. It was possibly in the hub, so he was leaving, but wanted me to know he was still working on it.
He finally came back and said it was fixed. It took an hour, but in this case, it was understandable. It was a problem in the central office. Normally, a CO problem would impact dozens of people, and they'd know the problem was there because of the high volume of calls. In this case, it was only me, so it didn't look like a CO problem. I guess I was just lucky.
Wednesday, December 16, 2009
Wednesday, April 22, 2009
Data-Driven Testing
A few years ago, I undertook a project to make automating printer drivers easier for QA. The idea was to use INI files to define the tests so that people who don't know how to develop can automate new tests. I wound up using a set of INI files to define applications, driver UIs, and test cases.
I didn't realize it at the time, but I was essentially making a data-driven test. Test cases were added simply by editing a simple file. It even went beyond the traditional data-driven test setup by supporting black box monkey testing, default values for most functions (such as falling back to CTRL + O to bring up the open dialog in an application), and remote monitoring and control of the test through the network.
Data-driven testing makes life easier for QA as well as the automation developer. Instead of having to add lines of code, anybody could just edit a file to add test cases. It's much more complicated to create data-driven automation, and it isn't appropriate for every test plan, but when it does work, it works well.
I didn't realize it at the time, but I was essentially making a data-driven test. Test cases were added simply by editing a simple file. It even went beyond the traditional data-driven test setup by supporting black box monkey testing, default values for most functions (such as falling back to CTRL + O to bring up the open dialog in an application), and remote monitoring and control of the test through the network.
Data-driven testing makes life easier for QA as well as the automation developer. Instead of having to add lines of code, anybody could just edit a file to add test cases. It's much more complicated to create data-driven automation, and it isn't appropriate for every test plan, but when it does work, it works well.
Thursday, April 16, 2009
Yet another "thank you" email
It's bad enough to get an email saying the interview didn't work out. It's worse when it's a form letter. I interviewed at a company in San Diego. I went through two phone interviews before being asked to come in, using up plenty of my mobile phone minutes. I drove an hour to the office, went through a 3 hour interview with 5 different people, was told they'd get back to me within a few days, sat in traffic on the way home for 90 minutes, sent some personal thank you emails, and waited.
A week later (this Monday), I got an email from the person in HR I had been talking to. She said that the team had finished the interviews and would be in touch no later than Friday with a decision. I was a little ticked because she sent it to all the candidates, but put our email addresses in the To field instead of using BCC, so everybody was able to see everybody else's email addresses and names. So much for privacy.
Since I had the email addresses of each of the other candidates, I could have done searches and got background information on each of them, then used this information to my advantage in a follow-up email. I didn't (though I wonder if any of the other candidates did). To make matters worse, it was HR that sent the email, and HR should have known better than to disclose the names and email addresses of all the candidates.
Anyway, today, 10 days after the interview, I got the following email:
However, if somebody invests their time, gas, and energy into showing up in person for an interview, then that person follows-up with a personal thank you after the interview, the least you could do is sign the rejection email with your own name instead of "HR/Recruitment Team."
Maybe it's better I didn't get the job. I'd hate to show up to work one day and find an email from "HR/Recruitment Team" telling me I've been laid off. I can just see it now:
A week later (this Monday), I got an email from the person in HR I had been talking to. She said that the team had finished the interviews and would be in touch no later than Friday with a decision. I was a little ticked because she sent it to all the candidates, but put our email addresses in the To field instead of using BCC, so everybody was able to see everybody else's email addresses and names. So much for privacy.
Since I had the email addresses of each of the other candidates, I could have done searches and got background information on each of them, then used this information to my advantage in a follow-up email. I didn't (though I wonder if any of the other candidates did). To make matters worse, it was HR that sent the email, and HR should have known better than to disclose the names and email addresses of all the candidates.
Anyway, today, 10 days after the interview, I got the following email:
Is it too much to ask for a personal email signed by a person? There were only four of us (which is information I shouldn't have known). It would have only taken a few minutes to send each of us an email. Even an email that was sent to all of us would've been better than just logging into Taleo and clicking a button to send a form rejection letter to all candidates. Form responses are great when you haven't even gotten a phone call about your application. In fact, I'm glad when I get those, since at least I know they got my application.
Subject: Thank you for your interest in [Name of company I applied to]
Dear Troy Hoffman,
Thank you for submitting your resume tofor consideration.
We are fortunate to have many qualified candidates apply to each of our positions. We have reviewed the qualifications of each candidate and after careful consideration, we have determined that the credentials of other candidates may better fit our needs at this time.
Please accept our best wishes and thank your for your interest in our company.
Sincerely,
The [Name of the parent company] HR/Recruitment Team
-------------------------------
Powered by Taleo
www.taleo.com
However, if somebody invests their time, gas, and energy into showing up in person for an interview, then that person follows-up with a personal thank you after the interview, the least you could do is sign the rejection email with your own name instead of "HR/Recruitment Team."
Maybe it's better I didn't get the job. I'd hate to show up to work one day and find an email from "HR/Recruitment Team" telling me I've been laid off. I can just see it now:
Subject: Thank you for your service at [Company Name]
Dear Troy Hoffman,
As you are aware, many companies have been forced to cut expenses due to the current economic situation. At [company name], we have been fortunate to have avoided workforce reduction.
However, due to a slowdown in sales, we have been faced with the difficult decision of how to reduce expenses. Regrettably, we have been forced to reduce our working staff. This decision was not made lightly and in no way reflects on the performance of those who will no longer be with us.
Since you have received this email, you have been one of those whose position has been eliminated. Please pack up your personal belongings and wait for security to escort you out of the building. Your final paycheck will be mailed to you. Regrettably, there will be no severance package.Please accept our best wishes and thank your for your many years of dedicated service to our company. We sincerely hope you are able to support your family. Security at [Company Name] will, of course, be increased in the months ahead, so please do not visit without an appointment and approval.
Sincerely,
The [Name of the parent company] HR/Recruitment Team
-------------------------------
Powered by Taleo
www.taleo.com
Wednesday, April 8, 2009
Exploratory Testing
If you would have asked me about exploratory testing a year ago, I probably would have said, "What's that?" I had never heard of it.
It turns out, I've been doing exploratory testing for years. I just never heard it called that. That's one problem with figuring out most of SQA on my own. I never learned some of the lingo.
Exploratory testing is pretty much what the name implies it is. It's manually testing a product without following a formal plan, or deviating from the test plan you were following. Simply put, it's what I've always called "strategically poking it with a stick to see where it bleeds."
Although randomly clicking and typing is a form of exploratory testing, that should be left for the monkeys. If you want to perform good exploratory testing, you need to think more about where that stick should be poking the program. Look at the release notes to see what changed. If you can, read the source code to see the code that changed. If you've done development in the past, ask yourself, "If I were coding this, what would I mess up?"
The more you know about your target, the more likely you'll be poking at weak spots.
It turns out, I've been doing exploratory testing for years. I just never heard it called that. That's one problem with figuring out most of SQA on my own. I never learned some of the lingo.
Exploratory testing is pretty much what the name implies it is. It's manually testing a product without following a formal plan, or deviating from the test plan you were following. Simply put, it's what I've always called "strategically poking it with a stick to see where it bleeds."
Although randomly clicking and typing is a form of exploratory testing, that should be left for the monkeys. If you want to perform good exploratory testing, you need to think more about where that stick should be poking the program. Look at the release notes to see what changed. If you can, read the source code to see the code that changed. If you've done development in the past, ask yourself, "If I were coding this, what would I mess up?"
The more you know about your target, the more likely you'll be poking at weak spots.
Tuesday, April 7, 2009
Time Flies When You're Out of Work
I can't believe it's been over a month since I last posted here. I've been pretty busy lately, and since nobody's really reading this thing yet, I haven't had much motivation to post. Looking for a job is hard work. I spend all day scouring the job boards, looking for local software companies that might have unadvertised positions, and looking in the papers. Then, when I finally do find something that looks good, it takes at least 1/2 hour, often more, to tweak my resume for the position and write the cover letter.
That being said, things are looking better. I had an interview yesterday that I think went well. I had a phone interview, and this was the in-person interview. When the manager said he was going to give me a test, then asked how I would test a phone. I was a bit surprised, since my resume shows that I've spent the last two years working with phones. Hopefully, it went as well as I thought. If not, I have another interview this week for another QA job. Two interviews in one week. I think it's a record.
On the subject, I've learned quite a bit about the job market and have a few tips.
Simplify Your Resume
Keep your resume short and simple. Two years ago, I had trimmed my resume down to two pages. The format I used worked great back then. These days, though, even two pages is too much. There's a lot more competition for jobs right now. SQA has been hit particularly hard since many companies think they can just have developers test their own code (that's a mistake, and probably could be a blog entry on its own). Hiring managers are looking at dozens of resumes. If it takes longer than a couple of minutes to read, it won't be read.
A person I used to work with is working for a fairly major software company now. They had a QA position open, so I sent her my two page resume. She gave me some tips to trim it down and make it much easier to read. Between her tips and some other tips I've picked up over the past couple of months, here's what I suggest.
Networking: Packet capture and analysis, troubleshooting multiple OSI layers
Development: C, C++, C#, QTP, Java, HTML, XML
That's not to say you shouldn't have a resume that lists all your skills. If you're posting your resume on a job board, you'll want the skills section to include everything you can do. In fact, if you've done a little bit of everything, having 1/2 a page or even a full page of lists of skills isn't bad in a resume you're posting on a job board. The reason is that recruiters and hiring managers often do a keyword search on the major job boards to find somebody who has specific experience. Listing everything in this resume will make it more likely to get picked up.
Customize Your Resumes Before Sending Them Out
I spend at least 30 minutes applying for a job. I look at the job description and requirements carefully, make sure I think I'm qualified for it, then customize a resume for that particular job. I'll remove accomplishments and skills that I don't think are important for that job and include the things I've done that are. In fact, even though I've sent out scores of resumes, no two have been exactly the same.
If a hiring manager sees a bunch of things on your resume that have nothing to do with the job you're applying for, they think you didn't read the description very carefully. With a job in SQA, attention to detail is important, so this would make a horrible first impression.
Write the Cover Letter from Scratch
Make sure you always include a cover letter. The only exception is when you have to use an application wizard online, and it doesn't let you add one.
Second, always start from scratch when writing the cover letter. It may take more time, but it will keep it from looking like a form letter.
Pay Attention to the Job Requirements
If the job says certification or a degree is required, make sure you acknowledge it. In your cover letter, bring attention to your degree. Mention the school. If you don't completely meet the requirements, but have experience that you think makes up for it, bring it up by saying something like, "Although I do not currently hold CSQA certification, I have 14 years of professional experience testing software." I've gotten interviews for jobs that require certification I didn't have because I was up-front about lacking it, but made up for it with experience. Again, this is an attention to detail. If you just apply without mentioning it, the person looking at your resume might think you weren't paying attention.
Track Your Applications
It's important to keep track of where you've applied. For one thing, if you've applied to Company X five times, and haven't gotten an interview once, when you see another position for Company X, you know you probably should either ignore it or, if you think you might have a chance the sixth time, you won't spend too much time on it and won't get your hopes up.
This is especially important if you deal with recruiters. When they call for a job opportunity and mention the company, you can check your list to see if you've already applied. If you have, let them know. Most of the time, they won't be able to represent you for the job, and you'll save everybody some time this way.
You don't need to do anything fancy. I just use a TXT file with a running list of where I've applied. I can then do a search for a company name and quickly see if it's a position I've already tried for.
Call Up Your Old Friends
Get on networking sites like Linked In. Look up your old friends and coworkers. Network, network, then network again. If you know somebody at a place you're applying to, let them know about the position. You might get some inside information about the job that could prove useful in an interview. More importantly, they might be able to hand deliver your resume to the hiring manager. It won't guarantee a job, but it could mean that your resume will at least be read.
Stay on Top of the Job Postings
Pay attention to multiple job boards. Don't forget places like Craigslist (but be careful of scams). Do a web search for software companies in major cities near you (or where you'd want to commute), then find their career pages and look for jobs that aren't posted anywhere else. When you do see a job, apply right away. The sooner you get your resume in, the more likely it will be seen. If you wait a day or two, the person reading the resumes will already have seen several and will be less likely to even look at your resume than if you were one of the first few in.
Don't Get Frustrated
If you are looking for a job, I wish you the best of luck. I know how frustrating it is to send out 20 resumes and not get a single phone call. I wish I had some of this advice two months ago; I probably would be working now. Since I've changed my resume format, I've been getting more phone calls and email responses. It's a tough job market, but there are still some jobs out there. You just have to be patient, professional, and quick.
That being said, things are looking better. I had an interview yesterday that I think went well. I had a phone interview, and this was the in-person interview. When the manager said he was going to give me a test, then asked how I would test a phone. I was a bit surprised, since my resume shows that I've spent the last two years working with phones. Hopefully, it went as well as I thought. If not, I have another interview this week for another QA job. Two interviews in one week. I think it's a record.
On the subject, I've learned quite a bit about the job market and have a few tips.
Simplify Your Resume
Keep your resume short and simple. Two years ago, I had trimmed my resume down to two pages. The format I used worked great back then. These days, though, even two pages is too much. There's a lot more competition for jobs right now. SQA has been hit particularly hard since many companies think they can just have developers test their own code (that's a mistake, and probably could be a blog entry on its own). Hiring managers are looking at dozens of resumes. If it takes longer than a couple of minutes to read, it won't be read.
A person I used to work with is working for a fairly major software company now. They had a QA position open, so I sent her my two page resume. She gave me some tips to trim it down and make it much easier to read. Between her tips and some other tips I've picked up over the past couple of months, here's what I suggest.
- Start off with your name and contact info at the top.
- Add a short paragraph (2-3 sentences) summing yourself up in a "profile" section.
- For each of your jobs, list no more than 6 bullet points.
- If you have a long job history with lots of different jobs, don't list them all. List your last 2 or three jobs, going back up to 10 years or so. Then, have a final "job" called "Earlier Career Summary." If they want more details, it will come up in the interview.
- Only list accomplishments in your job experience section. Do not list responsibilities.
- After your jobs, have a small section listing your skills. Break them down into categories. For example:
Networking: Packet capture and analysis, troubleshooting multiple OSI layers
Development: C, C++, C#, QTP, Java, HTML, XML
- Use a decent sized font. 11-12 points should be good.
- Make sure you have decent spacing between lines and sections. Keep it easy to read.
That's not to say you shouldn't have a resume that lists all your skills. If you're posting your resume on a job board, you'll want the skills section to include everything you can do. In fact, if you've done a little bit of everything, having 1/2 a page or even a full page of lists of skills isn't bad in a resume you're posting on a job board. The reason is that recruiters and hiring managers often do a keyword search on the major job boards to find somebody who has specific experience. Listing everything in this resume will make it more likely to get picked up.
Customize Your Resumes Before Sending Them Out
I spend at least 30 minutes applying for a job. I look at the job description and requirements carefully, make sure I think I'm qualified for it, then customize a resume for that particular job. I'll remove accomplishments and skills that I don't think are important for that job and include the things I've done that are. In fact, even though I've sent out scores of resumes, no two have been exactly the same.
If a hiring manager sees a bunch of things on your resume that have nothing to do with the job you're applying for, they think you didn't read the description very carefully. With a job in SQA, attention to detail is important, so this would make a horrible first impression.
Write the Cover Letter from Scratch
Make sure you always include a cover letter. The only exception is when you have to use an application wizard online, and it doesn't let you add one.
Second, always start from scratch when writing the cover letter. It may take more time, but it will keep it from looking like a form letter.
Pay Attention to the Job Requirements
If the job says certification or a degree is required, make sure you acknowledge it. In your cover letter, bring attention to your degree. Mention the school. If you don't completely meet the requirements, but have experience that you think makes up for it, bring it up by saying something like, "Although I do not currently hold CSQA certification, I have 14 years of professional experience testing software." I've gotten interviews for jobs that require certification I didn't have because I was up-front about lacking it, but made up for it with experience. Again, this is an attention to detail. If you just apply without mentioning it, the person looking at your resume might think you weren't paying attention.
Track Your Applications
It's important to keep track of where you've applied. For one thing, if you've applied to Company X five times, and haven't gotten an interview once, when you see another position for Company X, you know you probably should either ignore it or, if you think you might have a chance the sixth time, you won't spend too much time on it and won't get your hopes up.
This is especially important if you deal with recruiters. When they call for a job opportunity and mention the company, you can check your list to see if you've already applied. If you have, let them know. Most of the time, they won't be able to represent you for the job, and you'll save everybody some time this way.
You don't need to do anything fancy. I just use a TXT file with a running list of where I've applied. I can then do a search for a company name and quickly see if it's a position I've already tried for.
Call Up Your Old Friends
Get on networking sites like Linked In. Look up your old friends and coworkers. Network, network, then network again. If you know somebody at a place you're applying to, let them know about the position. You might get some inside information about the job that could prove useful in an interview. More importantly, they might be able to hand deliver your resume to the hiring manager. It won't guarantee a job, but it could mean that your resume will at least be read.
Stay on Top of the Job Postings
Pay attention to multiple job boards. Don't forget places like Craigslist (but be careful of scams). Do a web search for software companies in major cities near you (or where you'd want to commute), then find their career pages and look for jobs that aren't posted anywhere else. When you do see a job, apply right away. The sooner you get your resume in, the more likely it will be seen. If you wait a day or two, the person reading the resumes will already have seen several and will be less likely to even look at your resume than if you were one of the first few in.
Don't Get Frustrated
If you are looking for a job, I wish you the best of luck. I know how frustrating it is to send out 20 resumes and not get a single phone call. I wish I had some of this advice two months ago; I probably would be working now. Since I've changed my resume format, I've been getting more phone calls and email responses. It's a tough job market, but there are still some jobs out there. You just have to be patient, professional, and quick.
Saturday, February 21, 2009
SQA Term of the Day - Binary Search
These probably aren't going to come every day, but here's the first. Well, technically, it's the third, since I already posted three buzzwords earlier. In any case, a binary search is a useful method to find the source of a problem. It's similar to a traditional binary search algorithm, but is used in QA and other technical troubleshooting scenarios.
Basically, you look at the variables that could be causing or contributing to the problem you're experiencing, then eliminate or change 1/2 of them. If the problem goes away, you know it was one of the variables you changed. If not, it wasn't. You keep repeating until you narrow it down to the actual cause of the problem.
A realistic use would be discovering a new bug and trying to figure out when it was introduced. If you have a build every week and have 20 builds, you'd go back 10 weeks. If the problem is gone, you go back 5 weeks. If it's not, you go back 15. Using this method, you'd quickly find the build that introduced the problem.
Another use would be if you had a document that was causing a printer driver crash. Eliminate 1/2 the objects on the page and print. If it crashes, one of those objects is causing the crash. If it doesn't crash, it was one of the objects you removed. You'd then have a document with 1/2 the objects that still crashes. Then, take half of those objects away and print. It won't take long until you have a document with a single object on it that causes a crash. Add that document (and the original) to your bug report, and you'll keep your developers happy.
Basically, you look at the variables that could be causing or contributing to the problem you're experiencing, then eliminate or change 1/2 of them. If the problem goes away, you know it was one of the variables you changed. If not, it wasn't. You keep repeating until you narrow it down to the actual cause of the problem.
A realistic use would be discovering a new bug and trying to figure out when it was introduced. If you have a build every week and have 20 builds, you'd go back 10 weeks. If the problem is gone, you go back 5 weeks. If it's not, you go back 15. Using this method, you'd quickly find the build that introduced the problem.
Another use would be if you had a document that was causing a printer driver crash. Eliminate 1/2 the objects on the page and print. If it crashes, one of those objects is causing the crash. If it doesn't crash, it was one of the objects you removed. You'd then have a document with 1/2 the objects that still crashes. Then, take half of those objects away and print. It won't take long until you have a document with a single object on it that causes a crash. Add that document (and the original) to your bug report, and you'll keep your developers happy.
Firefox using tons of memory
I have to admit, I love tabbed browsing. If I have 7 tabs open, that's low for me. I usually have at least 15 tabs open, sometimes up to 30. When I see an interesting link, I'll open it in a new tab to look at later. That way, I don't interrupt my train of thought.
The problem is that Firefox was using tons of memory. I was usually seeing memory usage near 600 MiB. This may not seem like much, but when you only have 1 GiB of RAM and want to use a Solaris VM at the same time, it tends to make your system come to a crawl because of virtual memory swapping.
So, I did a little research and found a fix. Now, the VM Size in Task Manager for Firefox is around 140 MiB, which is much more reasonable for me. The problem is that, by default, Firefox keeps a history of pre-rendered pages for each tab. With 1 GiB of physical memory, it defaults to 8 pages per tab. Multiply that by 15 tabs, and it's keeping 120 pre-rendered pages in RAM. Add lots of graphics to those pages, and it starts adding up very quickly.
Fortunately, Firefox is very customizable, so it's easy to fix. This page explains the details (that's a great site for Firefox tweaks by the way), but the short solution is this:
1) Type about:config in the Firefox address bar and hit enter.
2) Find browser.sessionhistory.max_total_viewers and change the value to 0.
3) Restart Firefox.
Before doing this, check the memory usage of your system and Firefox. Bring up Task Manager. If your Commit Charge is less than your physical RAM, it won't improve system performance. In fact, it would make going back and forth in the browser history (using the arrows at the top) a little slower in Firefox, since it will have to render each page every time you go back.
However, if your commit charge is greater than your physical RAM, it might help. Sure, you'll take a performance hit when it has to render every page again, but the performance hit caused by using virutal memory is much worse. To confirm that Firefox is eating up your RAM, go to View/Select Columns in Task Manager and enable Virtual Memory Size. Then, see how much Vitual Memory Firefox is using. If it's more than 150 MiB or so, it's worth trying this tweak.
I still need more memory, but at least I can leave my browser open when I do other things now.
The problem is that Firefox was using tons of memory. I was usually seeing memory usage near 600 MiB. This may not seem like much, but when you only have 1 GiB of RAM and want to use a Solaris VM at the same time, it tends to make your system come to a crawl because of virtual memory swapping.
So, I did a little research and found a fix. Now, the VM Size in Task Manager for Firefox is around 140 MiB, which is much more reasonable for me. The problem is that, by default, Firefox keeps a history of pre-rendered pages for each tab. With 1 GiB of physical memory, it defaults to 8 pages per tab. Multiply that by 15 tabs, and it's keeping 120 pre-rendered pages in RAM. Add lots of graphics to those pages, and it starts adding up very quickly.
Fortunately, Firefox is very customizable, so it's easy to fix. This page explains the details (that's a great site for Firefox tweaks by the way), but the short solution is this:
1) Type about:config in the Firefox address bar and hit enter.
2) Find browser.sessionhistory.max_total_viewers and change the value to 0.
3) Restart Firefox.
Before doing this, check the memory usage of your system and Firefox. Bring up Task Manager. If your Commit Charge is less than your physical RAM, it won't improve system performance. In fact, it would make going back and forth in the browser history (using the arrows at the top) a little slower in Firefox, since it will have to render each page every time you go back.
However, if your commit charge is greater than your physical RAM, it might help. Sure, you'll take a performance hit when it has to render every page again, but the performance hit caused by using virutal memory is much worse. To confirm that Firefox is eating up your RAM, go to View/Select Columns in Task Manager and enable Virtual Memory Size. Then, see how much Vitual Memory Firefox is using. If it's more than 150 MiB or so, it's worth trying this tweak.
I still need more memory, but at least I can leave my browser open when I do other things now.
Friday, February 13, 2009
Buzzwords, Part I
I've heard many buzzwords in my years of SQA and have come to realize that, while the basic ideas behind these buzzwords make sense, there are too many people who misuse or overuse these terms. I'm only including a few for now.
Unit Testing - This term is misused far too often. I've heard many QA testers talk about running unit tests when they're really doing black box testing. True unit testing is done in a development environment, using development code.
Unit tests are important, but the problem is that developers often write their own unit tests. If they make assumptions in the code, such as the inputs being fed to a function, odds are they'll make the same assumptions when coding the unit tests.
Effective unit tests should be written by a second developer or, ideally, a SQA engineer with enough development knowledge to read the unit of code under test. Even if QA can't write the test, somebody from QA should be looking at the specs and suggesting what the unit test should feed to the code.
Inputs and Outputs - When interviewing interns straight out college, these words come up all the time. When asked what method of software testing they would use if employed, I constantly heard things like, "I'd feed it different inputs and compare the output to the expected result." It sounded like I was listening to a textbook. This type of testing is important, but the inputs and expected outputs come from the product specifications. These are the same specifications that the developers used to write the code in the first place. If the tester and the developer interpret the specifications the same way, there could be missed bugs.
Sometimes, you just need to poke it with a stick in random places to find the weak spots.
Black Box Monkey Testing - This is really underused, not overused, but I wanted to include it anyway. I first heard this term when reading Visual Test 6 Bible by Thomas R. Arnold II. The concept is based on the infinite monkey theorem. Using automation (preferably) or a person, you start randomly typing and clicking, looking for bugs. A few years ago, when creating automation to test printer drivers, I implemented monkey testing code. It would open the printer driver properties and start doing random things. It might randomly move the mouse around. It could click in random spots. Or, it could pick a random field on a random tab and set it to a random value. We actually found a few serious bugs this way that may never have been found otherwise. Well, they probably would have been found, but by the users, which is usually not who you want finding bugs.
Of course, you still need targeted testing, unless you have an infinite number of computers running an infinite number of test plans and an infinite number of people to go through the logs to find out what caused the BSoD. It's still a very valuable method of testing. If done properly, a monkey test would continue to work, even after your GUI is completely changed.
Unit Testing - This term is misused far too often. I've heard many QA testers talk about running unit tests when they're really doing black box testing. True unit testing is done in a development environment, using development code.
Unit tests are important, but the problem is that developers often write their own unit tests. If they make assumptions in the code, such as the inputs being fed to a function, odds are they'll make the same assumptions when coding the unit tests.
Effective unit tests should be written by a second developer or, ideally, a SQA engineer with enough development knowledge to read the unit of code under test. Even if QA can't write the test, somebody from QA should be looking at the specs and suggesting what the unit test should feed to the code.
Inputs and Outputs - When interviewing interns straight out college, these words come up all the time. When asked what method of software testing they would use if employed, I constantly heard things like, "I'd feed it different inputs and compare the output to the expected result." It sounded like I was listening to a textbook. This type of testing is important, but the inputs and expected outputs come from the product specifications. These are the same specifications that the developers used to write the code in the first place. If the tester and the developer interpret the specifications the same way, there could be missed bugs.
Sometimes, you just need to poke it with a stick in random places to find the weak spots.
Black Box Monkey Testing - This is really underused, not overused, but I wanted to include it anyway. I first heard this term when reading Visual Test 6 Bible by Thomas R. Arnold II. The concept is based on the infinite monkey theorem. Using automation (preferably) or a person, you start randomly typing and clicking, looking for bugs. A few years ago, when creating automation to test printer drivers, I implemented monkey testing code. It would open the printer driver properties and start doing random things. It might randomly move the mouse around. It could click in random spots. Or, it could pick a random field on a random tab and set it to a random value. We actually found a few serious bugs this way that may never have been found otherwise. Well, they probably would have been found, but by the users, which is usually not who you want finding bugs.
Of course, you still need targeted testing, unless you have an infinite number of computers running an infinite number of test plans and an infinite number of people to go through the logs to find out what caused the BSoD. It's still a very valuable method of testing. If done properly, a monkey test would continue to work, even after your GUI is completely changed.
Subscribe to:
Posts (Atom)
Posts
