Beware of Linkscanners (Link Scanners) -- the things that check search results for viruses and malware

Please keep this in mind: Most likely you're using these and don't entirely know it! Many anti-virus companies have these "link scanners" setup on your computer just by installing their anti-virus program. Others, such as siteadvisor (that was purchased by McAfee) are stand-alone applications. I haven't tested them all, nor do I care to do so. But some of these are very bad things!

Why? Well there's two main reasons, and they all fall into one of these categories.

Category 1 -- Sends the link to a central DB for checking.

Why is this bad? Well let's say a site is classified as "Good", then it gets attacked and a virus gets uploaded the second after the test. Guess what? The site's no longer safe. But until the database for the program (*cough* siteadvisor *cough*) is updated in a timely fashion you'll go browsing around the site in a false sense of security. With a hope that your anti-virus program will block anything bad that might be on it. Computer viruses and malware are kind of like STD's. You could get tested at noon, but if the second the last test sample is taken you have sex with the nurse -- the results are potentially invalid for your current state.

Category 2 -- Parses the search result page for links, and then downloads all the info from the link pages to scan

Need I say more? If you're like me and have set your search preferences to return more than the top 10 results (come on, who really finds the exact thing they're looking for in the top 10 every time?), without even visiting a single site your IP address (and all the other "visit" related info) is passed to every search result! Doesn't seem like a problem? Imagine if you searched for "penalties for child molesters" and you notice that a link to a "child 'love'" site is listed. Would you click it (if you knew what it was that is)? Doubtful. But if you have a link scanner such as some anti-virus programs include you just did -- as far as their server logs are considered. So guess what happens if the site gets busted by the FBI? That's right; you get a knock on your door. And who do you think is right -- you that may have purged your computer history, or the server that has the date/time/ip/browser/etc of "your accessing their material" that's been confirmed by your ISP?

And yes, that's a worst-case scenario for the search things -- but here's an example that might hit closer to home.

Virtually every website uses tracking information to see how many people are accessing their website. They use this information for funding, website design changes, rates for advertisers, and countless other things that I can't think of offhand. Those examples are what my company uses the information for (we use Google Analytics as do many other sites). By these programs scanning a website, they're actually downloading the page (causing the site in question to get a "visitor"). If you click the link, they get a "second" visitor for the same information. If the search results have 4-5 results for the same place, they get 4-5 visitors even if you don't click the link. Every visitor access to a website uses your and the company's bandwidth! With some ISPs considering making you pay based on the amount of bandwidth you use (and the companies having to pay for every bit they use -- even if it's in large bulk quantities), these programs are potentially doubling your bandwidth use and the drastically multiplying the company's which can increase the cost.

Don't think that's a problem?

Let's say you search for a company's name. You get 10 results returned on your search page. You've just used bandwidth equivalent to viewing those 10 pages for both you and the company (let's say a total of 1MB). Now you decide to click one of the links. You've just visited 11 pages of that company. Now for you and the company that's not a big deal for the one user. Now let's say 1 million people search for the same thing…. Instead of 10 pages, that's 10 million. Instead of 1MB being downloaded, that's 1million MB (or just shy of 1 terabyte [TB]). Not to mention all the processor cycles required on the company's servers to display the page (php/jsp/asp/etc). And not a single true visit to the web server. Just think of all the bandwidth being wasted, not to mention company computer resources. If a company (even a small one) has to get more and better servers (costing thousands, potentially each month/year) to handle the traffic, don't for a second think they won't pass the cost on in their services. You'll even start to see a lot of index pages that have virtually nothing on them but "company X" followed by a search box or a few links to the main site (funny, isn't it, that this is already being done….). And guess what that does, invalidates the "safe" linkscan result since it's only scanning the index page and not any true content -- at least once companies start having to do that to save costs. What this causes (one page, minimal time, no clicking from that page) is referred to as "bounces" by some; high-bounce pages are frequently redesigned to either minimize bandwidth/processor usage on the server or redesigned for different content to prevent bounces (if possible).

And I know what some of you are thinking: But we'll know if the site is virus/malware free!

Guess what? If you click the link you're virus scanner will do something remarkable -- it'll scan the website for viruses and block them!

And that's the problem. These programs either show potentially out-dated information, or provide redundant scanning that wastes resources and time. Do yourself a favor -- disable these in your programs. Then write to the tech support of the companies and tell them that you don't want this huge "critical error!" looking thing in the system tray when you disable a unnecessary component. Because that's just what many of these do. Disable this "feature" and you get warnings shoved in your face. Then you're either forced to reactivate them, or not be notified (or notice) if your virus scanner or firewall (you know, the actual important things) have stopped running!