The incident highlighted the vulnerability of firewalls that haven’t been properly maintained. Hackers, by scanning servers for vulnerabilities, are quick to locate firewalls that have security holes they know how to exploit. That’s what happened with Proinet, one of several Swedish Web-hosting companies targeted in the hacker attack last month. “We had been notified by Red Hat, the manufacturer of our operating system, that there was a security hole in the core of the system,” says support manager Kjetil Jensen. “They told us Sept. 27 to update the system, but we didn’t get around to it right away. Three days later we got hit.” With a stolen password, the hacker logged on to Proinet and erased 1,600 Web sites.

With such security breaches on the rise, companies are turning to software that doesn’t need constant updating. Rather than rely on scanning programs for signatures or fingerprints of known viruses and worms, the idea is to use behavioral and cognitive science to recognize when an infiltrator is trying to do harm. These security systems inspect, in real time, all data that flow through the network and monitor them for suspicious activity. Ideally they intercept or disarm computer worms, Trojans and phishing e-mails before they strike. “It’s like the guy sitting at the door to an art museum inspecting people who walk in. He doesn’t have to have a picture of the bad guy, but he recognizes him when he sees him,” says Yuval Ben-Itzhak, chief technology officer of San Jose, California-based Finjan.

Some security firms are building intricate user profiles based on millions of bytes of daily data traffic to catch suspicious-looking behavior. Tier-3, an Internet security firm based in Sydney, Australia, tracks each e-mail and file transfer, Web search, instant message and file deletion that moves through the data system. For each user, it builds a baseline of “normal” activity. If anything varies from the baseline—or, say, somebody in the company sends classified files outside the firm—the system prompts the Web administrator. The system may also notice e-mail traffic at odd hours or the sudden purge of a large number of documents. “Instead of looking for a certain virus, we look for any kind of anomaly,” says Geoff Sweeney, Tier-3’s chief technology officer.

Another approach is to monitor external Web sites to figure out which ones are likely to be the source of viruses or other malicious programs. Increasingly, e-commerce sites and social-networking forums play involuntary hosts to a new generation of invisible viruses. Finjan’s security system constantly scans sites for suspicious commands such as “delete file” and “copy file,” and logs them in a database. When a corporate surfer clicks on a link to an infected site, the software blocks the interaction.

Jensen thinks that a behaviorally based security system might have stopped the Proinet hacker. The intruder exploited two weaknesses in Proinet’s firewall: a flaw in the server’s operating program allowing a non-administrator to override existing programs, and the ability of visitors to enter the server from any IP address. A system based on behavioral analysis might have spotted hackers whose IP addresses and passwords didn’t match. It might also have noticed unusual changes to other customers’ Web sites, or the sudden deletion of many files.

IT departments are voting with their budgets. Intelligence-based security products and other Web screening systems now make up 10 percent of the $2 billion antivirus market and are growing by 20 percent annually. By 2011, 70 percent of companies will scan Web traffic for viruses, up from just 15 percent in 2006, says the research firm Gartner Group. The big enemy, of course, isn’t nationalists so much as a rapidly growing underground of professional criminals. Behavioral technology isn’t a panacea, but it’s another weapon in the security arsenal.