Why government search engines can’t handle misspellings

In my post Thursday on the failure to connect the dots pointing to the underpants bomber (can dots point?), I declared without offering any evidence that:

Any halfway affluent individual can assemble a better set of communication devices and networks on her own than she’d ever get from the IT department of a large corporation (or large government agency).

Well here’s Noah Schachtmann at Wired’s Danger Room riffing (via Boing Boing) on the Obama administration’s report on what went wrong:

Government search tools weren’t even flexible enough to handle simple misspellings. As the White House review notes:

A misspelling of Mr. Abdulmutallab’s name initially resulted in the State Department believing he did not have a valid U.S. visa. A determination to revoke his visa however would have only occurred if there had been a successful integration of intelligence by the CT [counterterrorism] community, resulting in his being watchlisted.

This is a problem that commercial software firms largely solved years ago. (Try typing “Noa Schactmann” into Google, and see what comes up.) How it could persist in the CT community, I just don’t understand.

I think CT stands for “counterterrorism,” although I’m not ruling out “chicken tenders.” In any case, I don’t understand why Schachtmann (or Schactmann) doesn’t understand why the CT community uses a crappy search engine. One of the key technological realities of our age is that free software available on the Web is generally much better than the expensive stuff purchased by corporations and government agencies. I’m not entirely sure why this is so—although I know the cloud must be involved and maybe the singularity is too. Perhaps it’s just a passing phenomenon. But it really shouldn’t be surprising that in 2009 some proprietary search software purchased by the State Department didn’t work nearly as well as Google or Bing.

Related Topics: terrorism, Technology & Media
  • Latest on Business

    Don Emmert / Getty Images

    Apple Now Worth More Than Microsoft, Google Combined

    How high can Apple soar? The tech juggernaut is closing in on $500 per share, a dramatic psychological threshold that underscores the company’s stunning performance over the last decade. How massive has Apple become? It’s now worth more by market capitalization than Google and Microsoft combined. The company’s latest stock price surge is being fueled by rumors that a new version of the iPad — the iPad 3 — will appear next month.

    Ben Bernanke Is Wrong About Debt and Interest RatesSlate

    Photo-Illustration by Alexander Ho for TIME; Getty Images

    After Motorola Deal Approval, Can Google Hardware Be Far Behind?

    Now that federal regulators appear poised to approve Google’s $12.5 billion purchase of Motorola Mobility, speculation is mounting about what the Internet giant will do with its newly acquired assets. And what assets they are. Once the deal is formally approved — an announcement could come next week, according to multiple reports — Google will be in possession of some 17,000 patents related to mobile phone technology. And it will be well-situated to enter the hardware business, which could mean that consumers will soon see Google-branded phones and home entertainment devices.

  • dochosvet

    Heck my Mac can’t spell Obama correctly so why should the government get it right. I suppose I could correct it so it quit underlining it every time but why. And now my power just went out. Hope the worlds OK.

  • curmudgeon57
  • jomiku

    Think about this for a moment. Let’s say the search tools found all sorts of possible misspellings. How many names are close to one another and thus how many people are going to be “false positives”? What if the name was misspelled in 3 places or 1 or 5 or maybe the misspelling was in a double letter or in the last position? We can look at this one long name and think it would have been easy to pick this one guy out, but is that true? Aren’t a number of possible suspects likely to have Arabic sounding names? Aren’t a lot likely to have “abdul” in them? (And “muttalab” is a version of another common usage.) How do we assume that the system would then have clearly picked this one guy out? Is it that we have names like Fox or Smith?

    Think again. Why would a person, a human being, a trained human being using a search system for intelligence purpose, rely only on the spelling of a long name? That isn’t intelligent on a human level. The question of the nature of the search system is subordinate to the searching done by a trained intelligence operative who must have had ways to examine the data other than by typing in a specific spelling of a specific name.

  • parakori

    Readers are plentiful; thinkers are rare….

    http://japan-russia.jimdo.com/world-press/

    Beware when the great God lets loose a thinker on this planet.

blog comments powered by Disqus