More on assumptions about search engine use (and related research)

by Eszter Hargittai on April 7, 2004

I had a piece on the BBC News site yesterday. A few people have kindly sent me notes letting me know about this so I thought I should blog it so people know that I am aware of my article on the BBC site. ;-)

I should clarify that my motivation for writing this piece – or any other that mentions Google for that matter – is not a reflection of any personal love or hate relationship I may have with Google.. or any other search engine for that matter. My thoughts on the topic are a result of studying how average Internet users (as in not just me, or just some of my friends and colleagues) find information online. I have tried to make this increasingly explicit in my writing in order to avoid people sending me emotionally charged notes about how I am misunderstanding that one particular company. This part seems to be getting better as no one this time sent me messages explaining to me how to use Google to make the most of it. (Believe me, I know how to use search engines, learning those skills was the least I could do while writing a dissertation on how people find content online.:)

{ 24 comments }

1

Anita Hendersen 04.07.04 at 1:51 pm

“Referring to Google has become the high-culture status symbol of web use.”

Three or four years ago, maybe. Google’s market share is too high for it to be a status symbol now.

2

des 04.07.04 at 1:52 pm

If Google went rogue, it could easily be replaced, for sure, in which case what’s the big deal with our “emphasis” on it? Our (or at least my) loyalty is in the shallowest of soils planted and would not much storminess withstand.

The Deja-Google Usenet archive, OTOH, that’s a real hostage to fortune.

3

Maria 04.07.04 at 2:50 pm

Great piece Eszter!

Anita, when you say ‘market share’, think of who that market is, i.e. regular internet users. I think regular perusers of this blog / colleagues / people I knew in graduate school and so on probably wouldn’t think that referring to Google shows more than average knowledge of the internet. But, when I think of people I know through family,neighbourhoods, etc. I can come up with dozens who probably have only the fuzziest notion that google has something to do with the internet and beyond that are a bit stumped.

That’s just anecdotal – I think Eszter has the hard evidence that knowing vaguely what Google is is not at all the same as being able to use it.

4

dave heasman 04.07.04 at 3:00 pm

I think Maria has it. I am a “power” user of the net but my wife, when doing an internet search, hits the search key in the IE toolbar, which brings up MSN search. Which is better for finding North London kitchen designers but worse at finding the lyrics to “Choctaw Bingo”

5

eszter 04.07.04 at 3:05 pm

Thanks, Maria, yup, that’s precisely the kind of broader view I am attempting when writing about these things.

6

humeidayer 04.07.04 at 3:53 pm

Eszter, given your Google posts, if you’ve never seen Newsmap you may find it interesting.

(Actually a few of the members here may find it an interesting exercise in data visualization.)

7

Anita Hendersen 04.07.04 at 4:52 pm

Up until Yahoo dropped Google a couple months ago, Google had about 80% share of the US market in searches. This wasn’t all through google.com; it included AOL search, Netscape search, Yahoo, iWon, and others, all of which used the Google engine. A search on any of these engines produced substantially the same results. That’s why Google was so powerful.

Check out http://searchenginewatch.com/reports/article.php/2156431 for an estimate of market share breakdown in February.

It’s snobbery to think that most Internet users don’t know about Google. Knowing there’s a site called google.com is no special thing. A better hallmark of Internet search sophistication is probably knowing how to use the advanced search functions on Google or knowing about vivisimo.com, teoma.com, and scirus.com. Google is like Kellogg’s Cork Flakes now. It’s great, but it’s not some special tool of the elite.

“I think Eszter has the hard evidence that knowing vaguely what Google is is not at all the same as being able to use it.“ Yes, I agree with that.

8

eszter 04.07.04 at 5:05 pm

Anita, I addressed the issues you raise in this piece. Of course, if you know of other data, I would love to see pointers. (The data you refer to don’t address my concerns as per my discussion of issues in the linked to piece.)

9

Anita Hendersen 04.07.04 at 5:22 pm

I don’t know why you seem to think I’m disagreeing with you. I’m not. People should be better at searching on the Internet. And I appreciated that paper you wrote on how people search. (You sent me a copy.)

But it’s wrong to think that Google is less than a juggernaut. I do Internet marketing and look at the referral logs for a lot of sites (US based sites aimed at US residents.) These are not tech-oriented sites. They are aimed at ordinary people. And the largest referring domains is almost always Google. Typically the second largest is Yahoo and Google refers more than twice as many visitors as Yahoo. Check out the industry message board at webmasterworld.com and others will tell you the same thing. Google is the people’s search engine (whether they are using it correctly or not). I don’t think it’s healthy for the internet for Google to be as powerful as it is, and I wish people would use other engines.

10

Maria 04.07.04 at 5:32 pm

Anita – I think one of the points Eszter is making is that knowing ‘about Google’ and knowing ‘how to use Google’ are not the same thing. And she has the evidence to prove it.

Also, it’s not remotely snobbish to suggest (or even be able to prove) that a significant portion of average internet users don’t know what Google does. Sure, your friends do. Most of my friends do too. But many, many people don’t. Would it be snobbery to suggest that, say, 15% of Americans are functionally illiterate? (I just made that number up for illustration.) Or that – again, just making these figures up – 40% of them have never learned how to navigate a mass transit system? It’s not snobbery to show that many people simply don’t know how to do something that others take for granted. Especially if doing so helps us to start thinking about how to address that. For me, that’s the whole point of research like this – to find out counter-intuitive things and challenge our assumptions.

I agree that using (and getting the most out of) sites like Teoma is a measure of search sophistication – but that is probably the very highest level of sophistication. Eszter’s research seems to show that your ballpark guess of how sophisticated the average internet user is way off the mark.

Also, you might want to consider that ‘80% of US market share’ of searches is a very big number of people, but not remotely representative globally… ;-) Further, all this stat shows is that of the people competent to perform internet searches 80% are using Google. By definition, it doesn’t show the people who have heard of Google but don’t know how to use it. And the 80% includes but does not measure another group Eszter mentions, people who can type a few words into the box, but don’t know enough about how search engines work to get good results.

Bottom line; for many people, even knowing about Google is rather special, and for many more, knowing what it does and how to use it is very special indeed.

11

Ed Zeppelin 04.07.04 at 7:47 pm

Kellogg’s Cork Flakes

Mmmmmmmmmm . . . . cork flakes.

12

paul 04.07.04 at 9:48 pm

The argument from the BBC piece seems to more about learning how the different search engines work and how to work them than about any one search engine, just as one learns how to read a reviewer’s comments to see if the latest opera at the Met is worth going to see. A better example might have been MSN search since it lives in 90% of the world’s browser toolbars: it seems using Google has confused the issue.

What’s interesting to me is how this impacts the meta-search engines that use results from different companies (meta-crawler was the first of its kind). If you know how to drive Google (you have mastered how it weights keywords and what stop words it will ignore), what do you lose when you run the same query through a meta-search tool that might rewrite the query as it hands it off?

I wish there was more work being done on keywordless searching. I worked on a startup company that was building out a service for online publishers: the nutshell summary was that each page on a given site was analyzed and modelled in an n-dimensional matrix, and when a user wanted to find pages similar to the one they were viewing, computations of mathematical distance between matrix nodes would generate results that were often uncanny.

Keywords are so 20th century ;-) The guys who invented this technology are still actively developing it but on a self-funded (ie, slow) pace @ http://www.thinktank23.com.

13

John Quiggin 04.07.04 at 9:49 pm

Is Google different from older information sources in this respect?

For example, most people are aware of libraries, have been to a library at some time and so on. But the proportion who can effectively use a library, and the books in it, to get information is quite small. You need essentially the same skills as for Google – understanding what an index is and how to use it, distinguishing credible from less-credible sources, and so on.

14

eszter 04.07.04 at 10:15 pm

John, what makes it potentially a bit different is that once people have Internet access at home (or even at work if fairly autonomous use is assumed) then the costs of using the medium to look for information are significantly reduced. You don’t have to get to a location at a particular time when it happens to be open, you can just sit down at your own machine. Also, most public libraries would only have so much material available locally. (Of course, as materials are increasingly moved to password-protected sites, the amount of online information available immediately even to those who do know how to find content will decrease considerably.)

15

yabonn 04.07.04 at 11:26 pm

Pet peeve of mine.

I’m wondering if there’s not a whole category of jobs that will take a hit as the search engines become more popular, or at least make some professionnal lives a bit trickier.

Search engines/net work tremendously well at trivialising relatively arcane know-hows. Nutrition. Fiscal counselling. Plumbing. Applied statistics. Whatever.

Experts at something are often people simply able to situate the consumer’s case into their field of knowledge, then pick up the Big Book, and apply. Most often fields are too wide to master in every of its technicalities.

But the consumer doesn’t need anymore the years of Big Books study proudly shoe horned into the expert’s mind. Global knowledge of the field is not necessary anymore : simply add criteria to reach the very info about your very case.

And there’s too, increasingly, the global knowledge on the net, if you want to. For the expert, it all the same stopped to be a privilege and to be simply marketably useful.

Mmhm. There are unconvincing tones of new economy baloney to that last chapter.

Bah. Add a few years at your convenience, and it’ll be ok :-)

16

John Quiggin 04.08.04 at 12:52 am

A more general version of the question.

Is Google reducing or amplifying inequality of knowledge? At first glance, it appears to reduce it, as in Yabonn’s comment. But in important respects, inequality of knowledge is amplified.

For example, I have a lot of half-remembered quotations floating around in my head. With a library, these are almost impossible to track down. With Google, I can frequently do it almost instantly. But if I didn’t have the fragment, I wouldn’t know what to look for.

17

eszter 04.08.04 at 5:09 am

John, you raise the question that is of central interest to me in my research, but at a bit more general level: does the spread of IT increase or decrease (or leave unchanged?) social inequality? This is an empirical question so I look for (and sometimes collect my own) data to try to answer it. Regarding search engines in particular, my data suggest that there’s quite a bit of variance in people’s ability to find content online efficiently. These abilities are sometimes related to levels of education and income. This would suggest that in some ways these new media and services may contribute to inequality rather than reduce it. But much more work remains to be done on all this.

18

Motoko Kusanagi 04.08.04 at 12:13 pm

Didn’t you see Maid in Manhattan, Eszter?

19

Abiola Lapite 04.08.04 at 12:26 pm

“the nutshell summary was that each page on a given site was analyzed and modelled in an n-dimensional matrix, and when a user wanted to find pages similar to the one they were viewing, computations of mathematical distance between matrix nodes would generate results that were often uncanny”

This sounds like your old latent semantic indexing to me.

20

Abiola Lapite 04.08.04 at 12:33 pm

“This would suggest that in some ways these new media and services may contribute to inequality rather than reduce it.”

The idea of increasing returns in the context of knowledge acquisition’s been around for a while, and I’d say you’re very probably on the right track here. To those that have, even more will be given …

21

yabonn 04.08.04 at 1:54 pm

Eszter,

This would suggest that in some ways these new media and services may contribute to inequality rather than reduce it. But much more work remains to be done on all this.

Even after these tools go mainstream, all possibilities of search engines will not be available to all, so i guess in that sense it will contribute to inequality within the population. Add a layer to illiterate/litterate/literate-and-googling.

But is that contributing to inequality? They only fail in that they can’t make themselves available to all. So yes, there are inequalities deriving from these tools but, as for the contributors, i’d tend to look for the usual suspects of social determinants.

Similarly, it can be said that litteracy created inequalities in that illiteracy subsists, but as for the _contribution_ to inequality, i’d tend to look for the state of the educational system, familial cultural capital, etcetc.

There’s always that sound reflex to check who’s left on the pavement when some New Thing arrives, to see if it’s really a progress or some trickle-down, rising-tide-lifts-all-boats bollocks. I think these tools – just like litteracy – pass the test. Barring the social determinants (and then it’s more of the same), it’s simply a common good without a rarity constraint, and then the more the merrier.

Btw doesn anyone know a search engine that looks for the real string? Without discarding the – and / and ‘ and ² etc? Couldn’t get google to do that.

22

Ruben Puentedura 04.08.04 at 2:59 pm

A complementary issue is the appearance of fundamentally dishonest search aggregators such as Dogpile, which merge results from “pure” searches with paid ad placement services, while deleting all info that would clearly identify the latter. It would be very interesting to conduct a study to see how users from a range of educational backgrounds respond to Dogpile’s interface. I fear it may not be just Internet novices who are taken in here…

23

eszter 04.08.04 at 9:33 pm

Yabonn – I do look at those other factors in my studies as well. I certainly include information about people’s education and income, etc. In fact, one of my goals has been precisely to see how those social factors influence differences in people’s online abilities. (The answer is that some do, some do not, it depends on the type of task, it’s a complicated question.)

Ruben P – Yes, the issue of how commercial considerations influence search engine results is also of interest to me. I have a few papers on that.

24

John Quiggin 04.10.04 at 12:10 am

A different, though obviously related, question. How should education be modified in the light of Google?

In general, I think, technological development has favored abstract knowledge and meta-knowledge over concrete facts. I don’t need to know which US President succeeded US Grant, since I know where to look it up. So people should be taught more political theory and research skills and less dates and battles.

Google is ambiguous here. Obviously, I can look stuff up that I didn’t know. On the other hand, unlike a reference book or textbook, Google is an index to a gigantic unsorted mass. So abstract knowledge isn’t of as much use in formulating a good search string as is some related concrete knowledge.

By comparison, the old Yahoo approach (which I never use these days) was much more like a traditional reference text.

Comments on this entry are closed.