A basic overview of how Google ranks pages in their search engine results pages (SERPS) follows:

1) Find all pages matching the keywords of the search.
2) Rank accordingly using « on the page factors » such as keywords.
3) Calculate in the inbound anchor text.
4) Adjust the results by Page Rank scores.

In reality, it’s slightly more complex and we’ll discuss this in more depth later, but for now the above description serves our purposes. It’s worth noting that Page Rank is a multiplier and is not just simply added to the score. Thus, if your page had a PageRank of zero, it would rank at the very end of the SERPS.

How is PageRank determined?

The Google theory goes that if Page A links to Page B, then Page A is saying that Page B is an important page. PageRank also factors in the importance of the links pointing to a page. If a page has important links pointing to it, then its links to other pages also become important. The actual text of the link is irrelevant when discussing PageRank.

How can you tell what a page’s PageRank is?

To learn what a page’s PageRank is, you can download a toolbar for Internet Explorer from http://toolbar.google.com. Once installed, there will be a bar graph at the top of the browser showing a version of PageRank for the page you’re browsing. When you hold the mouse over the bar, you see a number from zero to ten. (If you don’t see the number, you may have an older version of the toolbar installed. You will need to completely uninstall it, reboot your computer and reinstall the latest version. Once this is done, you should be able to see the PageRank number.)

1825 South Grant St., Suite 410 San Mateo, CA 94402
search-support@ altavista. com (650) 295-2500
Fax: (650) 295-2532

It is virtually impossible to reach a live person at AltaVista. Every extension we dial off the voice-mail greeting landed us in voice-mail hell. Remarkably, when you choose selection option #1, which claims to send you to « technical support, » you are transferred to a voice-mail message that instructs you to use the « feedback » button from the AltaVista Web site if you are calling about your Web site’s listing. Unbelievable!

Search for keyword: « Mortgages »
Yahoo! Site Matches (1 -3 of 2165)
Yahoo! Search Engine August 2nd, 1997 (Bolded italics denotes a Yahoo! category)

Business and Economy: Companies: Financial Services: Financing: Real Estate

BayBank Mortgages -mortgage information, daily rates, calculators, and online applications. Home Mortgages of the Palm Beaches Ltd. -lender originating loans throughout Florida.

Business and Economy: Companies: Real Estate: Regional: U. S. States: Colorado

First City Financial Mortgages -low rate residential mortgages for purchase or refinance. Offices in Boulder, Colorado Springs, Denver, Evergreen, Fort Collins, Greeley and Loveland.

This example* elegantly demonstrates prominence and frequency ranking relevance. The first entry has the keyword twice as the second word of the site title and as the first word of the Web site description.

The second entry has the keyword as the second word of the title, but no additional points scored for frequency as the keyword does not appear in the description.

* Note pertaining only to this « mortgage » example: Yahoo! has modified its search algorithm (since August 1997) and how it determines relevancy. However, these search results illustrate the general concepts of keyword prominence and frequency rules neatly so we kept this older example in this report to help you better understand these concepts. The other rules and tips in
this section on Yahoo! still apply.

Learning to Alphabetize – The ASCII Hierarchy

A search engine is a database, and databases have rules. Have you ever noticed that when you rename a file to start with an « ! » it moves to the top of your file list in Windows 98? The same holds true for some search engines, certainly Yahoo!, which ranks alphabetically as well as by myriad other criteria.

The ASCII hierarchy dictates which symbol, yes, « symbol, » will lead the pack. The highest ranking symbol is a space. This invisible symbol ranks highest in the hierarchy of the ASCII
alphabet.

Here is your new alphabet. Learn it well.

The ASCII Hierarchy:
1. space 13. , 25. B 37. N 49. Z 61. f 73. r 85. ~
2. ! 14. -26. C 38. O 50. [ 62. g 74. s
3. " 15. . 27. D 39. P 51. \ 63. h 75. t
4. # 16. / 28. E 40. Q 52. ] 64. i 76. u
5. $ 17. : 29. F 41. R 53. ^ 65. j 77. v
6. % 18. ; 30. G 42. S 54. _ 66. k 78. w
7. & 19. < 31. H 43. T 55. ‘ 67. l 79. x
8. ‘ 20. = 32. I 44. U 56. a 68. m 80. y
9. ( 21. > 33. J 45. V 57. b 69. n 81. z
10. ) 22. ? 34. K 46. W 58. c 70. o 82. {
11. * 23. @ 35. L 47. X 59. d 71. p 83. |
12. + 24. A 36. M 48. Y 60. e 72. q 84. }

Above are the symbols and letters in their ASCII rank. There are of course more characters in the ASCII character set. However, the above are the ones you’d be more likely to use and are in ranked order. While we’re certain you can read the chart above, let us point out several connections which may not be obvious at first.

Capital letters rank higher than non-capitalized letters. « MORTGAGE » will rank higher than its lowercase equivalent, « mortgage. » Yahoo! specifically asks you not to use text in all caps in your site description, but you can at least capitalize the first letter.

Where appropriate, replacing letters with certain symbols puts you at the head of a list. It may be inappropriate or blatantly offensive to a search engine’s rules to randomly include an exclamation mark ahead of your company name, or worse, doing the phone book trick and adding « AAA » to your site name (especially if that’s not what you go by on your Web site).

But what if your product or site really is named « @ DVANTAGE! » or something similar? You’re ahead of any competitor named simply « advantage. » What if your Web site is all about $ales $uccess?

So long as the name appears to be used in trade [throughout the Web site] with the ASCII character, we shouldn’t have a problem with it.

- « Rose, » Surfing Manager at Yahoo!

Remember that Yahoo! does not rank your site by keyword weight on the page itself, popularity of your page (number of other sites linked to you), or by anything other than the quality and content of your submission to Yahoo!. Keep in mind that while using a symbol may improve your ranking within a category, it will also hurt your chances of having people find you based on a keyword search. People are much more likely to search for « sales success » than for « $ ales $uccess. »

Alphabetically – sometimes:

Yahoo! does rank sites alphabetically within categories – this we know for certain. What you may not know is that when Yahoo! returns matches for keyword searches, after the categories are returned, Yahoo! returns Web sites within each category, and only the sites within those categories that are most relevant to the keyword search. This means that Yahoo! looks at the prominence and frequency of keywords in the site title, description and URL.

You’ll notice that there is no alphabetical advantage in sites returned in response to a keyword query that appear within a category. Yahoo! only gives an advantage to sites within the selected category when the person visits that category, not when search results are listed in response to the keyword search.

WARNING! Yahoo! has corrected sites that use overt ASCII hierarchy tricks such as leading with an exclamation point or other non-alpha character. If you do this, you risk a delay in your site being listed, and it will most certainly be corrected by the category editor. If your site is legitimately named something that starts with a symbol, such as « $ ales $uccess, » and when the Yahoo! staff member visits your site they find that usage throughout, you may be allowed to use this approach and gain an advantage.

You should consider renaming your site if your company or products are alphabetically challenged as in the case of a sheet music dealer named « Xylophones Sheet Music. » Consider renaming the page « $ 1 Xylophone Sheet Music. » Don’t use the classic Yellow Pages trick of prefacing your page name with « AAA. » Yahoo! staff will see right through that and adjust your feeble attempt to spam its directory.

SPECIAL NOTICE: Some Web sites that contain timely news or information can sometimes get indexed very quickly – if you ask for it! For instance, Yahoo! has a « very specialized movie surfer » who specializes in getting movie sites listed as quickly as necessary for release dates.

If you are promoting an event or some product with an expiration date and you must get your site listed quickly, you should consider contacting Yahoo! and seeing if they will help. The Yahoo! staff is awesome (especially « Rose ») and are very committed to making Yahoo! a terrific product. They will help you if they can.

First – Check to see if you’re already listed in Yahoo!

Go to http:// www. Yahoo. com and search for your domain name. Simply type your domain into this search screen:
http:// www. yourcompany. com
Then, search for categories that you would list yourself under and review the categories for your site. If you are confident that you’re not already listed by Yahoo!, proceed to step 1.

How to pay $299 for guaranteed seven-day review and possible listing of your Web site in Yahoo!:
In February 1999, Yahoo! announced a new fee-for-review service that would allow site owners to submit their site to Yahoo! with a promised seven-day review. NOTE: Paying this $299 does NOT guarantee that your site will be added to Yahoo! – it only guarantees that one of their editors will review it and give you a decision on whether or not they will add it to Yahoo!.

If they turn you down, they offer an explanation as to why you were turned down, and you are allowed to resubmit under the same program without paying an additional fee within seven days.

It’s not a perfect system, but for those of you who have waited eight to ten weeks after submitting only to discover that your submission was ignored, it may well be worth the money.

To use this service, go to this link:
http:// www. Yahoo. com/ info/ suggest/ busexpress. html
NOTE: This service is only available to e-commerce-enabled Web sites for businesses based in the United States! Here’s how Yahoo! defines it: « ‘ Online transactions’ means that a customer enters their credit card number on the site in a secure or standard online environment for the purpose of purchasing goods or services from the site, and the credit card number is authorized for payment by the site. For purposes of this Agreement, the following will not (emphasis added) be considered « online transactions »:

(1) if any part of the transaction other than a general confirmation of the customer order occurs via e-mail, fax, or phone,
(2) affiliates or associates programs, multi-level marketing programs and similar promotional arrangements where
the purchase occurs on a third party site, and
(3) if the only transaction is a process where customer accesses the site via a verification system (e. g.: an adult verification system) where the customer credit card number is entered on a different site. »

Step-By-Step Instructions for getting listed in Yahoo! without paying:

Step 1: Make sure your Web site looks good. Yahoo! category editors have no obligation to list your Web site. They include a lot of disclaimer language on their submission pages telling you they reserve the right not to list your site -

believe them! Do not submit your site to Yahoo! until it is ready – do not submit any site that has an « under-construction » graphic on it and nothing else. It will probably not get indexed.

Step 2: Submit to Yahoo! manually. Do not use submitting software, submitting service or any other automated tool.
Sure, some may work and get you listed, but Yahoo! is so important that you will want to take the time and do it by hand. Besides, some submission services and tools won’t put you in the right category, as Yahoo! categories change constantly.
Submissions from the submitting software tools and even the submitting services are often gibberish. We much prefer it if you do your submissions manually. That way you can read our rules and do it properly – it makes our job easier.
- « Rose, » Surfing Manager at Yahoo!

Step 3. Choose two categories where you would like your Web site listed. To do this, you should make a list of your top keywords and search on them. Yahoo! will only allow you to be listed in two categories. You should make an effort to pick the best ones. Yahoo! returns categories first, and then categories with a few major sites in them next. After doing a search on the keywords or keyword combinations that you expect your customers to search on, decide if they will spot a category and click on it first, or if they will scroll down and look for a site that is listed in a category below.

You will likely be put in the category with other Web sites or businesses like yours. For instance, though you might really want to be in the ‘teachers category’ because you sell to teachers, you will end up in a the text books category if that’s what you make.
- « Rose, » Surfing Manager at Yahoo!

For instance, if you think one of your keywords will be « real estate » because maybe you’re a real estate broker in Virginia, you’ll quickly see that Yahoo! returns thousands of categories. None on the first page are even close to something that someone looking for real estate in Virginia would select. You might surmise that the person conducting this search would try their search again
using « Virginia Real Estate. » For this search, the first category returned is:
Regional: U. S. States: Virginia: Real Estate

Following this link brings you to an alphabetical list of real estate agents, services, etc. for Virginia. Further down, another category:

Regional: U. S. States: Virginia: Cities: Alexandria: Real Estate is an even better category for the real estate agent specializing in that big city. For my client, Roger Derflinger, an established Virginia real estate agent, we chose this category. The Yahoo! editors might still move our site to another category that might more aptly deserve this listing. We then conducted a similar search, « Northern Virginia Realtors, » and this was the first category returned:

Top: Business and Economy: Companies: Real Estate: Regional: U. S. States: Virginia: Complete Listing
We added his site here, too. We always try to find a category that matches a client’s business or service, and also a category that appears closer to the top in a series of search results.

Step 4: Select the category and then look for the « Add URL » button from the top of this section’s page. You will be presented with a series of questions about your site, specifically the site’s URL, what you would like the site to be titled, and a site description. You will also be asked to suggest another category where your Web site should be listed. Input your second choice in that field.

Step 5: Carefully construct a site title making sure to include important keywords near the beginning of the title, preferably as the first word or two.

As a Yahoo! category editor, I will do everything I can to find the legitimate name of the business. We prefer it if you go by the name you gave your business as a site title. »
- « Rose, » Surfing Manager at Yahoo!

This site title should not exceed 40 characters. Yahoo! ranks sites alphabetically, so try to choose a site title that is alphabetically superior. Don’t be overt and name the site « A+ techniques to selling paint » or something that obvious – you will get caught, and Yahoo! will likely add you with your actual business name. See the discussion below about the ASCII hierarchy and which letters and symbols rank higher than others.

If you’re a commercial site, you won’t get added [by using a title that describes the site content and not the company name]. Don’t use marketing lingo like, ‘We’re the best. ‘ If I have a choice between a site title like, ‘a really good book’ or ‘The Cambridge Paperback Encyclopedia, ‘ I will spend more time on the one using the correct title.

- « Rose, » Surfing Manager at Yahoo!

You may wish to consider alphabetical superiority if you have control over your company name – naming your company « A d. b. a. (doing business as) company name » for the purposes of your Web site to be first in a category can result in a tremendous volume of traffic. This is no different than what people have been doing in the phone book for years:

AAACME Vacuum Cleaner Repair AAA Auto Service.

There’s value in being first, but only you can determine if this would be ethical or appropriate given the type of business or organization you lead. If you are about to build a Web site to sell a product, and you haven’t really established a company or product name yet, consider a name that is alphabetically advantageous. If you are « Zebra Systems » and well-known in the marketplace as such, you’re out of luck. Renaming your company or product for the purposes of your Web site will mean that people who know your company and are simply trying to check for your business hours won’t find you.

Step 6: Carefully construct a site description, repeating one or two of the important keywords in the first couple of words that make up your site description. This field can be only 200 characters, including spaces, or 25 words. Use the description wisely to make it keyword rich. Try to avoid any stop words, like « the » or « and » and so on. Don’t waste words when you don’t have to.

If [using keywords early in your site description] is done appropriately and it really describes the Web site, it will go through. Otherwise we’ll change it – but you can try it!

- « Rose, » Surfing Manager at Yahoo!

Step 7: If you don’t get listed after two weeks, try improving your site and submitting it again.
Or, call Yahoo! ‘s listing service line at (408) 731-3333 and leave a message asking why you weren’t listed.

Yahoo! Notes:

Yahoo! indexes characters, not just whole words.
Yahoo!, like most engines, indexes words, phrases AND characters. This means if you’re searching for « travel, » you will also find « traveling » and « travels. »

Examine your keywords and try to use the longest variation of your important keywords. In our « travel » keyword example, the word « travel » is also present in the words « travels » and « traveling. » With Yahoo!, by using the word « traveling » you are assured hits when people search on « travel » OR « traveling. » You are, in effect, including two keywords for the price of
one!

How to Change Your Yahoo! Listing

Here is a process for to persuading Yahoo! to change your existing listing. This technique may speed the process.

First, request your change using their online form located at:
http:// add. yahoo. com/ fast/ change
If nothing happens after five days from your online submission, call Yahoo! ‘s advertising phone number: 408-731-3300
Press Option 2 for Other Press Option 3 for Yahoo Online

Press Option 2 for Web site listings (Note: they could change the « menu » at any time).
Leave a message with your URL, date you filled out the change form, and any other information they ask for. We have experienced changes occurring within four days of a phone call – very fast by Yahoo! standards.

You will have more success in changing an existing listing if your reason is perceived as something more serious than just for the sake of improving your ranking (which they could care less about). For example, if you’ve created a new domain name for your site, request that the description be updated, and the new domain name be used as indicated in the change request form you filled out. Other credible reasons you could offer would be a change in your company name (if it has changed), a change in the emphasis of your business, thereby requiring a new description, or a typo that needs to be corrected.

YAHOO! Yahoo!, Inc.
3400 Central Expressway Santa Clara, CA 95051

Main Corporate Telephone: (408) 731-3300 Fax: (408) 731-3301

User Support Telephone: (408) 731-3333 Extension #1: For information on submitting a new URL
Extension #2: For information on making changes to an existing listing in Yahoo! Extension #3: For information about Yahoo! search results

URL to the engine http:// www. yahoo. com

Must submit EACH page? JUST ONE
How long to index my page? 2 to 8 weeks (or $199 for a 7 day
review)

Recognizes and supports META tags? NO

Are searches case sensitive? NO

Does a spider index the site? NO

Uses data entered on submission form only? YES

Is page popularity a factor? NO

Alphabetical ranking used? YES (only within Yahoo! categories)

Keyword weight plays a role? NO, frequency and prominence only

Title tag considered for relevancy? YES, but only for the title you enter on their submission form.

Prominence of keywords in title tag important? YES, but only for the title you enter on their submission form.

Frequency of keywords in title tag important? YES, but only for the title you enter on their submission form.

Comment tags considered for relevancy? NO

Max. length of title accepted 40 characters (unless company name is longer)

Max. length of keyword META tag N/ A

Max. length of description field you can submit 200 characters – no more than 25 words total

How to check to see if you’re listed Type in full URL: http:// www. yourcompany. com or your site title as you submitted it.

If you are a business, you are listed by your business name – Yahoo! will accommodate bona fide long business names as best as possible.

How to check your link popularity Yahoo! has no spider, so no links are tracked.

E-mail support Go to:
http:// www. Yahoo. com/ info Many addresses are available. Visit this link to find appropriate e-mail address. The most important is: url-support@ Yahoo-inc. com

Total documents indexed 730,000 total sites in their directory. Searches for sites or keywords not found in Yahoo!, automatically sent to the Inktomi search engine (which also power HotBot, among others).

Total page views 167 million page views per day in December 1998 (source: Fourth Quarter and 1998 Fiscal, Yahoo! press release).

Add/ Remove URL Select the appropriate Yahoo!
category and then select the « Add URL » button on the top of that page. OR: Pay $299 for a guaranteed seven-day review of your site (if you accept credit card transactions on your site only!), though Yahoo! does not promise your site will be added to their index. For paid submissions to Yahoo!, visit this link: http:// www. Yahoo. com/ info/ suggest/ busexpress. html

Yahoo! is the most important search service on the Web. Just look at the number of page views a day, more than five times the nearest competitor! Yahoo! was the first search engine and is still the leader. Get your site listed in Yahoo! first, and do everything else second. Any questions? Our guide is the ONLY guide on the market that features an interview with an actual Yahoo! executive who told us the specifics on what Yahoo! looks for in adding new Web sites. No other guide includes the following interview. Please read it carefully – it’s golden.

We had the distinct pleasure of interviewing a Yahoo! « Surfing Manager » for our research on getting listed in Yahoo!. We’ll call her « Rose » as she asked me not to list her real name for fear that she’d get thousands of calls. Much of this instruction comes as a direct result of that interview. It was quite an honor to have a Yahoo! senior staffer (she’s been with Yahoo! for four years – almost since day one), share this rather sensitive information. We are very honored to have such a great source at such an important search service.
Yahoo! Listing Support: (408) 731-3333. This is a very interesting service. In the past, when
you dialed this number you used to reach any of six staffers who answer this line who would answer specific questions – now you get an informational voice-mail message only. At the end of the messages, you will hear recorded instructions on how to leave a message concerning any problem you have had getting listed with Yahoo! ‘s main page. This is a great service for anyone struggling to get a site listed in Yahoo!.

Considering that by some accounts Yahoo! adds only one in ten of the submissions to its index, this number can be valuable in finding out why you are encountering problems, or can help in getting information on how you can get your listing included in Yahoo! ‘s directory. « Rose » at Yahoo! suggested waiting a couple of weeks before contacting anyone at this number and to search on your submitted title and URL to confirm that you’ve not been listed yet before calling this number.

Yahoo! ‘s relevance scoring is based on a very complex and proprietary algorithm (like all the search services) and they, of course, do not wish to reveal it. It seems to be based on the submission’s category first, then title, then description, and then (hint) your Web site URL, depending on the search. Knowing that Yahoo! considers your site’s URL as part of its relevancy scoring system is new information, and not confirmed, though we have good reason to believe it is accurate.

WARNING #1: Yahoo! is one of the three search services reviewed in this book that do not allow you to submit multiple doorway pages. Yahoo! is populated entirely by user submissions – each one reviewed by a professional editor. Do not attempt to use a multiple URL strategy to gain an advantage in Yahoo!. It won’t help you.

Yahoo! is not technically a search « engine; » it is a search « directory. » Every submission to Yahoo! is reviewed by a human being. A real person visits your site and determines how it should be indexed. Yahoo! does not use spiders to index your site. Therefore, META tags are not necessary on your Yahoo! pages, nor will they help you. The result of this policy is a smaller, but cleaner index of Web sites and businesses on the Net.

WARNING #2: Do not play games with Yahoo!. While our informant at Yahoo! told us that they won’t necessarily reject a submission wholesale if it appears to be gaming their system, they might decide not to get to it right away. And since some site managers believe that Yahoo! accounts for most of their site traffic – well, draw your own conclusions. You cannot afford to be denied a listing in Yahoo!.

Please do not read only the Engine by Engine Analysis. There are valuable tips in the previous sections that apply to nearly all engines. Those sections will also allow you to better understand the information contained in this section.

Each search engine listing includes a quick reference table. This table is intended to provide at-a-glance information about the particular search engine’s relevancy scoring criteria and other related information. It primarily includes information that varies by engine. There are many factors that apply to most engines that are described in previous chapters.

It is important to note that different search engines change their relevancy scoring criteria from time to time, and as a result, this information may need to be updated frequently.

Quick Reference Tables Explained

1. URL to the engine - The Web site address for the search engine.

2. Must submit EACH page? - Many search engines spider much of your site, but you may submit pages individually to ensure they get indexed, since the spiders generally only explore so many levels of your site before stopping. Others, such as Infoseek, only index the pages you submit.
3. How long to index my page? - How long it will take you to get added to each search engine.

4. Recognizes and supports META tags? - Whether the search engine pays any attention to META tags. If the answer is YES, then make sure that you include META tags on a doorway page that is specific to this search engine. If NO, you need not waste your time preparing META tags for this engine.

5. Are searches case sensitive? - Whether the search engine notices if keywords have been capitalized or not. In other words, if the engine is case sensitive, a default search for « bank » would yield a different list of sites than a search for « Bank. » If the search engine is case sensitive, include capitalized AND lowercase versions of important keywords in your submission or META tags.

6. Does a spider index the site? - Whether the search engine sends a spider to explore your web site.
7. Uses data entered on submission form only? - Whether the search engine uses the data you fill out on the submission form to determine your Web site’s rank rather than spidering the site and indexing the actual contents. You can assume that if the engine uses submission form data that it doesn’t spider the content of your site. However, a real person may look at your site to make sure it matches the description you typed in.

8. Is page popularity a factor? - Whether the search engine uses some measurement of how many other Web sites are linked to you as part of its relevance scoring criteria.
9. Alphabetical ranking used? - Whether the search engine uses any alphabetical or ASCII hierarchy measurement to determine your site’s relevancy score.

10. Keyword weight plays a role? - Whether the search engine visits your Web site and measures the number of keywords as compared to the total number of words on the page as part of its relevancy scoring criteria.

11. Title tag considered for relevancy? - Whether the search engine considers the title tag as part of its relevancy scoring criteria.
12. Prominence of keywords in title tag important? - Whether the search engine considers the prominence of keywords in the title tag as part of its relevancy scoring criteria.

13. Frequency of keywords in title tag important? - Whether the search engine considers the frequency of keywords in the title tag as part of its relevancy scoring criteria.

14. Comment tags considered for relevancy? - Whether the search engine considers the comment tag (the tag that looks like this: <!–) as part of its relevancy scoring criteria.

15. Max. length of title accepted - This line tells you how many words or characters this search engine will accept in the title tag. It’s best to not exceed this number.

16. Max. length of keyword META tag - How many words or characters this search engine will accept in the keyword META tag.

17. Max. length of description field you can submit - How many words or characters this search engine will accept in the description field of their submission form, if they have a
submission form.

18. How to check to see if you’re listed - How to see if the search engine has a record of your Web site in its database.

19. How to check your link popularity - When supported, this is how you would check each engine for the number of other URLs or pages that show links to your Web site. Not all search engines support this feature.

20. E-mail support - An e-mail address for communications concerning your ranking in a particular search engine or directory. Don’t expect all search engines to reply to your questions, or at least to reply promptly. Many are very busy and appear to make it a policy to ignore most e-mail. At times, you may get a response though, so it’s worth a shot.

21. Total documents indexed - The total number of Web sites or other Web documents that the search engines represent as included in their index.

22. Total page views - Total number of times a search engine’s main search page was accessed and used. This number does not necessarily suggest the number of people who conducted actual keyword searches. It is an important number as it gives you some indication of how many people are using the search engine. When this information was not available, we attempted to substitute another metric that offers insight into the search engine’s or directory’s relative value to Web site marketers.

23. Add/ Remove URL - The location within the search engine where you submit your URL for indexing or where you submit your site title and description. Some engines dynamically generate or serve this page so the actual URL to that page cannot be listed. In those cases, we identify the location of the button or link to this page.

We discovered a critical shortcoming in Microsoft FrontPage that Web site marketers must address. It may apply even if you use one of the other visual or WYSIWYG (what you see is what you get) editing tools, so pay close attention!

Your <TITLE> tag plays a crucial role in determining your rank in many of the spider-driven search engines. Normally, when writing HTML, the <TITLE> tag is placed immediately after the <HEAD> tag and is followed by assorted <META> tags.

However, if you use Microsoft FrontPage or another visual editing tool, the software writes the HTML and handles all those tedious tags for you. Unfortunately, FrontPage 97 and even

FrontPage 98 insert the <TITLE> tag AFTER the <META> tags like this:

<html>
<head> <META http-equiv= » Content-Type » content= » text/ html; charset= iso-8859-1″>
<META name= » GENERATOR » content= » Microsoft FrontPage 3.0″>
<META name= » Microsoft Theme » content= » global 101, default »>
<META name= » Microsoft Border » content= » tl, default »>
<title> My test title</ title>
</ head>
</ html>

Although this is technically correct and won’t interfere with someone viewing your Web site, it gains you no points with the search engines and could hurt your Web site’s ranking. We have been advised that some search engines expect the <TITLE> tag to immediately follow the <HEAD> tag. If it doesn’t, the search engine may determine that your Web site doesn’t have a <TITLE> tag and this would hurt your ranking considerably.

Additionally, as a general rule of thumb, you want your keywords to appear as close to the top of your page as possible. The standard <META> tag is not supported by at least half of the major search engines, but the <TITLE> tag is almost universally recognized. For this reason, having four lines of META tags proceed your title tag can make it easy for your competitors to outrank
you.

FrontPage 98 allows you to modify the HTML code simply by clicking on the HTML tab at the bottom of the screen. To correct the problem in the above example, you would simply move the

<TITLE> line up as follows:
<html>
<head> <title> My test title</ title>

<META http-equiv= » Content-Type » content= » text/ html; charset= iso-8859-1″> <META name= » GENERATOR » content= » Microsoft FrontPage 3.0″>
<META name= » Microsoft Theme » content= » global 101, default »> <META name= » Microsoft Border » content= » tl, default »>
<title> My test title</ title> </ head>

</ html>

If you have an editor that does not allow you to edit HTML directly, you should consider getting an HTML editor such as HomeSite or Luckman Interactive’s WebEdit: http:// www. allaire. com (HomeSite) http:// www. luckman. com (WebEdit)

The referrer log also tells you the address of the Web site (other than a search engine) the visitor came from. In all likelihood, the site listed in the referrer log has a link to your Web site on one of the pages indicated in the log. This can give you an idea which links from which kinds of sites are working.

Most Web servers can provide referrer logs, but you may have to ask your Web hosting provider to set this up for you. If they won’t, or if they claim they can’t, change hosting providers immediately. This information is too valuable to do without.

The referrer log includes information about the type of browser the visitor was using and whether or not they downloaded the whole page or only a part of it (which would suggest that it took too long and they moved on).

Information contained in the referrer log can shed light on how people are searching and finding your Web site. For instance, we were once under the mistaken impression that very few people on the Web searched with multiple keywords. We believed that the vast majority of people surfing the Web searched on one keyword at a time and chose a Web site to visit from those results. To our surprise, after reviewing referrer logs we discovered that most people visiting one of our client’s sites were searching on multiple words.

Referrer log data are often retrieved by logging onto the Web hosting provider’s server with an FTP client like WS_ FTP_ 95. The log file is usually stored in a file on the hosting provider’s server and can be quickly downloaded as a text file to your local hard drive.
There are myriad log analysis programs on the market that will take the volumes of data contained in the log and help you to easily understand it graphically.

However, if you don’t have such a tool, you can simply open the log file within a text editor or in Microsoft Word (or other word processor). The log file contains strings of data that at first glance look rather difficult to understand, but if you look closely, there are only about eight entries and they are easy to interpret.

When referrer log information is captured, here’s an example of what you will see:
198.178.25.131 –[07/ Jan/ 1998: 14: 09: 45 -0800] « GET /onepixel. gif HTTP/ 1.0″ 200 43 « http:// www. preowned. com/ » « Mozilla/ 2.0 (compatible; MSIE 3.0; Windows 95) »

Here is what each part of this log file means. Some referrer logs will contain a few more pieces of information, some a few less. These are what I consider the most important entries:

Referrer Log Entry Meaning

198.178.25.131 The remote host name: the IP address that identifies the location of the visitor’s Internet Service Provider.

[07/ Jan/ 1998: 14: 09: 45 -0800] The date and time of the request or access of
the Web site.

GET /onepixel. gif The actual request – the name of the first file downloaded by the browser. A search engine’s spider will almost always request the « robots. txt » file. You can review the section on robots and spider blocking to learn more about search spider’s activities.
200 The status code of the request. (Status code
« 200″ means a completed request).

43 The number of bytes that were transferred to the browser. In this case, the first graphic downloaded was 43 bytes.

http:// www. preowned. com/ The referrer site, or the previous URL that the visitor came from. This is gold to Web marketers. This tells you which search engine or other Web site sent you this traffic. When this entry contains a search engine name, it will also include the keywords that were searched for. See an example of this below.

« Mozilla/ 2.0 (compatible; MSIE 3.0; Windows 95) » The user agent or browser information – in this case the visitor was using Internet Explorer v3.0. Keep an eye on this entry as it will tell you if visitors to your Web site are using current technology. If, for instance, you learn that a large percentage of your visitors are using older browsers you should make sure that you aren’t using frames or tables as some older browsers don’t support them.

7 See, this log is not so hard to read. Now, look at another log entry:
208.207.98.111 –[08/ Jan/ 1998: 18: 19: 11 -0800] « GET /home. html HTTP/ 1.0″ 200 4723
« http:// www. altavista. digital. com/ cgi-bin/ query? pg= aq& text= yes& d0= 1% 2fjan %2f98& q= ballroom+ dance% 2a +OR+ ballroom dancing% 2a& stq= 30″ « Mozilla/ 2.0 (compatible; MSIE 3.0; SK; Windows 95) »

Note that the referring Web site is the AltaVista search engine. Pay special attention to this section:

%2f98& q= ballroom+ dance% 2a +OR+ ballroom dancing% 2a& stq= 30″ When you remove the code between the percentage signs you are left with what the visitor queried in the AltaVista search engine: ballroom+ dance OR ballroom dancing
A fast and easy way to search through a large log file for keywords that were searched on in search engines is to use the « find » command in your text editor or word processor and search the document for « cgi » as a keyword. Most of these will be search engines. You can also search on the individual search engine names using the « find » command.

We don’t yet recommend a particular log file analysis software program, but you can go to any major search engine and search for the keywords « referrer log software » and see what you get. Hopefully, they are using this book too and have optimized their pages to rank well for such a search.

For instance, maybe you constructed an elaborate direct marketing site that requires the visitor to enter through your main page and then proceed through a highly structured series of links that lead them to a buying decision. The internal pages would only confuse visitors who entered through those pages and they would be less likely to buy a product or service.

Whatever your reason, there is a standard that you can implement that will keep most of the major search engine spiders from indexing your Web site.

Here’s how to block the spiders. Create a file called « robots. txt » that includes the following code:

user-agent: * Disallow: /*

The first line specifies the agents, browsers or spiders that should read this file and adhere to the instructions in the following lines of code. The second line stipulates which files or directories the spider or browser should not read or index. The example above uses the « /* » which means the agent should not read or index anything as the asterisks denotes « everything. »

The robots. txt file must be placed in the root directory of your Web site. What this means is that if you are hosting your Web site using one of the free services and your domain looks something like this:

http:// members. aol. com/ Joesmith/ home. htm you cannot use the robots. txt file to keep out the spiders, since you don’t have a primary domain name. The primary domain name is aol. com – and America Online will probably not allow you to block all the search engines spiders from indexing their site and the Web sites of the 11 million other subscribers.

This robots. txt file could look like this if there were specific directories and files that you wish the search engines not to index:

user-agent: * Disallow: /clients/*
Disallow: /products/* Disallow: /pressrelations/*
Disallow: /surveys/ survey. htm

In the above example the robots. txt file asks the search engines spider to omit all pages within the following directories:

http:// www. yourcompany. com/ clients/ http:// www. yourcompany. com/ products/
http:// www. yourcompany. com/ pressrelations/

And the following specific page:

http:// www. yourcompany. com/ survey/ survey. htm
If you are one of the millions of people hosting a Web site on America Online’s server or one of the other free or subdirectory Web site services and you can’t place a robots. txt file in their root directory, you can use a META tag that talks to some of the spiders:
<META NAME= » ROBOTS » CONTENT= » NOINDEX »>

You will need this META tag on every page in your Web site that you don’t want indexed. If your Web site has 30 or 40 pages (or more), this will take a lot of time. Here’s another reason to buy a good HTML editor like Luckman’s WebEdit or Allaire’s HomeSite. These programs allow you to do a global search and replace and add an HTML tag to every Web page that you open in the program. As with all META tags, this META tag goes at the top of your HTML document between the <HEAD> and </ HEAD> tags.