Optimization of Web Pages - spidering, spam, submissions, meta tags
FreeBonanza
 Find Free Stuff
Search Engine Secrets - discussed are spiders/robots spidering, spam, spamming search engines, submissions, meta tags, title, description, keywords, imbedded keywords, comment tags, hidden tags, alt tags, gifs.

Welcome to Search Engine Optimization

This page contains all the material discussed in search engine secrets on one page. To read it in smaller bits Go Here. Technically the contents herein are not really "secrets" to seasoned webophiles, but may be helpful to others.

-What is spidering? - Search engine spidering is briefly examined and considered.

-Do not SPAM search engines - What spamming the search engines is and how to avoid doing it.

-How many pages should I submit and how often? - Tips and information related to submitting web pages to search engines.

-What are META TAGS? - A simple definition of meta tags and how search engine robots use them.

-The Meta Tags Themselves: Title, Description, Keywords - More detailed insight into meta tags touching on their three major elements.

-The TITLE Tag - The title tag is examined and considered with suggestions for their proper use.

-The DESCRIPTION Tag - The "title" tag is examined and considered with suggestions for their proper use.

-The KEYWORDS Tag - The "keywords" tag is examined and considered with suggestions for their proper use.

-Imbedded Keywords - Imbedded keywords are explained.

-Comment Tags - A note about comment tags and how to use them.

-Hidden Tags - A note about hidden tags and warnings about invisible text.

-"ALT" Tags and Gif Names - The benefits of using "alt" tags with gifs are considered, also contains notes on naming gifs.

All Copyrightable material on this site was created by Ken J Wagner, at57.com
C.2001 All Rights Reserved.

What is Spidering?


A search engine robot is called a "spider." The reason being that when the robot/spider goes to a page, it follows all the links on that page. In the imagination this action resembles a many legged spider. The robot follows the links to see if the pages go where they say they go. To know what these pages are the spider reads their meta tags. Robots normally always follow down at least one level when they index a page, and many spiders return later to do a "deep crawl" and index every page they find. For this reason if you submit one page of a site to a search engine that crawls, eventually every single page will be indexed. If you ever want a page not to be indexed you can add a line in the HEAD tags (placed between arrows, of course) like this: META NAME="robots" CONTENT="noindex,nofollow" - you can also use variations: index,nofollow or noindex,follow - depending on what you want the spider to do. The default for no tag is that the robot will index and follow the page. A"robots.txt" file can also be used for a domain, a subject too broad to go into here, and you don't need to know about it if you have a free site since you can't use it anyway (it only works for domains).

Do not SPAM search engines


What is SPAM? Spam is most closely associated with sending unwanted, unsolicited emails to unwitting recipients, but it is also considered SPAM to post unwanted ad's to people on message boards, in chat rooms, and on news groups. Search engine SPAM is when search engines consider you to be flooding their data bases by submitting too many pages too often or by submitting the same pages again and again. Other forms of SPAM include pages that contain meta tags that have nothing to do with the actual page, and submitting the same exact page from different addresses.

How many pages should I submit and how often?


Never ever submit the same page twice in a 24-hour period; in fact, it is best not to submit any single page more often that every 30 days, and ideally a page doesn't really need to be submitted more often than every 90 days. Search engines that "crawl" will eventually index every page of your site (so you don't need to submit every page). This process takes anywhere from 2-6 months. For this reason you should only submit the MOST IMPORTANT pages of your site. Don't submit more than two different pages in a 24-hour period; and it is preferable to only submit one page at a time allowing 2-3 days to pass between submissions. This approach offers you the benefit of spreading out your spidering. You need to remember that when you submit a page all the pages it is linked to will also be indexed again. If you have 4 main pages, you can submit 1 a week and eventually all of your pages will be indexed once a week. Pages are also ranked higher when they are crawled rather than submitted, so you want to leave some "gem" pages for the spider to find. If you have more than one site, be sure they are linked together so that when one gets crawled they all do.

What are META TAGS?


Meta tags are tags that appear in HTML at the top of each web page; these tags tell the spiders what's on the page in a fashion that is easy for them to index. The robot reads these tags and compares them to the content on the page. The better the tags add up with the actual content of the page the higher the spider will rank the page's TITLE, DESCRIPTION, and KEYWORDS. When the spider finds material in the tags that does not reflect the actual content of the page, it will give the page a low ranking or will not index the page at all.

The Meta Tags Themselves - TITLE, DESCRIPTION, KEYWORDS


To see meta tags a visitor can while viewing most web pages, select "view" on their browser tool bar, then select "source," where they will be able to look at all the HTML of that page unless it is protected or hidden by a frame. The most important part of the meta tags are the Title, Description, and Keyword tags. However, the HEAD /HEAD tags are important because they tell the spider that it is the area where the tags exist. Also, there must be a BODY tag after the /HEAD tag or the spider can not read the tags at all.

The TITLE Tag - TITLE your title goes between these two tags /TITLE


The Title tag should be between 4 and 60 characters long, or thereabout, and preferably over 30. If the title isn't descriptive enough it may not receive a good ranking from the robots. Be sure to include keywords you want to rank well for in the title, because it is one of the most important ranking components of the page. "Mandy's Playhouse" may be a cool site name, but it's unlikely that anyone is searching the engines for those particular words. It's better to place in the title tag what is available at "Mandy's Playhouse;" for instance: "Games, Music, Chat, and Fun" - do not repeat words here or the engines will penalize your ranking for the repeated words. Then, whatever your title says is what you want it to say at the top of your visible page, preferably in H /H tags. This way the spider can read the title in the meta tag, compare it to the page, see that it adds up, and will rank the page better.

The DESCRIPTION Tag - META NAME="description" CONTENT="your description goes here"


This tag contains a short blurb describing the page contents. Once again it is vital to get in some keywords. Remember at the search engine level you are first talking to a robot, and secondly, only later (after the page is indexed) to humans. You need to try to create descriptions that will appeal to a spider ranking you in the first place and that will appeal to a reader potentially clicking through in the second place. There is some dispute between various sources about what length a Description tag should be, but apparently bettwen 110 and 150 characters works best. Shorter (110 characters) is probably better. Get your keywords in but do not repeat them and do not repeat the "Title" tag in your Description tag (it's a waste of quality words and will hurt your ranking). The description written in your Description tag ought to appear in some variation on your page, the closer to the top of the page, the better. Once again, the spider will compare the tag to the content and see that they add up.

The KEYWORDS Tag - META NAME="keywords" CONTENT="your keywords go here"


Robots are beginning to ignore this tag because so many people add words here that do not relate even remotely to their page. A form of SPAM is repeat keywords here again and again in hopes of ranking high for them; but the spiders are onto this and pages with words overly repeated in this tag are penalized by being ranked low or being disqaulifed altogether and not being indexed. Various sources have various angles that dispute one another on the subject of how many times a keyword can be repeated. Some say a keyword should be repeated 7 times, some say no more than 3 times, and some say no word should not be repeated at all (meaning place the word in the tag once and once only). In light of this controversy I suggest not normally using a word more than once unless it is unavoidable. Keywords work the same as words in the Title and Description tags. 90% (better yet 100%) of the keywords listing in your tag should appear on your page, and not in a "list" and not overly repeated on the page; they should be used relevantly in real sentences that actually matter to the overall content of the page. Characters in the Keyword tag should not exceed 800 or so and less is better. Many spiders ony read the first 7 keywords anyhow and ignore the rest; for this reason place your most important keywords first. Do not use commas; they are not needed and they separate words that could be better used in "key phrases." Key phrases are two or more words used together that people might potentially use in a search. For instance someone might search for "new cars," instead of just "cars." If you use commas in your keyword tag then you'll have to repeat words to get phrases: "new" , "new cars" , and "cars." Without the commas you simply list "new" before "cars" and it becomes all three potential search inputs. Do not include a lower case and upper case version of keywords; most spiders are not case sensitive, and most searches are conducted in small letters, even when it's a proper name. Note that some engines will not list sites that do not have commas between the keywords, so it's a catch 22. Sometimes I use them, sometimes I don't.

Imbedded Keywords


Imbedded keywords are keywords that appear in the addresses of your pages, including the URL itself. An example of an imbedded URL is mp3.com. This domain name provides the imbedded keyword "mp3," which mp3.com then ranks higher for. This is why it is vital to properly name your site, your pages, and your page addresses. When a robot follows a link it reads the text of the link on the page it's following, it reads the address of the page it's going to, and reads that page. If everything adds up the robot ranks the page well. Imbedded keywords in the name of your site are not all that is important. The addresses of your pages should reflect the content of those pages. Let us consider a hypothetical web site; we'll call it flowers.at57.com. Automatically the site has a better ranking for the keyword "flowers." Now the links on the page might be "roses" , "tulips" , and "marigolds." Here's what the optimum addresses should look like: flowers.at57.com/roses.html, flowers.at57.com/tulips.html, and flowers.at57.com/marigolds.html. Carefully consider all your addresses before creating them, and do not abbreviate - abbreviations are meaningless to a robot.

!-- Comment Tags


Comment tags are tags that can be added to the HEAD section with meta tags. Go to "view" on your browser tool bar and select "source" to read the comment tags I've added to this page. The proper format (and I've noticed that a lot of people blow this) is a left arrow, an exclamation point, two hyphens, a space, then text, like this: !-- text goes here. It closes after the text with two more hyphens and a right arrow. Inside the arrows you can write comments telling what your site's about; be sure to include keywords. How effective this is, is hard to say. From what I understand the only engine that definitely reads this tag is Hotbot; however, it is likely that the presence of this text may help to reinforce the validity of your tags and page for other robots. Do not overdo it with comment tags. These tags can also be used in the BODY of your page to point out what exists there when a spider can't read it, like a java search box or script. doing this also helps you to locate whatever it is follows the tag in the HTML (don't overdo it with these either).

Hidden Tags - input type="hidden" value="put keywords here"


It is best to use as little "hidden" data as possible and never ever use "invisible text" (text that doesn't show up because it is the same color as the background of the page - engines react harshly to this SPAM tactic). At the top of the page directly following the BODY tag you can place (inside arrows) input type="hidden" value="put keywords here" - this may increase the validity of some of your keywords, but it messes up your page design by leaving a large gap at the top of the page where the hidden text appears; and it's questionable how effective it even is. I've included it here to help make you aware of it, but I don't recommend it, and some engines may even penalize you for it....

"ALT" Tags and Gif Names - ALT="describe image here"


Some spiders read "alt" tags. These are the tags that tell what a .gif or .jpg is - without the "alt" tag a robot crawling your page has no clue what the image is; so, you explain to the spider what the image is by creating this tag. It can be placed in various places in the HTML code, between "IMG" and "SRC" seems to work well for me. It's good to always add BORDER="0" if it's an HREF or else an unsightly border will surround everything inside the anchor tag (A /A). This too helps increase the validity of your page. Also when you name gifs and jpegs, name them what they are, examples: roses.gif, tulips.gif, marigolds.gif. This potentially increases the validity of your page because the textual words recurr, and frankly, it makes managing your files much easier.

What is Link Popularity?


Link popularity relates to the number of links pointed to a web site. Over the years apparently the number of links pointing to a site has become of more prominent importance in terms of search engine positioning, ie: websites with lots of sites linking to them rank better in searches. Some writers on the subject even claim that search engines take into account the "relevance" of the links, ie: sites linked to by similar sites rate better. Search engines are secretive about their exact practices and it's hard to tell what's true. One thing is certain, though link poularity is touted as a newly emergent factor on the scene, it actually has been significant since the beginning because of the "crawl" factor. Regardless of whether or not most search engine software is sophisticated enough to actually meter link popularity as a crucial factor, and especially whether or not it is sophisticated enough to guage themed links, the more links a site has pointing to it, the more often it is crawled by spiders, and the more often it is indexed. Therefore, it is crucial for a site to have at least a few external links pointing to. This is more true than ever now that some engines are even dropping sites that don't hae at least one external link, and this could possibly become a trend with search engines to help conserve database space.

If you have more than one site, be sure they are pointed at each other to increase your link popularity. In fact you want to get a link everywhere you can: directories, classifieds, link exchanges, etc. Note that a link from a search engine like Alta Vista or an index like Yahoo don't count as link-backs because they are stored in a cgi-bin database, which is disallowed in the robots.txt. The best place to get link-backs are on directories that publish pages in the public areas of their site. When you go to a directory go down a level to look at the site listings and note the full URL address. Does the last part say, "cgi-bin/," or does it say something like, "home/internet/submissions.html"? The former is in a potentially disallowed cgi-bin. The latter is in the public docs area, and that's where you want your links. I've set up a link exchange directory with just this in mind. If you'd like to trade a link and get a solid link-back Check it out (opens in a new window).

All Copyrightable material on this site was created by Ken J Wagner, at57.com
© 2001 All Rights Reserved.





Search AT57.com



Other Stuff