Spider

301 Redirect

Also known as a permanent redirect, an instruction to Web browsers to display a different URL from the one the browser requested, used when a page has undergone a lasting change in its URL. A permanent redirect is a type of server-side redirect that is handled properly by search engine spiders.

302 Redirect

Also known as a temporary redirect. An instruction to Web browsers to display a different URL from the one the browser requested, used when a page has undergone a short-term change in its URL. A temporary redirect is a type of server-side redirect that is handled properly by search engine spiders.

Agent

A piece of software, such as a browser or spider, that interprets the content on a web server and presents it to the user as a web page. Examples include Internet Explorer, Opera, Netscape and various search engine spiders.
Examples: MS Internet Explorer, Netscape Navigator, Googlebot, Slurp, T-Rex

Agent name delivery

The act of presenting one set of content to search engine spiders based on the name of that spider and another set of content to human web users. This is done to present content that has been specifically optimized to rank well at each search engine while still presenting the same content to each human visitor to the web site.
This technology is easily detected as web surfers are able to use an agent name faking program to appear as if they are the named spider and view the cloaked content.

Applet

A small program, often written in Java, which usually runs in a web browser, as part of a web page. It is possible that the use of such a program may cause spiders and robots to stop indexing a page.

Bot

Abbreviation for robot (also called a spider). It refers to software programs that scan the web. Bots vary in purpose from indexing web pages for search engines to harvesting e-mail addresses for spammers.

Cloaking

Serving a different page to a search engine spider than what a human visitor sees. This technique is abused by spammers for keyword stuffing. Cloaking is a violation of the guidelines of most search engines and could be grounds for banning

Crawler

Also called bots or spiders; programs that follows links to visit web sites on behalf of search engines. Crawlers then process and index the code and content of a web page according to an algorithm and store the pages in the search engine's database. Googlebot is the crawler that travels the web finding and indexing pages for the Google search engine.

Flash

A technology invented by Macromedia that brings a far richer user experience to the Web than drab old HTML, allowing animation and other interactive features that spice up visual tours and demonstrations. search engine spiders are unable to index content in the flash programs, and may not index the web page containing heavy flash programming.

Googlebot

Googlebot is the Google's spider or crawler or bot; it travels the web finding and indexing pages for the Google search engine. Googlebot leaves it's identity on your web server's log file. Webmasters should study their server log files closely to see the googlebot's visit is successful and is able to crawl the whole site.

Hidden Links

A spam technique in which hypertext links are written to be seen by spiders hut not by human visitors. Spammers place lots of links from high-ranking pages to other pages they are trying to boost.

Indexed Pages

Pages that are included in a search engine's index. An important step in search engine optimisation in insuring your web sites pages are included, indexed, in the search engine databases. Search engines index pages through the spiders or robots.

IP Delivery

A technology by which a webmaster can deliver customized advertising based upon the IP address of the visitor. It allows a site to serve local language or content to specific visitors.

Is the technology used in cloaking, to identify search engine spiders from their IP address and route them to different version of the requested page.

Can be used ethically to route spiders around dynamic webpages which require a database query, to simpler pages that spiders can process. All the while avoiding duplicating content.

Javascript

A programming language that can provide special effects inside a browser that cannot be performed in HTML. Search marketing success depends on certain standards about how and when JavaScript programming is used, because JavaScript, when misused, cat, prevent search spiders from indexing certain pages.

Optimisation (optimization) Services

Service offered to a web site aimed at improving the organic search rankings (not paid search) for a set of keywords. Search engine optimization involves making the pages of a site more easily accessible to search engines spiders and clarifying the keywords relating to a web pages. Search engine optimisation is also often referred to as SEO, search engine positioning and search engine promotion.

Paid Inclusion

A service offered by some search engines, such as Yahoo!'s premium service, that guarantees a Web site’s pages are stored in the search index in return for a fee. Paid inclusion does not guarantee high search rankings for those pages, just that the pages are included in the index and that the spider will frequently revisit the web site to keep the index up to date.

Permanent Redirect

Also known as a 301 redirect, an instruction to Web browsers to display a different URL than the one the browser requested, used when a page has undergone a lasting change in its URL. A permanent redirect is a type of server-side redirect that is handled properly by search engine spiders.

Promotional Domains

An alternative domains for a companies web site. They are secondary domains, but are integrated with the main domain so the spiders visit them. Promotional domains are often built if the primary domain has problems with the engines. Used in cloaking and smamming.

Robots.txt

Robots.txt is a file on a web sites root directory which spiders are supposed to read to determine which parts of a website they may or may not visit.

Robot

Also known as spider, bot or crawler. The part of a search engine that locates and indexes every page on the Web. Successful search engine optimisation depends on robots finding many or all a Web site's pages.

Search Engine

A searchable index of web sites that is traditionally compiled by a spider that visits web pages and stores the information from each page in a database.

Search Engine Optimisation (SEO)

The work of improving organic search rankings (not paid search) for a Web site and a set of keywords. Search engine optimization involves making the pages of a site more easily accessible to search engines spiders and clarifying the keywords relating to a web pages. Search engine optimization is also often referred to as SEO, search engine positioning and search engine promotion.

Slurp

Inktomi's search engine spider.

Spamming

Any search engine optimisation method that a search engine deems to be detrimental to its efforts to deliver relevant, quality search results. No firm definition are given and the search engines reserve the uni-lateral right to take action, for instance banning, against any web site

Some search engines have written guidelines on spamming, but ultimately any activity deemed harmful may be considered spam, whether or not there are published guidelines against it.

Example of spam include the creation of nonsensical doorway pages designed to please search engine algorithms rather than human visitors or heavy repetition of search terms on a page. Any form of hidden text meant for spiders.

Determining what is spam is complicated by the fact that different search engines have different standards. A particular search engine may even have different standards of what's allowed, depending on whether content is gathered through organic methods versus paid inclusion. Also referred to as spamdexing.

Ethical marketing is strongly recommended for long term marketing strategies.

Spider

Also known as a crawler, the part of a search engine that locates and indexes every page on the Web that is a possible answer to a searcher’s query. Successful search engine optimisation depends on crawlers finding many or all a Web site's pages .

Temporary Redirects

Also known as a 302 redirect, an instruction to Web browsers to display a different URL from the one the browser requested, used when a page has undergone a short-term change in its URL. A temporary redirect is a type of server-side redirect that is handled properly by search engine spiders.

Trusted Feed

A means of sending your data to a search engine, instead of having the spider crawl your site. Some niche search services and almost all shopping directories and search engines require the use of trusted feeds to load your data into their search index.

URL Rewrite

A web server technique which changes on the fly, so that dynamic URLs look like static URLs. URL rewrite gives you a readable URL for your human , but are also key to getting spiders to crawl your site.


Generate your Google Sitemap for free!

Once you have visited through your site (or waited till your normal traffic has browsed all your site), complete the form and our sitemap generator will produce the sitemap.xml file. Enter the full address, including the http, to set the level of from which your sitemap is based, e.g. "http://www.thesite.com/" or "http://www.thesite.com/home/"

Top Level URL: (must end in "/")


Lastmod: (blank to leave out)
 Priority
 Changefreq:

 




yourgooglesitemap.com


Home | Contact | Glossary


2004,2005 © yourgooglesitemap.com - Generated 2.918.189 Google Sitemap pages