|
Unfortunately, this is an all too common question. If it
makes you feel any better, you're not the only one frustrated
about the length of time it takes to be indexed, or the many
pitfalls involved. It often takes anywhere from two days to
as much as six months to be listed on a search engine. For
example, last month Excite finally updated its index for the
first time since last August! Luckily, Excite is the most
extreme case lately, but waiting several weeks to a month
can also be extremely frustrating.
The WebPosition Submitter report will give you current time
estimates for each engine so you'll know what to expect. However,
an engine at any time could choose to delay their indexing
beyond the "norm" for maintenance or other reasons.
On the flip side, you could get lucky and submit just a couple
days before an engine does a complete refresh of their database.
Therefore, submission times can never be an exact science
since we're all ultimately at the mercy of the engine.
If you've submitted your site and have waited the estimated
time to be indexed and there's still no listing, what do you
do now?
The net's best multimedia
video screen capture and production tool!
Download it direct...
Here are 16 tips that should help you solve this problem:
1. First, be sure you're not already indexed but just
don't know it. Unfortunately, none of the major engines are
kind enough to e-mail or notify you as to if and when you've
been indexed.
The method to determine if a page or domain has been indexed
varies from one engine to another, and in many cases, it's
difficult to tell for sure. Never assume that you're not indexed
just because you searched for a bunch of keywords and you
never came up in the first few pages of results. You could
be in there but buried near the bottom.
In addition, it's not very practical to check the status
of a number of pages on each major engine each week. Fortunately,
WebPosition has a URL verification feature in the Reporter
that makes this process much easier. Each time you run a mission,
it will report which URLs exist and do not exist in each engine.
If you're using WebPosition and are not finding your URLs
after submitting, be sure to see this page for common pitfalls
to watch out for:
http://www.webposition.com/urlnotfoundhelp.htm
2. Make sure you have uploaded the pages to your site
before submitting them. This one seems obvious, but submitting
a page that does not exist or submitting with a subtle typo
in the URL is a goof we might all make at one time or another.
If you're using WebPosition's Submitter, there's a checkbox
on tab 2 that forces WebPosition to verify that all your URLs
are valid before submitting them.
3. If you have information inside frames, that can
cause problems with submissions. It's best if you can create
non-framed versions of your pages. You should then submit
the non-frames versions of your pages which can of course
point to your framed Web site. Alternatively, you can enter
your relevant text within the NOFRAMES area of a framed page
which most search engine spiders will read.
4. Search engine spiders cannot index sites that require
any kind of registration or password. A spider cannot fill
out a form of any kind. The same rule applies regarding indexing
of content from a searchable database, because the spider
cannot fill out a form to query that database. The solution
is to create static pages that the engines will be able to
find.
5. Dynamic pages often block spiders. In fact, any
URL containing special symbols like a question mark (?) or
an ampersand (&) will be ignored by many engines.
6. Most engines cannot index text that is embedded
in graphics. Text that appears in multimedia files (audio
and video) cannot be indexed by most engines. Information
that is generated by Java applets or in XML coding cannot
be indexed by most engines.
7. If your site has a slow connection or the pages
are very complex and take a long time to load, it might time
out before the spider can index all the text. For the benefit
of your visitors and the search engines, limit your page size
to less than 60K. In fact, most Webmasters recommend that
your page size plus the size of all your graphics should not
exceed 50K-70K. If it does, many people on dial up connections
will leave before the page fully loads.
8. If you submit just your home page, don't expect
a search engine to travel more than one or two links away
from the home page or the page that you submitted. Over time
they may venture deeper into your site, but don't count on
it. You'll often need to submit pages individually that appear
further down into your site or have no link from the home
page.
9. If your Web site fails to respond when the search
engine spider pays a visit, you will not be indexed. Even
worse, if you are indexed and they pay a visit when your site
is down, you'll often be removed from their database! Therefore,
it pays to have a reliable hosting service that is up 99.5%
of the time. However, at some point a spider is going to hit
that other 0.5% and end up yanking your pages by mistake.
Therefore, it pays to keep a close eye on your listings.
10. If you have ever used any questionable techniques
that might be considered an overt attempt at spamming (i.e.,
excessive repetition of keywords, same color text as background,
or other things that the WebPosition Page Critic warns you
about), an engine may ignore or reject your submissions. If
you're having trouble getting indexed in the expected amount
of time, make sure your site is spam-free.
11. If your site contains redirects or meta refresh
tags these things can sometimes cause the engines to have
trouble indexing your site. Generally they will index the
page that it is redirecting TO, but if it thinks you are trying
to "trick" the engine by using "cloaking"
or IP redirection technology, there's a chance that it may
not index the site at all.
12. If you're submitting to a directory site like
Yahoo, Open Directory, NBCi, Looksmart, or others, then a
human being will review your site. They must decide the site
is of sufficient "quality" before they will list
it. I recommend you read the submission guide on the directory
tab of the WebPosition Submitter. It contains tips to improve
your chances of obtaining a good listing on these directories.
13. A number of engines no longer index pages residing
on many common free web hosting services. The common complaint
from the engines is that they get too many "junk"
or low-quality submissions from free web site domains. Therefore,
they often choose not to index anyone from those domains or
they limit submissions from them. It's always best to buy
your own domain name (very important) and place it on a respected,
paid hosting service to avoid being discriminated against.
14. Some engines have been known to drop pages that
cannot be traveled to from the home page. HotBot has been
rumored to do this. You may want to consider submitting your
home page that links either directly or indirectly to your
doorway pages.
15. Make sure you're submitting within the recommended
limits. Some engines do not like more than a certain number
of submissions per day for the same domain. If you exceed
the limit, you may find that all your submissions are ignored.
Fortunately, WebPosition's submitter will warn you regarding
current limits and recommend you stay within them. Some submission
consultants feel it is dangerous to submit more than ONE page
a day to a engine for a given Web site. For those who wish
to be ultra-conservative in their approach, the WebPosition
Submitter includes a checkbox to limit submissions to one
URL per day per engine.
16. Last but not least, sometimes the engines just
lose submissions at random through technical errors and bugs.
Therefore, some people like to resubmit once or twice a month
for good merit in case they do lose a submission. Certainly
if you've followed all the "rules" and are still
not listed, re-submit! Sometimes a little persistence is all
that's needed.
If any of the above scenarios apply to your submission, you
should make the necessary adjustments and re-submit. If that
still does not work, you should consider e-mailing or calling
the search engine and asking them politely why you have not
been indexed yet. Sometimes they will reply back with "Sorry,
there was a problem with our system and I've now made sure
you'll be indexed within the next couple days." Or, sometimes
they'll tell you why you were not indexed. In other cases,
they will ignore your e-mail and you'll have to keep e-mailing
or calling them until they respond. Still, it's definitely
worth the effort to get your site listed with the major engines
assuming you also take the time to optimize your pages so
you'll achieve top rankings.
The above article, or portions of it, have been reprinted
with
permission from the MarketPosition Newsletter and FirstPlace
Software, Inc. and is copyright 1997-2001. FirstPlace
produces
WebPosition Gold, the award-winning software product to track
and to
improve your search engine rankings. You may download a FREE
trial
copy of WebPosition Gold from:
http://www.webposition.com
|
