Oh no, whats going on? You casually visit your web site and are faced with the dreaded server reaponse message..
And now you want to know what it means.
Web server responses to user and search engine requests are a critical aspect of search engine optimization and Web site usability. Relay the wrong response, and you could create a world of hurt for your Web site in the search engines, or have some dissatisfied users on your hands.
Server Response Codes
Following is a definition of the basic server response codes.
200 — OK. The request has succeeded. The vast majority of all requests should generally end with this response. In layman’s terms, this means the request was understood, accepted, and the page you requested will appear.
301 — Moved Permanently. The requested resource has been assigned a new permanent URL and any future references to this resource should use one of the returned URLs. Clients with link editing capabilities ought to change the existing URL to the new URL if it happens to rank for a given keyword. If the URL does not rank, and if the current URL structure has many dynamic parameters, it may be wise to rewrite the URLS (using either ISAPI rewrite or Mod_Rewrite) to include keywords within the URL string. When undertaking any rewrites of URLs, you should use 301 redirects to ensure that the value of the existing URL is passed along to the new URL. In summation, this means the document has permanently moved to a new location, and search engines should reference the new location.
302 – Found. The requested resource resides temporarily under a different URL. Since the redirection might be altered on occasion, the client should continue to use the old URL for future requests. This way, the search engines cache the old URL and not the new one. This keeps any current rankings that this page may have until you decide where you might want to permanently redirect this page. This is commonly referred to as “temporarily redirected,” which means the document has moved to this location for a while. Search engines don’t forget about the old location.
404 – Not Found. The server has not found anything matching the Request-URL. No indication is given indicating whether the condition is temporary or permanent. The 410 (Gone) status code should be used if the server knows that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable. See the common pitfalls below on how the misuse of these redirects can impact SEO efforts.
For more information and definitions on server response, check out the status code definitions on the W3.org site.
Common Pitfalls
Below are some of the common pitfalls in using server response codes, as this relates to search engine optimization:
- Using 302 temporary redirects when the change is permanent. When you’ve changed the URL of a document and have no plans to change it back, you should use a 301. Using a 301 redirect will help you pass along any value/rankings that the legacy URL had. That being said, some search engines do better than others in following the 301 redirect. If your experience is anything like ours, you will find that Google does a fine job of associating the two, though Yahoo! will take a while to fully value change.
- Custom 404 pages resolving to a 200 OK server response. When a page can’t be found on some content management systems, it should provide a custom 404 page letting the user know that the page no longer exists and pointing out main sections of the site. While this is great for the user experience, serving these pages incorrectly can cause a bunch of these “page not found” pages to be indexed in the search engines. By ensuring that these pages are properly referenced as 404s, you can properly redirect the visitor to a relevant page and ensure that there are not any issues of “duplicate content” (pages for which the only text on the site says “page not found”).
- Sometimes, the search engines aren’t as good as advertised in following redirects. As recently as a few months ago, MSN Live was indexing 301 redirects as blank pages, and examples of 302 hijacking seem to make a comeback every now and then. We have witnessed very quick responses to 301s from Google and are still hoping to fully recover from some 301 redirects with Yahoo! after several months. MSN is hit or miss, depending on what they’re doing in any given month. Bottom line: do 301 redirects when you have to and consult with an SEO professional if you believe there could be any chance that you could lose rankings as a result of any URL changes.
- Canonical Issues. Your first response to this may be, what the heck is a canonical issue? Here’s the thing. You should reference your site — in particular your home page — in one way. You’ll notice that https://www.ripefruit.com/ will redirect if you try www.ripefruit.com to the version without the “www.” Many sites link to their home page internally as, www.sitename.com/index.html. Each instance of a site resolving to a given URL is a different address. To get the inside scoop on canonical issues, I would suggest that you read Matt Cutt’s blog.
Replace ripefruit with your domain and add this code to .htaccess
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.ripefruit.com$ [NC]
RewriteRule ^(.*)$ https://www.ripefruit.com/$1 [L,R=301]
Domain name redirection
Why redirect a domain name?
When the hosting of a domain is changed (moving site), it is a simple matter of pointing the domain to the new server by changing DNS. When a new domain name must replace an old domain, then would be a problem with search engines. Two domains leading to one site means duplicate content and duplicate content for search engines is unacceptable leading to both your domains being penalised or, even worse, banned from a search engine listings.
To avoid duplicate content penalty, you need to “tell” search engine spiders that your domain has moved to a new domain, (i.e. use redirect from old domain to new).
The best method is to use a permanent (301) redirection in a .htaccess file redirection – mod rewrite (apache server)
Case One: old domain to new domain – same hosting account
New domain to replace primary domain? You can’t do that. Primary (main) domain is connected with your hosting account, and all add-on and parked (alias) domains are connected with your main domain. If hosting is good, and you want to keep it, you can contact support, they can do that. So, old domain could be parked, and new domain set as primary domain (account moved).
Anyway, if you wish to try, you can set new domain as “main domain”, there are two ways:
Using parked domain as main domain
Parked domain (alias) is pointing to root (only if parked on your account), what means, it is the same as main domain. It could be used as “main domain” on your account, if traffic from your old domain is redirected (permanent redirect). It is not problem, but, you still need to keep your old domain, it is connected with your account, and if it is removed (or expired), all other domains (parked, ad-ons) will disappear. So, keep your primary (old) domain in good health.
In this case (alias as main domain), it is not neccessary to move any page from your site, but it is neccessary to remove any instance of old domain from page. If you are using relative linking (without full url), you don’t need to change anything. In this example below, we are redirecting our primary (old) domain to new (parked, alias) domain.
#Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www.olddomain.com$[OR]
RewriteCond %{HTTP_HOST} ^olddomain.com$
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]
Now, with this setup, every request for olddomain.com/page.html is permanently redirected (301) to newdomain.com/page.html, About htaccess, and how to add this code above, see on bottom.
There is one problem with this setup. If your stats or logs are on olddomain.com/stats/, you need to disable redirection (uploading old htaccess) at that moment. Due to redirection, your stats directory is “moved” to newdomain.com/stats/,and you will get login screen again.
Using Add-on domain as main domain
Difference from parked domain (alias) is that add-on domain is pointed to subdirectory of main (primary) domain, (i.e. addondomain.com is pointing to maindomain.com/addondomain/), while alias is pointing to root. In this case, we need to move (copy) all files to /addondomain/ directory of main domain and add redirection from root to /addondomain/.
Code is the same as above for parked page, just be sure that everything is working on add-on domain before old domain is redirected.
Anyway, you should ask your hosting support is this above possible. You can test redirection to check is it working, but not all servers are the same, so, first ask support.
Case 2 (best): Old domain to new domain – different accounts
This is the best way to change domain name. Set up new account, host new domain on that account, test if everything is working, and add redirection from old domain (account). Keep old domain and account for a few months, or for a year, leaving redirection on place.
When you are sure that new domain is indexed on search engines, and google PR updated for new domain, and there is no links to old domain, and redirected visitors, you can do with old domain and account what you wish.
Sure, htaccess code (directives) is the same as above.
Subdomain to new domain redirection
Say that you have popular site on subdomain, an wish to move to the new domain on other account. Just use code below.
#Options +FollowSymlinks
RewriteEngine On
RewriteCond %{HTTP_HOST} ^yoursite.hosting.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.yoursite.hosting.com$
RewriteRule (.*)$ http://www.yournewdomain.com/$1 [R=301,L]
That code could be used if you have account without domain (your account is connected with subdomain.hostingcompany.com), just add new domain to the same account, and use above redirect.
Non WWW to WWW version redirection
Believe or not, there are still sites you cannot reach without using www in domain name. That is server configuration “problem”, but what if you find that your site is indexed with two “domain names”, with www.yoursite.com and yoursite.com. For search engines, these are still two different sites with the same content. This happens when your link partners are using version without WWW. Search engine spiders are following those links, and in that case your site is indexed without WWW. Although you can check on YAHOO Site Explorer who is linked to your site and what version of domain name they are using, it is better to add redirection to WWW version. Working example, tested.
Uncomment first line if you have any problem.
#Options +FollowSymlinks
RewriteEngine On
RewriteCond %{http_host} ^yoursite.com
RewriteRule ^(.*) http://www.yoursite.com/$1 [R=301,L]
WWW to NON WWW version redirection
If you prefer non WWW version of domain name, code is below:
#Options +FollowSymlinks
RewriteEngine On
RewriteCond %{http_host} ^www.yoursite.com
RewriteRule ^(.*) http://yoursite.com/$1 [R=301,L]
To test it, just add www. in front of domain name in browser address bar, and you will be redirected to non WWW version.
One version of domain name will boost your Google PR.
Other examples of domain redirection
All pages from old domain to main page (root) of new domain:
redirect 301 / http://www.newdomain.com
More examples soon…
Problems?
Whatever example you wish to use, don’t forget two things. Htaccess is a text file, it must be uploaded in ASCII (text) mode, not binary. It will be good to keep it offline with .txt extension, to be sure that will be uploaded in ASCII mode. Don’t forget to rename it to .htaccess after upload. Second, always test it.
To avoid any interruption during testing, keep two versions of htaccess, i.e. htaccess_redir.txt and htaccess_original.txt. Upload htacces_redir.txt, rename it to .htaccess, and with opened FTP program, test domain name redirection in browser. If it is not working (error 500), upload original file back (htaccess_original.txt) or edit and try again.
Use some online header checker (or check in website log) to confirm that server response is 301 (moved permanently). AND, do not promote new domain (if content is the same) before redirection from old domain is on place.
Search engines and 301 redirection – example
In recent experiment (Feb 2007) main domain was redirected to alias domain (same account – IP). That site is small (30 pages), it was easy to track indexing under new domain. So, month later, here is situation:
Google – new domain indexed, old domain deindexed, rank lowered (domain is new) PR 0 (3 before)
Yahoo – new domain indexed, old domain – a few pages left, indexed under both domains, rank lowered
MSN – No sign of new domain, all pages still indexed under old domain, cache shows redirect (document moved here)
What about traffic from search engines? If you are redirecting old domain to new domain, your search engine position will change. Not only due to different domain name (keyword), other factors are involved too. One month is small time frame to get things on place again. When you are sure that pages are indexed under new domain, replace/edit/ask links from other sites.
Update May 2007 : After last Google PR update, new domain has PR 2 , all pages indexed, old domain could not be found. For PR 2, we can talk about good internal linking, not that PR is transferred to new domain. New domain is new domain.
YAHOO: Old domain removed, new domain indexed. Pages are slowly getting positions where they were before, under old domain.
MSN is still waiting for something. New domain still not included in index, and main page (only that page) is still indexed under old domain. According to above, I am not sure how MSN spider is handling with 301 redirection.
Update Jun 2007 : End of experiment, no success. While google has all pages indexed under new domain, Yahoo don’t want to listen. Main page and a few others are again indexed under old domain (redirection was on place all the time.). MSN still refuse to index new domain, keeping main page indexed under old domain. Since many links are still pointing to the old domain, now the new domain is redirected back to the old domain. New domain AND new hosting account is still the best solution.
Moving site from free hosting to new domain
This is not a permanent redirection, here is META tag used to redirect old page to the new page on new domain. So, if you still have an old site hosted on “free” hosting, and want to move that site to new domain, make and upload (different) page with the same name, add redirection META tag with 10 – 15 seconds, add page moved, bla, bla… in body, with link to newdomain.com/samepage.html, and your new site will be indexed fast. Maybe will a problem with duplicate content for the first time, but that will disappear after search engine database update. Old site will remain in index, so you can use it to to get other new site indexed faster, or add some other content there. Google “page moved” or “site moved”, and you will see some examples how to do this kind of redirection.
Using this “redirection” new site (domain) was indexed from Google and Yahoo fast, but two months later, domain still could not be found using MSN search (although spidered).
Update: It is fully indexed after three months.
BASE HREF tag and search engines
Base href META tag is useful when you are using relative linking on site (i.e. ../index.html instead of http://domain.com/index.html). When that tag is added to pages on new domain, it helps search engine spiders to determine real website domain.
So, it will not hurt when it is used in combination with domain redirection. Include domain version (with or without www) and page:
For main page (or subdirectory index page) could be added without page name (don’t forget slash on the end):