Web Design

404-200 Do you know whether your visitors are landing on non-existent web pages in your site? Do you know whether your site has broken HTML or image links?

I certainly didn’t until I added a real-time 404 not found email notification system to my websites. Yes, I know your log files give details of errors. But how often do you check your log files? Unless you check them for errors every day, it could be weeks or months before you discover the errors.

Once the 404 email notification system was up and running, my mailbox was flooded with reports of broken links, search engine bots trying to crawl old nonexistent files, broken image links and missing favicon images.

I was shocked just how many broken links my sites had - 196 404 not found error messages so far!

It even caught an issue with one of my affiliate campaigns. If I didn’t catch that and fix the issue, AdWords would have sent traffic to a non-existent URL. Over time, it could have cost me thousands of dollars in wasted advertising and lost business.

Can you tell I am really glad I set up this real-time 404 error email notification system on my sites? So much so that I immediately thought of writing about it so that you can also benefit.

Okay, here’s how I did it. By the way, your server must have PHP enabled otherwise this method doesn’t work. Ask your server administrator whether your server is PHP enabled and if not, ask them to enable it.

Also remember to backup any files you intend to edit. If something goes wrong, you can always revert back to your previous file.

The system uses two files; the .htaccess file which tells the server where to send users when there is an error, and the 404 error file which sends the 404 not found email notifications to you and informs the user of an error.

How to Create a .Htaccess File

A ‘.htaccess file’ (hypertext access) is the default name of a directory-level configuration file. If you don’t have a .htaccess file in the main directory of your server where you upload your files, create one by opening up a blank file in your favorite text editor and add these 3 lines of code:

ErrorDocument 401 /404.php
ErrorDocument 404 /404.php
ErrorDocument 500 /404.php

Definitions:

Error 401: Unauthorized - This error appears when someone tries to enter a part of the site they are not authorized to access.
Error 404: Not Found - This error appears when the URL someone is trying to access is incorrect and therefore the server cannot access it.
Error 500: Internal Server Error - This error usually is caused by CGI in which the script used in the process does not work properly.

Save the file as ‘.htaccess’ (a period followed by ‘htaccess’, without quotes. There’s nothing after htaccess)

How to View Hidden Files

If you can’t see the .htaccess file in Windows Explorer or Mac Finder, you have to enable the option to view hidden files.

On the Mac, open up Terminal and type this:

defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder

When it come to Windows users, there are different ways to enable hidden files depending on which version of Windows you have. I suggest you follow the instructions at How to see hidden files in Windows.

How to Create a 404 Not Found Email Notification File

Create a blank file with your text editor and add the following code:

<html>
<head><title>404 Error</title>
</head><body>
<h1>Sorry!</h1>
<table width=”60%” align=”center”><tr><td>Sorry but we couldn’t find what you’re looking for! Don’t panic - it’s not your fault. We have been notified of the error.</p>
<p>Here are a few options to find what you’re looking for:</p>
<ul>
<li>Double check the web address for typos
<li>Click the ‘BACK’ button in your web browser
<li>Search the site using the search form above
<li>Go to the <a href=”http://www.yourdomain.com”>homepage</a> (replace yourdomain.com with your website’s homepage)
</td></tr></table>
<?php
$ip = getenv (”REMOTE_ADDR”);
$requri = getenv (”REQUEST_URI”);
$servname = getenv (”SERVER_NAME”);
$combine = $ip . ” tried to load ” ;
$url = $servname . $requri ;
$httpref = getenv (”HTTP_REFERER”);
$httpagent = getenv (”HTTP_USER_AGENT”);
$today = date(”D M j Y g:i:s a T”);
$message2 = “On $today, $combine:\n
http://www.$url\n
User Agent = $httpagent \n
$httpref “;
$to = “email@yourdomain.com”;
$subject = “404 Error Report”;
$from = “From:email@yourdomain.com\r\n”;
mail($to, $subject, $message2, $from);
?>
</body>
</html>

I’m not a programmer so I borrowed the PHP code from Free Custom 404 Error php script with htaccess tutorial n block folder script. Thanks IBD Host!

Change ‘email@yourdomain.com’ to your email address and save the file as ‘404.php’. Now upload both the .htaccess and 404.php files to your main directory on your server. Test it by visiting a URL that doesn’t exist on your server. You should receive a 404 error email notice in your mailbox within a few minutes.

How to Redirect Broken/Missing Links

So what do you fix the broken links? First, you fix it if possible. If not, you add a ‘Redirect’ command to your .htaccess file like so:

Redirect /brokenlink.html http://domain.com/redirectedfile.html

The code above contains 3 parts. The first part is the word ‘Redirect’, followed by a space, then the broken/missing URL, followed by another space, then the URL you would like to redirect the user to.

Note that the broken/missing URL must be a local URL (meaning on the same domain). The redirected URL does not have to be on the same domain as the missing URL.

How to Set Up One 404 Not Found Webpage for All Your Domains

I have a dozen or so sites and didn’t want to have to maintain a 404 webpage on each site. It’s far easier to maintain just one 404 webpage. So I came up with this idea.

I originally tried to set it up so that all errors pointed to the a 404 file on the one website using these ErrorDocument commands in the .htaccess file:

ErrorDocument 401 http://redirecteddomain.com/404.php
ErrorDocument 404 http://redirecteddomain.com/404.php
ErrorDocument 500 http://redirecteddomain.com/404.php

The problem is that the error email message simply showed the error page (http://redirecteddomain.com/404.php) as the referring URL that caused the error, which didn’t help at all.

I couldn’t work out why it didn’t give me the referring URL, so I turned to my trusted techie guru, Peter Cooper, for help. And once again he came to my rescue. Thanks Peter!

He said, “You cannot use a canonical URL with ErrorDocument or it will generate a 302-Found (redirect) response, which will then result in a 200-OK on the error page itself. Use a local URL path instead.”

Now I understood why it wasn’t working. So I came up with this solution.

404 Not Found Email Notification File for Multiple Domains

I set up the 404 error email notification file using the code below. Remember to create a blank file with your text editor and add the following code:

<html>
<head><title>404 Error</title>
</head><body onload=setTimeout(”location.href=’http://www.redirecteddomain.com/404.html’”,10)>
<?php
$ip = getenv (”REMOTE_ADDR”);
$requri = getenv (”REQUEST_URI”);
$servname = getenv (”SERVER_NAME”);
$combine = $ip . ” tried to load ” ;
$url = $servname . $requri ;
$httpref = getenv (”HTTP_REFERER”);
$httpagent = getenv (”HTTP_USER_AGENT”);
$today = date(”D M j Y g:i:s a T”);
$message2 = “On $today, $combine:\n
http://www.$url\n
User Agent = $httpagent \n
$httpref “;
$to = “email@yourdomain.com”;
$subject = “404 Error Report”;
$from = “From:email@yourdomain.com\r\n”;
mail($to, $subject, $message2, $from);
?>
</body>
</html>

Change ‘redirecteddomain.com’ to the domain with the 404 landing page, and change ‘email@yourdomain.com’ to your email address. Now save the file as ‘404.php’. Upload this file to all your web sites.

Now when a user encounters an error on your site, the above file sends you a 404 error email notification. The user is then redirected to the 404 landing page (below) on your main domain.

404 Not Found Landing Page

Now create another blank file and add the following code:

<html>
<head><title>404 Error</title>
</head><body>
<table border=”0″ cellspacing=”0″ cellpadding=”0″ width=”100%”><tr><td bgcolor=”#FFFFFF” valign=”top” width=”55%”>
<h1>Sorry!</h1>
<table width=”60%” align=”center”><tr><td>Sorry but we couldn’t find what you’re looking for! Don’t panic - it’s not your fault. We have been notified of the error.</p>
<p>Here are a few options to find what you’re looking for:</p>
<ul>
<li>Double check the web address for typos
<li>Click the ‘BACK’ button in your web browser
<li>Search the site using the search form above
<li>Go to the <a href=”http://www.yourdomain.com”>homepage</a> (replace yourdomain.com with your website’s homepage)
</td></tr></table>
</body>
</html>

Save this file as ‘404.html’. Note that the file extension is ‘.html’ as opposed to ‘.php’. Upload this file to the domain to where you want the 404 error landing page to be located.

Don’t forget to upload the 404 error email notification 404.php file to your main domain as well. So you should have both the 404.php (email notification file) and 404.html (landing page) on the main domain server.

You can view my 404 error page at:

http://www.mikes-marketing-tools.com/404.html

So there you have it. Set up your very own real-time 404 error email notification system and you will be notified the instant an error occurs on your websites.

A Word of Warning

You may experience a flood of 404 not found error email messages. I have received a total of 196 error messages so far, although many of them were duplicates which were sent for the same error until I fixed the relevant issue. I am still receiving error messages as I type out this article.

Don’t worry. Simply go through each email and fix every issue you find. Once you fix all the issues, the flood should slow to a trickle as and when new issues pop up. If you get annoyed with all the error messages, simply remove the ErrorDocument code from your .htaccess file.

I hope you found this article useful. There may be other ways of setting up a real-time 404 error email notification system but that’s beyond the scope of this article and my technical knowledge.

Disclaimer: Please note that the PHP/HTML code in this article may not work on all servers. If you use the suggested code, you understand and accept any advice you follow in this article is done so at your own risk, and you do not hold me liable for any damage done. Sorry, but I gotta cover my own butt!