Search engine optimization: Getting the process right from scratch part 1

Hello again 2.0′ers!

You’ve told me you’re loving the SEO blogs and I love my search optimization as well – so here goes another post. Today we are going to be looking at how to get the Search engine optimization process correct – from scratch!

That’s right people! We’re going to go through just about everything you need to do, and in what order you have to do them for your site, blog or online application to have as much chance as it can get with the marketing push that comes as a result of getting search optimization right.

The post will at many stages branch off to what you should do if you are running a blog and what to do if you are dealing with a general website. Because Wordpress is the most popular blogging platform online, we are going to assume you are running it! I am also going to go out on a limb and assume you as the faithful reader also have a working knowledge of HTML as you will need to do some handy-work at some stages.

Please not that in this blog post we will not be going into specifics such as keyword research and competitor analysis, etc. It is simply a guide to get the SEO process right from the start!

So now that the introduction and caveats are out of the way; without further ado – let’s kick some SEO but!

Step 1: Build your website or install a Wordpress blog on your server.

Okay, this step isn’t rocket science – but for the sake of argument, we are going to assume you already have a website designed/developed and sitting there ready to be optimized. On the other hand, you could have just installed a Wordpress blog on your server and it has come with the default theme with a pre-installed comment by ‘Mr Wordpress’ to get you going

Step 2: Setup your robots.txt file

Wow, wow, slow down cowboy! What’s a robots.txt file?
Good question! Guys who don’t know this one, don’t worry – many people don’t really understand the underlying meaning of the robots.txt file.

In essence, the robots.txt is literally a text file which resides in your home directory; generally ‘/public_html/’ ‘/home/’ or ‘/www/’. It contains instructions or simple rules that robots, or search bots, such as Googlebot use when crawling your website or blog.

What does it do?
It tells search engines, through their spiders/bots (which crawl your website, and look for this ‘robots.txt’ file) what content to allow people searching content from your site to find.

Give me an example!
Geez! ask for much?

Okay, let’s take an example from my own robots.txt file.

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-login.php

User-agent: Googlebot-Image
Disallow:

# Does anyone care that I love my sitemap?
Sitemap: http://www.cheb.com.au/sitemap.xml

Okay, it’s not as gibberish as it looks, believe me! Let’s dissect it…
User-agent: is basically stating which search spider/bot we want to target. By me putting * it is basically saying I don’t care – I want all bots to follow the following rules. Basically, any rule(s) specified under the ‘User-agent: *’ will make all bots who crawl your website or blog cooperate! Cool hah!
If you look at the next section, my user agent is ‘Googlebot-image’. What I’m saying there is I don’t mind Googlebot-image crawling my server for relevant images to add to Google’s image search at http://images.google.com/imghp?hl=en.
Anything that you pre-mark with a “#” is a comment, and will just be ignored; so when it comes time for me to be smart with my robots.txt file, having “# Does anyone care that I love my sitemap” doesn’t affect anything.

Okay, this is pretty cool… Are you saying I get to tell Google, Yahoo! and other search engines what to crawl?
That’s exactly what I’m saying! It’s that simple! Create a ‘robots.txt’ file, fill in the relevant information, save and upload to your main folder (where you can see the file you call when you type in www.your-domain.com) and you’re off on your SEO dream start.

A few more examples?
To exclude all robots from the entire server and its contents

User-agent: *
Disallow: /

To allow all robots complete access to the server and its contents

User-agent: *
Disallow:

To allow only one bot (Google) complete access to the server and its contents

User-agent: Google
Disallow:

User-agent: *
Disallow: /

Because there is no ‘Allow’ rule, we have two separate rules: Google is disallowed from nothing; whilst everyone else (bot-wise) is disallowed everything in ‘/’ which is basically anything after your domain. i.e. www.cheb.com.au/anything here should NOT get crawled.
I say ’should not’ as apposed to ‘will not’ because some bots choose not to agree and will just crawl your content anyway. Although, the prominent ones we care about like Yahoo! and Google usually listen!

There is a huge index/website which has everything you need to know about robots.txt alongside more examples and information at The Web Robots Pages

Step 3: Setup your sitemap

Wow, wow, slow down cowboy! What’s a sitemap?
A sitemap in very basic English is a listing of where to find ’stuff’ on your server or website. Having a well structured sitemap will help a search engine spider and index your pages more efficiently and ultimately lead to the possible quicker addition to the search index of that particular search engine.

How do setup a sitemap if I am running Wordpress?
Try installing the cool Wordpress Google Sitemap Generator by Arne B. It takes the (albeit minor) pain out of site mapping.

How do setup a sitemap if I’ve designed/developed my own website?
That’s easy too! There are plenty of XML generators online which generate Google-friendly sitemaps. Check out the aptly-named XML Sitemaps. The script will run through a domain you input and look for any links it finds on that page. Consequently, it will then follow any link which is fiound on the pages off the main page – just like a search bot does, and then generate a sitemap for you! The free version only crawls up-to 500 links, and might take its time – so be warned!
You will want to download either the uncompressed or compressed XML version of the file – and if you like keep the HTML version for yourself. Google (and other engines) will need the XML format! Here is an example (of one section only) of what you should see if you load the XML file you download into a browser:

<?xml version="1.0" encoding="UTF-8" ?>
- <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
- <url>
<loc>http://www.cheb.com.au/</loc>
<priority>0.5</priority>
<lastmod>2008-03-26T16:56:09+00:00</lastmod>
<changefreq>weekly</changefreq>
</url>

Step 4: Setup your Google Webmaster Central & Yahoo Site Explorer accounts

This step is important so you can track a lot of things about your SEO experience. Google Webmaster Central for example let’s you track your sitemap file and update it ‘on the fly’ as well giving you detailed information about your particular URL and who has searched for what to get to your site/blog.

Yahoo! is pretty much the same, being able to track domains to your account and allowing you to run diagnostic tools to nut out any problems.

Both GWC and YSE ask you to verify that the domain actually belongs to you by either asking you to upload particular file to your web server (therefore proving you can login to the hosting account) or adding a meta tag such as:

<META name="y_key" content="2b21b3727a881f" >

Once you register for the accounts you will be able to make sure your SEO campaign is on track by finding and diagnosing any issues your sitemap, robots.txt file; or anything else for that matter – may be inhibiting the search engine from being able to successfully crawl your content. Remember, if search bots cannot get to you – you cannot be listed in their index!

Register for Google Webmaster Central and Yahoo! Site Explorer today!

Step 4: Setup Feedburner account

If you are running a blog, or even a website with RSS – you need to have a Feedburner account. Feedburner is an online service which allows you to ‘burn’ your RSS feeds – practically allowing you to find out how many people have subscribed to your syndication as well as allowing you to monetize your blog or website’s traffic through various sources.

More than anything else, Feedburner is the foremost RSS feed management service online. Their feed management tools will make sure you can track any potential problems with your RSS feeds – which could potentially be a problem for your search engine optimization because generally speaking most of the time RSS/XML problems are caused by issues in your code! Remember, we want to make sure nothing comes in our way of a successful SEO campaign.

If you are running an XML or Atom-based RSS feed, make sure you snap up your Feedburner account today if you don’t already have one!

Step 5: Content, content and more content

Make sure that at this stage you have all your content ready to go where it should be. There already is a possiblity that search engines have grabbed you into their index for many reasons. If you are writing a blog, have a post or two already written and as much as possible try to stick to a posting pattern – at least once every week to two weeks.

There are a lot more content rules we can follow, including how to link and write content in a way that it is SEO-friendly. Check out our previous article on SEO-driven blog writing techniques.

Where are my headings?
It is also important, for SEO purposes that you should have at least one ‘heading one’ tag appear at the top-most part of the site/blog. Only have one <h1> tag per page, and make sure that it is well-written using main keywords you want to tackle as well as keyword phrases.

You might think about running an SEO-experiment like I have, but the safest thing to bet on is that every page should have only one H1 (<h1> tag) which is generally the actual title of the blog post or page.

Please note that search engines can and do penalize you for using more than one heading one tag. Do NOT keyword-stuff your headings. Remember, write for humans first, search bots second! Not only do you risk losing a visitor/reader – but you could end up in supplemental results!

Finally, you can use any number of H2s and subsequent levels of header tags in a page, but if we wanted to get somewhat picky, semantically, you shouldn’t jump between headings. For example, H2 should follow H1, as H4 should proceed H5 – but that’s just being picky

Step 6: Add yourself in…

Okay, at this point, if you cannot find yourself in search engines, or after a search for “site:www.you.com” in Google or Yahoo! – then add yourself in. Please note, if you are running Wordpress, and most other blogging platforms – unless you have unchecked the option during install, you will ping Technorati every time you blog, so there is a high chance you could already be indexed! Check Technorati to be sure.

So there you go! Believe it or not, you have started your SEO campaign. Believe me, what you have already done has got you miles ahead of those who just ‘launch’ a standard website or blog.

You just need determination and drive to keep it going – and more importantly believe it will work because you will invest more time in making sure it does! Stick around for part II of this post which will go through in more detail what to do now that you have setup the foundation for search optimization.

We will be going through link-building and backlinking techniques as well as where to market your website or blog to make sure you get the extra step forward!

Be sure to subscribe to our RSS feed so you can get posts in your email inbox and also find out whenever we post new content. Share this article with a friend or digg it if you found it beneficial!
Till next time, Cheb.