DIGITAL MEDIA

The search engine optimisation (SEO) industry has grown at a rapid pace – so rapid that no one in Australia has much of an idea how big it actually is. And just like similarly fast-growing opportunities, such as America’s Wild West or the Australian goldfields, SEO has attracted its share of cowboys.

SEO is the art of ensuring that one website outranks others in the natural results of a search engine such as Google, Yahoo!7 or ninemsn for particular search terms. A high appearance in the natural listings is very desirable, as these generally account for 80% or more of clicks. And unlike paid search tools, such as Google’s AdWords, these clicks cost nothing.

Various techniques can be used to ensure that a site ranks highly, depending on the algorithms used by each engine. Some of these are considered acceptable by the search engines – these are called white hat techniques. Others are not, and these are called black hat techniques. Inevitably there are also some grey areas.

One black hat technique involves creating so-called doorway pages stuffed full of key words, designed to confuse the software tools used by search engines to compile their listings (also known as crawlers, spiders or bots). End users, however, are generally unaware of this and are whisked seamlessly along to another page.

Another, known as link farming, involves creating hundreds or thousands of illegitimate links to a website. This is particularly effective for Google’s search algorithm, which rates sites by the number of pages that link to them.

Beating the traffic

More basic techniques involve loading a page with popular but irrelevant terms to lure unsuspecting traffic. Often the text is the same colour as the background it appears on – invisible to the visitor, but highly visible to the search bot.

It can be a lucrative business, with stories of one service provider in the US earning $1m for keeping its client at number one position for the term ‘online casino’ for six weeks.

“A lot of people don’t appreciate the extent of power and importance there can be in having a very good search engine ranking,” says Tim Macdonald, director of search marketing firm The Found Agency.

The Found Agency has also suffered the consequences of going further than the search engines judge to be acceptable in the pursuit of vital clicks - more on that in a moment.

“I’ve come across online and offline businesses where very large proportions of their new business is generated by their rankings in Google, both natural and paid. It can for small businesses be their number one source of inbound leads and sales,” says Macdonald.

Numerous service providers have emerged to help tailor sites. Is it so surprising then that a few may chose to bend the rules to help their clients?

According to Google’s search evangelist Adam Lasnik, SEOs provide useful services that range from writing copy to giving advice on site architecture and helping to find relevant directories on which a site can be submitted.

“However, a few unethical SEOs have given the industry a black eye through their overly aggressive marketing efforts and their attempts to unfairly manipulate search engine results,” Lasnik says.

But the search engines have shown they have ways of fighting back.

In February 2006 the luxury car maker BMW had its German website blacklisted by Google after it was caught trying to artificially boost its ranking through use of a doorway page. The blacklisting meant that for a period any search term such as BMW of BMW Germany would not provide a direct link to the car company’s German website.

Always consequences

BMW’s doorway page contained 40 repetitions of the German word for ‘used car’. Users were unknowingly redirected to the proper homepage for BMW Germany, which contained the term only twice. The German site was eventually reinstated to the index.

Lasnik says Google aims to put a dampener on sites using techniques that deceive the user or attempt to manipulate rankings. These efforts are headed by Google’s search quality team, and are ongoing. But Google would rather work on the big picture of improving the overall quality of its index through improving the way it interprets the web, rather than whacking individual non-compliant websites with a proverbial mallet.

“Often times, when a specific ranking adjustment affects a particularly large or public site, that site is actually just the most visible part of a broader group that’s been affected by refined algorithms,” Lasnik says. “However, we do reserve the right to take one-off actions against specific sites that violate our guidelines.”

While the damage to BMW Germany was primarily to its reputation, various clients of the Las Vegas-based SEO company Traffic Power were not so lucky. According to The Wall Street Journal, in 2005 Traffic Power was judged guilty by both Google and Yahoo of using black hat techniques such as doorway pages, and both its own corporate site and those of many of its clients were pulled from the listings. One Traffic Power client, a website for selling stadium seating, disappeared from Google’s search results for four months.

“There are plenty of examples of businesses in the US that have gone down a black-hat route and been kicked out of the Google index or penalised, and their businesses folded,” says Robbie Hills, managing director of 24/7 Real Media.

“But we believe that a good SEO strategy is really developed over the long-term, and there are things that you can do that will give you good results on all of the search engines, and not harm you in the longer-term.”

Rules are not clear

What is and isn’t allowed is set out on the webmaster help pages of each search engine, but the algorithms used to determine the search results are prone to changes, leading to disagreement amongst some SEOs as to what does and does not work. There is also some disagreement on how these instructions are to be interpreted. What might be considered to be okay yesterday may no longer deemed so today.

Hence the boundaries are prone to getting pushed. DGM’s managing director, Fionn Hyndman, says the risk of being banned is ever-present, regardless how tantalising black hat techniques might appear.

“It’s that balance of how business critical it is. If it is that business critical to be number one, then how damaging is it to your business to be banned?”

Hyndman says for this reason many SEOs set up test sites to see what will and won’t fly with the engines.

“Some people use their own site as a test bed, and others use it to push the boundaries themselves, but it’s not something we believe in. It paints quite a bad picture to clients if you are getting your own site banned.

“What people have tended to do is rather than taking one rule and breaking it in a big way, they are trying to push the boundary on several of the rules.”

Fall from grace

One company to push those boundaries a little too hard in Australia was The Found Agency, which in May fell from a top 10 position in Google’s natural listings to one in the 50s for the term ‘search engine optimisation’ after it had been discovered offering a free ‘click counter’ to other web sites. The click counter acted as a link that pointed back to The Found Agency’s website, driving up the number of links it received and boosting its position in Google’s rankings.

According to Macdonald, there was no warning from Google that action would be taken – one day the site’s ranking just plummeted.

“We haven’t disappeared – nothing else has happened except that our rankings have dropped,” Macdonald says. “We ranked extremely strongly for two and a half years, and I expect that our rankings will go back up – it’s just a matter of time.”

While The Found Agency may simply have been unlucky, one report in The Sydney Morning Herald in May quoted an unnamed executive from an Australian SEO company as believing that as many as eight of the top 10 SEO firms in Australia behave unethically.

It is a charge that is refuted by Macdonald and others in the industry.

“I don’t see people doing unethical stuff for clients, because realistically there is no need to do anything that would be considered black hat or grey hat to get a client to rank,” Macdonald says. “We have been approached by people with get-rich-quick schemes and all kinds of business that we don’t really want, from people who want to zoom out to the top four. There is so much work out there that you just don’t need to do those kinds of things for clients.”

No guarantees

Stephen Murphy, director of search marketing firm Paperclick, says it is important that any company wishing to engage an SEO specialist do their homework thoroughly to make sure the agency is reputable. In the absence of any form of industry accreditation, this means seeking out testimonials from other satisfied clients. At an international level the Search Engine Optimization Professionals organisation has been created to develop a set of best practices for SEO professionals, but only five Australian consultants are listed.

Murphy says he would be in favour of some form of independent accreditation for the Australian industry.

“Between testimonials, reputation and industry bodies, there should become a default list that any new client thinking of search covers off,” Murphy says.

And he cautions particularly against any service that offers guaranteed results.

“If anyone does say they can guarantee results, walk away,” Murphy says.

One of the problems for SEOs is that the search engines often act as judge, jury and executioner, with little correspondence between themselves and the SEO community that relies on them. Some executives feel that with Google holding such a position of dominance over search engine traffic, they have no option but to play ball with an organisation whose policies and decision have been described as secretive at best.

Lack of transparency

In March 2006, the San Jose based parental advice service moved to sue Google for downgrading its search ranking without reason or warning, leading to a 70% fall in audience that led to an 80% decline in revenue. Google won the case in March this year when a judge ruled that KinderStart had failed to prove how Google had caused injury to it by a false statement.

According to Murphy, Google’s current alert tool in many cases is no more sophisticated than having competitors dob the offending site in.

Google hints at it in its own guidelines to webmasters, urging, “Avoid tricks intended to improve search engine rankings. A good rule of thumb is whether you'd feel comfortable explaining what you've done to a website that competes with you”.

“And it also decides for itself whether you are getting removed or go into the supplemental index,” Murphy says. “The whole degree of the penalty – the waving of the stick – that does not seem to be consistent at all. And only in the last year was there a way to have your site re-included.”

Lack of transparency is an issue that Lasnik acknowledges, but says he has witnessed Google move to a position of much greater openness from where it was when he joined a year ago. He believes that with its Webmaster Tools, it already offers far more information than any other search engine together with diagnostic information on how the Googlebot (Google’s search index bot) sees each site.

“We also encourage anyone with a site that’s violated our guidelines to file a reinflation request,” Lasnik says.

Lasnik says the company has been working hard to inform webmasters of its changes, while also learning from them, getting its own staff out among the webmaster community in many parts of the world. The head of Webspam at Google, Mark Cutts, has his own blog as a means of furthering communication, and the company now has its official Webmaster Blog operating in many countries, including Australia. Finally, more Googlers are now dedicated to answering questions and escalating concerns through the Google Webmaster Help Group.

Lots more to do

“I’m confident that we’ll continue increasing the depth and scope of communication, because by helping webmasters, and also help their users, and our users, at the same time, to find the most relevant information on the web,” Lasnik says. “It’s a winning situation for everyone.”

Everyone but the rule-breakers, that is. Hills is firm in belief that the SEOs are not getting cleverer at beating the system.

“I actually think it is the other way around – Google is getting better at detecting them,” Hills says. “So the search engines are increasingly becoming better at working at what is going on and the way that people are trying to fool their spiders. And that is why you have seen a few examples in recent times, because the search engines have gotten better at picking it up.”

Indeed, the whole nature of what is required to keep a site highly ranked will eventually supersede the effectiveness of many black hat tools. Murphy says competition among SEOs means it is no longer enough to make changes that do not change the body content or copy of a site.

“You cannot optimise a site on the factors that don’t affect the body copy alone, which means our whole industry and process has to engage the client from content creation, or even deciding what phrases to take upon your site,” Murphy says. “We have to do a lot more work these days.”

1 comments Leave a comment