Here’s another in the series of things to consider when planning a new web venture. Searching is big news - so don’t miss out. When building a site - anything with more than 5 pages you will pretty much always benefit from adding an internal search engine, basic or complex, open source or hand written.

There are several options for including a search engine in your site, we will start with Open source options, move to google and end up building our own - each has its benifits.

Open source - Sphider.

I chose sphider because I am fairly familiar with it, and for its few annoyances its good. Its essentially a php spider that can be deployed on a site to provide almost a google emulated experience. The spider works well and you can leave it too it without issue. It uses a mysql database, which I always like, so as I can mess with it should I need too.

The best point to sphider that it is easy as pie - easy to integrate, easy to install, easy to understand. However with this ease comes the requirement to re-crawl the whole site when you add a few pages. Fine if you rarely update or run only a few pages, but for a bigger site this becomes tedious very quickly.

So IMO use sphider when your site is fairly stable, not particularly dynamic and if you want a quick BAMMM solution.

Googleness

So you can use the google search engine - which is good also - I won’t talk about this too much because I only use it on one of my basic advertising sites because it annoys me slightly.

Essentially you can use it but it will either require a ’site or web’ radio button - or if you are using adsense then provide you with 1 page of ads that will probably snatch your user before he even looks at the results. If he does look then it can be a pain letting him see what you want him to see.

Essentially even easier to intergrate google internal search is also good for a quick result - more dynamic than sphider but you lack absolute control over the search and its formatting.

Finally the home grown method

Build it yourself - Now its usually a toss up for me between using open source software which is free and either living with the annoying little facets or modifying them slightly OR completely writing a new version. The latter is usually less chosen because in efficiency terms its negative.

By building it yourself you can of course provide a completely bespoke solution but it may take substantial time or money to produce.

Having built several, I would recommend the following:

• Include “hot words” - words that when entered take the visitor directly to a call to action page, that is a common page where they can fulfil one of your aims, e.g. typing ‘contact’ takes the user directly to your mail form, ‘order’ strait to your order form etc.
• Look to match each and every part of the phrase - if its a string - explode it by ‘ ‘ so as you get an array to pass through your net. searching for ‘red box’ will then search for box’s of other colours and other red things for example.
• Build a weighted system - Make it possible to force specific pages up and down the list for keywords - Imagine you had control of ebay for a day - you would stick your most important, sales or contact - call to action pages at the top - so when you build your own include this control.
• Use Get rather than Post - By using get you can then later on provide tag like support for people e.g. if you like red boxes try looking at ‘red’ (then you can easily just link red too ’search.php?query=red’ - or whatever!)
• Log Your searchs - like all data, search data is useful. Whether its so you can improve the site, pick better keywords, weight pages or provide some sort of relayed content such as ‘most searched’ or whatever - save it down.

---

Just a quick glimpse into the internal search engine, Based on the fact I have been working on ours today (wnu.) If you liked this basics to web planning then check out some of the others in my series on web development: Planning:

•  Pagex - Bolt on CMS for SEO people
• 10 Steps to submitting your site
• Phpbb, wordpress and oscommerce
• Mod-Rewrite - File Structures