Once of the most important things when developing a website is making sure that it is easy for people to find the information they need. Site maps and site searches are probably the most commonly implemented functionalities for making a sites content easily accessible. Whenever I build a site that is more than just a few pages, I usually create a site map that dynamically generates links to every page on the site. Then I use the script below which reads the sitemap and then crawls the whole site and indexes the content into a verity collection to power my search functionality.

[More]

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Michael Evangelista's Gravatar Thanks for this article.

This part especially:
=====
You will see in the code that as the script is crawling each page of the site, it adds "search=Y" to the URLs query string. I set up my sites so that if URL.Search equals "Y", the pages do not display the sites header, footer, or side navigation. This way my verity index only contains the content in the body of the page.
=====
is SO smart!

I found a similar solution for the indexing via sitemap but ended up jumping through a few hoops to strip out everything before and after the main content area of the page, using a comment in the code. Of course, without that comment I'd be out of luck on any given page. Very cool solution here.

Also appreciate the info about the lock and possible corruption. I will be sure to revisit this code when I do my next verity-via-sitemap setup... soon!
# Posted By Michael Evangelista | 11/30/07 3:05 PM
Jason's Gravatar This code looks great and I would really like to try it out as a standard in my coding for verity search.

What should the sitemap.cfm page contain?

Just a list of links to pages on the site like below?

eg: <a href=index.cfm>home</a>
<a href=index.cfm?pageid=2>about us</a>
<a href=index.cfm?pageid=3>products</a>
<a href=index.cfm?pageid=4>services</a>
<a href=index.cfm?pageid=5>contact us</a>

I look forward to your reply.
Many thanks in advance.
# Posted By Jason | 11/3/08 6:55 AM
Scott Bennett's Gravatar @Jason,

This script is set up to read a sitemap where all the href attributes in the links contain full urls.

<a href='http://www.mysite.com/index.cfm' >home</a>
<a href='http://www.mysite.com/index.cfm?pageid=2" target="_blank">http://www.mysite.com/index.cfm?pageid=2' >about us</a>
<a href='http://www.mysite.com/index.cfm?pageid=3" target="_blank">http://www.mysite.com/index.cfm?pageid=3' >products</a>
<a href='http://www.mysite.com/index.cfm?pageid=4" target="_blank">http://www.mysite.com/index.cfm?pageid=4' >services</a>
<a href='http://www.mysite.com/index.cfm?pageid=5" target="_blank">http://www.mysite.com/index.cfm?pageid=5' >contact us</a>

However in indexsite.cfm you can change cfhttp tag that reads the sitemap to set the resolveurl attribute to "yes" and then cfhttp will change all your relative links into full urls.

<cfhttp url="http://www.mywebsite.com/sitemap.cfm"; resolveurl="Yes" method="GET"></cfhttp>
# Posted By Scott Bennett | 11/3/08 1:08 PM
Jason's Gravatar Cool! Thanks Scott.

Hopefully this provides me with a effective solution.
I'll post back and let you know how it works or if I have any other questions.
# Posted By Jason | 11/3/08 8:16 PM
Jason's Gravatar Hey Scott,

When testing the search, it doesn't seem to be searching the contents/body of the pages. It only returns results where the search term used matches what is in the page s <title></title>.

How do I get it to search the body as well?

Also, I cannot display what is stored as "body" and "URLpath".

Sorry I am sounding like such a newbie... this is my first time using Verity, normally I use queries across multiple tables, which is fairly slow.

Thanks in advance for all your help.
# Posted By Jason | 11/4/08 11:49 PM
Katty Lee's Gravatar Thanks for that to work on new ideas, ColdFusion perfectly complements Google!
Welcome to the site http://www.queentorrent.com
Here you can download a lot of interesting information.
# Posted By Katty Lee | 7/8/09 3:53 PM
https://www.bababorses.de/Louis-Vuitton-Damier-Ebene-Canvas-Clapton-PM-Bag-N44243-Magnolia-2361-it.html https://www.bababorses.de/Louis-Vuitton-LV-Trainer-Men-s-Sneakers-Top-Quality-15-5322-it.html https://www.bababorses.de/Celine-Small-Cabas-Bag-In-Black-Leather-it-2087 https://www.bababorses.de/Louis-Vuitton-Heel-10cm-Call-Back-Sandals-Nude-6162-it.html https://www.bababorses.de/LOUIS-VUITTON-BREA-MM-Monogram-Vernis-Leather-In-Magenta-4069-it.html https://www.bababorses.de/Louis-Vuitton-Ring-09-557-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-LV-Square-Espadrilles-Slipper-Sandals-Brown-6371-it.html https://www.bababorses.de/Prada-Golden-Saffiano-Calfskin-Leather-Top-Handle-Bag-it-2956 https://www.bababorses.de/Dior-Diorissimo-Small-Bag-Black-Nappa-Leather-Silvery-Hardware-8001-it-22 https://www.bababorses.de/Louis-Vuitton-Idylle-Blossom-Charms-Necklace-Q94360-406-it.html https://www.bababorses.de/Louis-Vuitton-Color-Blossom-BB-Star-Pendant-Necklace-Red-Gold-309-it.html https://www.bababorses.de/Bvlgari-Serpenti-Original-Leather-Framed-Pochette-Sky-Blue-82121-it-1938 https://www.bababorses.de/Louis-Vuitton-Horizon-55-Trolley-Travel-Luggage-Bag-Taiga-Leather-M30331-Red-6892-it.html https://www.bababorses.de/Fendi-By-The-Way-Small-Croc-Satchel-White-it-2731 https://www.bababorses.de/Louis-Vuitton-Monogram-Canvas-and-PVC-Nano-Bag-M61114-3176-it.html https://www.bababorses.de/Louis-Vuitton-Sarah-Multicartes-Wallet-M61273-Hot-Pink-7624-it.html https://www.bababorses.de/louis-vuitton-speedy-30--Damier-Azur-Canvas-n44367-2300-it.html https://www.bababorses.de/Hermes-Birkin-35cm-cattle-skin-vein-Handbags-blue-golden-it-907 https://www.bababorses.de/Saint-Laurent-Baby-Sac-De-Jour-Bag-In-Rose-Grained-Leather-it-3322 https://www.bababorses.de/Louis-Vuitton-Twist-MM-M53531-M53532-2775-it.html https://www.bababorses.de/Louis-Vuitton-Sunglasses-133-978-it.html https://www.bababorses.de/Louis-Vuitton-Neverfull-MM-M54185-Black-2705-it.html https://www.bababorses.de/Prada-Saffiano-East-West-Medium-Tote-Bag-Nero-it-3042 https://www.bababorses.de/Louis-Vuitton-Compact-Wallet-in-Monogram-Canvas-M63041-7399-it.html https://www.bababorses.de/Prada-Mens-Leather-Pouch-3312-Black-it-3099 https://www.bababorses.de/Louis-Vuitton-Women-s-Escale-Lock-It-Flat-Mule-1A7TOX-Pink-5965-it.html https://www.bababorses.de/Fendi-Baguette-Micro-Monster-Bag-Purple-Multi-it-533 https://www.bababorses.de/Louis-Vuitton-LV-Angel-Stud-Earrings-M64293-435-it.html https://www.bababorses.de/Louis-Vuitton-Damier-Ebene-Canvas-Zippy-Wallet-Evasion-M61360-7219-it.html https://www.bababorses.de/LOUIS-VUITTON-CATOGRAM-SQUARE-SCARF-MP2266-4818-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Empreinte-Triangle-Shaped-Messenger-Bag-M54330-Black-3865-it.html https://www.bababorses.de/Balenciaga-Velo-Anthracite-store-it-1723 https://www.bababorses.de/Chloe-Marcie-Medium-Satchel-Bag-Cobalt-it-2283 https://www.bababorses.de/louis-vuitton-epi-leather-Soufflot-BB-bag-m55613-black-2580-it.html https://www.bababorses.de/Louis-Vuitton-Dauphine-MM-M55735-4512-it.html https://www.bababorses.de/Louis-Vuitton-Crafty-NeoNoe-MM-bag-black-M45497-2980-it.html https://www.bababorses.de/Louis-Vuitton-Men-Box-Bag-Shoulder-Body-Bag-M44157-Brown-3136-it.html https://www.bababorses.de/Prada-Saffiano-Double-Zip-Executive-Tote-Bag-Gray-it-3025 https://www.bababorses.de/Louis-Vuitton-Epi-Leather-NeoNoe-BB-Bucket-Bag-M53610-Indigo-2564-it.html https://www.bababorses.de/Saint-Laurent-Small-Monogram-Tassel-Satchel-In-Red-Crocodile-Leather-it-3158 https://www.bababorses.de/Louis-Vuitton-Iphone-Case-LV18-59-it.html https://www.bababorses.de/LOUIS-VUITTON--CLASSIC-MINI-PACKBACK-2872-it.html https://www.bababorses.de/Balenciaga-Velo-Anthracite-store-it-1723 https://www.bababorses.de/Louis-Vuitton-Monogram-Ebene-Canvas-Pegase-Legere-53-Business-Rolling-Luggage-6950-it.html https://www.bababorses.de/Louis-Vuitton-Crocodilien-Brillant-Capucines-Mini-Bag-N93429-Black-2231-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Hoodie-Jacket-Black-1526-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Coated-Canvas-Popincourt-PM-M43462--Raisin-3345-it.html https://www.bababorses.de/Louis-Vuitton-Lockme-Cabas-Tote-M55028-Black-4530-it.html https://www.bababorses.de/Givenchy-Antigona-Small-Leather-Satchel-Bag-Black-it-2432 https://www.bababorses.de/Louis-Vuitton-Monogram-Canvas-Small-Malle-Chain-Bag-3294-it.html https://www.bababorses.de/Louis-Vuitton-Epi-Leather-Zippy-Wallet-M62304-Red-7304-it.html https://www.bababorses.de/Louis-Vuitton-Iphone-Case-LV113-38-it.html https://www.bababorses.de/Louis-Vuitton-Heel-10.5cm-Eyeline-Pumps-Python-Pattern-Suede-Black-5999-it.html https://www.bababorses.de/LOUIS-VUITTON-PEGASE-LEGERE-REGATTA-N41620-MONOGRAM-CANVAS-6974-it.html https://www.bababorses.de/Louis-Vuitton-Kimono-Wallet-M56175-Pink-7437-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Tapestry-Denim-Bidart-Espadrilles-Blue-5726-it.html https://www.bababorses.de/Louis-Vuitton-Women-s-Escale-Shirtdress-Blue-1709-it.html https://www.bababorses.de/Louis-Vuitton-Montaigne-MM-M41048-Black-4597-it.html https://www.bababorses.de/Louis-Vuitton-Heel-10cm-Crystals-Call-Back-Sandals-Suede-Red-6160-it.html https://www.bababorses.de/Hermes-Bolide-31cm-Togo-Leather-Green-Bag-it-1070 https://www.bababorses.de/Replica-Hermes-Wallet-H001-Wallet-Cow-Leather-Green-it-1558 https://www.bababorses.de/Celine-Medium-Luggage-Tote-Black-Brown-White-Bag-it-2168 https://www.bababorses.de/Louis-Vuitton-Pochette-Voyage-MM-Bag-Damier-Graphite-Canvas-Pixel-N60176-Green-7278-it.html https://www.bababorses.de/Fendi-Black-Snake-Veins-Leather-With-Beige-Ferrari-Leather-Top-handle-Bag-it-469 https://www.bababorses.de/Louis-Vuitton-All-over-Monogram-Sleeveless-Belted-Dress-Navy-1375-it.html https://www.bababorses.de/Louis-Vuitton-Ring-02-560-it.html https://www.bababorses.de/Louis-Vuitton-Iphone-Case-LV32-76-it.html https://www.bababorses.de/Louis-Vuitton-Dauphine-MM-M55071-Blue-4511-it.html https://www.bababorses.de/Louis-Vuitton-Supreme-Iphone-Case-White-Red-212-it.html https://www.bababorses.de/Louis-Vuitton-Epi-Smooth-Leather-Twist-Shoulder-Bag-MM-Pink-Black-2639-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Empreinte-Leather-Cosmetic-Pouch-Bag-M80502-Bouton-de-Rose-Pink-By-The-Pool-Capsule-Collection-4296-it.html https://www.bababorses.de/Louis-Vuitton-Bracelet-21-271-it.html https://www.bababorses.de/Louis-Vuitton-Heel-9.5-cm-Star-Trail-Ankle-Boots-Black-5511-it.html https://www.bababorses.de/Louis-Vuitton-Geronimos-Belt-Bag-M43502-Black-Epi-Leather-2646-it.html https://www.bababorses.de/Louis-Vuitton-Damier-Ebene-Canvas-Vavin-Chain-Wallet-N60222-Bordeaux-Red-7221-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Empreinte-Leather-Zippy-Coin-Purse-M80408-Cream-Saffron-By-The-Pool-Capsule-Collection-7741-it.html https://www.bababorses.de/Louis-Vuitton-Vintage-Monogram-Vernis-Bleecker-Box-Top-Handle-Bag-Burgundy-4172-it.html https://www.bababorses.de/Prada-Saffiano-Mini-Galleria-Crossbody-Bag-Beige-it-2708 https://www.bababorses.de/Louis-Vuitton-Sunglasses-39-1042-it.html https://www.bababorses.de/Louis-Vuitton-Monogram-Canvas-Leopard-Print-Onthego-Tote-Bag-M44674-Black-White-3232-it.html https://www.bababorses.de/Louis-Vuitton-Epi-Leather-Twist-PM-Bag-with-Crystal-embellished-Chain-M55412-White-2630-it.html https://www.bababorses.de/Fendi-Chameleon-Red-Cross-Veins-Leather-Tote-Bag-it-488 https://www.bababorses.de/Louis-Vuitton-Monogram-Canvas-Onthego-Tote-Bag-M44571-Kaki-3270-it.html https://www.bababorses.de/Fendi-Earth-Yellow-Leather-with-Multicolor-Striped-Fabric-Shopping-Handbag-it-771 https://www.bababorses.de/Louis-Vuitton-Croco-Pattern-Petite-Boite-Chapeau-Bag-Black-4090-it.html https://www.bababorses.de/Prada-Saffiano-Small-Double-Handle-Tote-Bag-Light-Gray-Pomice-it-2849 https://www.bababorses.de/Louis-Vuitton-Lvxlol-Speedy-BB-M45202-Golden-3125-it.html https://www.bababorses.de/Louis-Vuitton-Gloria-Flat-Open-Back-Loafers-Monogram-Canvas-5798-it.html https://www.bababorses.de/LOUIS-VUITTON-BREA-PM-Monogram-Vernis-leather-IN-MORDORE-4074-it.html https://www.bababorses.de/Replica-Hermes-Steve-H2810-Ladies-Shoulder-Bag-Cow-Leather-it-1428 https://www.bababorses.de/Louis-Vuitton-Noe-bag-M42226-Brown-3496-it.html https://www.bababorses.de/Louis-Vuitton-Damier-Azur-Canvas-I-2260-it.html https://www.bababorses.de/Christian-Dior-Multicolor-PeachYellow-Zipper-Wallet-118-it-223 https://www.bababorses.de/Fendi-By-the-Way-Small-Tricolor-Satchel-Bag-it-2916 https://www.bababorses.de/Prada-Medium-Vitello-Diano-Open-Tote-Bag-Pomice-it-2728 https://www.bababorses.de/Luxury-Hermes-Wallet-H001-Unisex-Wallet-it-1598 https://www.bababorses.de/Louis-Vuitton-Iphone-Case-LV42-104-it.html https://www.bababorses.de/Prada-Saffiano-Small-Gardeners-Tote-Bag-Blue-it-2803 https://www.bababorses.de/Louis-Vuitton-Sac-Tricot-Bag-Epi-Leather-Red-M52805-2736-it.html https://www.bababorses.de/Louis-Vuitton-Crazy-in-Lock-Strass-Bracelet-Silver-316-it.html