Author: Gizmo (James of VNC Web Design)

Requirements: UBB 7.x

About:
This addon takes just moments to install, it gathers your forum topic links and exports them as a sitemap that you can add to Google, yahoo, or just about any search engine which utilizes sitemaps.

The configuration options allow you to specify which type of sitemap you wish to use and weather you wish to plugin threaded or flat view. The sitemap type and view type can both be set either hard coded or via url string.

Simply enter the URL to your sitemap install into your Google sitemap list (here) and google will automatically grab copies of your thread links every week.

What are the security aspects?:
Well, the sitemap will ignore threads in private forums (if a guest cannot *see* them then the sitemap will not publish the link). Also, threads that are not approved will not show either (same aspect as above).

What is a sitemap?:
A sitemap will take and provide all of your current threads to search engines, thus eliminating the wait of "new discovery" of new pages on your forums.

You can read about the benefits of using sitemaps via the below quoted article from Google Webmaster Tools.

Please note that this addon does not create a sitemap for your entire site, and only creates one for your forum.

Demo: As this addon doesn't utilize a cache, no active demo is available, however you can see example output via the attachment screen shots.

Install Instructions:
The install is quite simple, after receiving the software via email simply open the php file and place your MySQL connect information where indicated. Now upload the file to your web root (not your forum directory) and then simply open the script via the web and you'll see the default XML (google valid) sitemap.

Usage Instructions:
By default, the script uses flat mode and the XML map. These defaults can be easily changed from within the script itself.

As for web-based switches:
http://www.yoursite.tld/sitemap.php?type=&view=&se=
// Optional Paramater -- se=# Where # weather or not to use SE Friendly URLs (1/no, 2/yes)
// Optional Paramater -- type=# Where # is a map type (1/XML, 2/Text, 3/html, 4/RSS, 5/ASP, 6/SitemapIndex)
// Optional Paramater -- view=# where # is your view type (1/ShowFlat, 2/ShowThreaded)
// Optional Paramater -- offsets=# where # is weather or not to use offsets.

Submissions:
Not all search engines will automagically re-check your sitemap; in fact, I've only noticed that Google will re-check on its own. You'll want to go to Yahoo or anyone else every couple of weeks and tell them to re-grab a copy of your map.

Version Additions:
v0.3 - Support for SiteMap Indexes has been built in; this works well with the "offset" feature added in v0.2. The sitemap index will look at the xml sitemap data and build the proper number of "valid" links to offset pages. This is extremely useful for larger forums as they can set what level they want to offset their data (as to not go over the 50k maximum link limit with most providers) and they just need to submit one url (versus numerous to cover additional sitemap pages.
v0.2 - This version introduces offsetting; Google has a maximum of 50k links allowed in a sitemap, some larger (huge) forums have more than 50k threads on their forums. This setting is enabled by default, and can be utilized by using the offset variable in your urls submitted to google/yahoo. Additionally, it can be disabled completely in the configuration file.

Attachment Information:
sitemap-xml.jpg - An XML map.
sitemap-rss.jpg - An RSS map.
sitemap-text.jpg - A simple "text link list" map.
sitemap-link.jpg - A simple "html link list" map.
sitemap-asp.jpg - An ASP style sitemap.
sitemap-index.jpg - A screen cap of the Sitemap Index feature.

Pricing: $50 per install (each seperate install requires a separate license)

How to Buy: You can pay via PayPal to James[at]VirtualNightClub[dot]net (please replace "[at]" with "@" and "[dot]" with ".") or click here to use our order form (PayPal as well).

Notice:
This addon is not freeware, and as such you cannot freely provide it to others without prior written permission from its author(s). Permission was obtained from the UBB Dev administration to post this non-free addon's information in advance.

Quoting Google's Webmaster Help
What is a sitemap, and why should I have one?


Licensed Sites: 8 (not including my own or ubbdev)

Thanks Gizzy, should be a very worthwhile add-on

hehe dankies allen ... I've pretty much had it in private development and word of mouth for 2 months now; current users rather like it ...

And eliminating the need for google to "discover" your links is definatley a plus... Helping them along should help get content listed much faster for users :nods:

It would be ideal if it were to list pages, which are not forums, yet use the forum header / footer - i.e. articles.

I have around 200 of these pages, which will grow to 500 very soon.

Those are kind of out of scope I think, it's designed to just pull the threads themselves, not forum topic indexes (which will likely be in a future build).

I'd have to see how you're calling these extra pages (header/footer) to be able to give a for sure answer.

I'm pretty sure that I could make an index page which could take all files in a directory and build a sitemap for you; but keep in mind that if it's just a custom header and footer that google won't really care; you're not feeding your content to them, you're simply feeding urls of your threads to them; the feed does not include any post content (as they don't want it).

Another product i'll be coming out with soon is a simple sitemap lister that you can manually enter urls on your site; but I'm working on a couple of other projects until I can get to that.

I've been toying with Curl for building automagic sitemaps, I have something that works for the most part, but you have to think that php has an execute script limit which would make it die prematurely on large sites (or forums for example).

I am calling the pages with a script, template and php file.

So, they're never actually *called* as their own page?

Keep in mind, literally, all a sitemap is would be the link to a topic/forum; there is no data presant, you're helping google/yahoo/etc with discovery, you're not feeding them content...

Since they're not actually *called* as data for yoru forum by themselves, theres no reason to have them in a sitemap...

Outstanding!

Since I moved from 6.5 to 7.X Google has all but stopped crawling my threads/topics and as a result the numbber of new members I get has dropped dramatically! I hope this will help (should I decide to buy it)!

Thank!

Basil

i went thru all kinds of hell with google indexing my forum once i went to version 7. gizmo made it all beautiful again.... i thank him very much.

It turns out that Google allows a maximum of 50k links in a sitemap; so I'm working on v0.2 right now that will allow you to set an offset via the URL so that you can split links up for larger forums.

EX:
ubb.sitemap.php?type=1&se=2&offset=0 - Will display threads 1-25k
ubb.sitemap.php?type=1&se=2&offset=1 - Will display threads 25k-50k

You can set the max offset in the file up to 50k, I put 25k down as a default to help conserve some load.

BTW, this will be enabled by default in v0.2; but you can disable it for the classic behavior of loading everything at once. I've mainly coded this in for large forums lol...

Also, an example of how insanely huge 25-50k threads is... UBBDev only has about 36k threads.

Google + Gizmo's Site Map

I Have recently moved my forums to a new Domain.
So it was important to me to get my site under its new Domain name back into google as soon as i could.

Gizmo has been raving about the site map program he made,
so it thought to my self (He seems like a nice chap)
I think i will give it a go and see if it really does work.

The answer is YES

21 days after i installed the site map Google has listed my
site under its new domain name with 6,000 + links.
Not all my forums are public

It is fair to say what would have happened if i didn't have
the site map installed? Well i cant answer that to be honest
as I'm now listed. I like to think that it has helped
in my situation where a Domain move for my forums.

The Site Map has worked for me, but if you have any questions
about it functionality then please contact Gizmo via PM
as i can only report back the results i have.

Gizmo's site map isn't free, but for my needs worth the money.
You will also need a Google web masters account.
It also works with Yahoo.

My Results Click Me After 21 days.

Glad to hear that you've had good results mark .

Just for anyone wondering, with moving to a new domain with 0 data in Google for it, it'd likely take several months to allow discovery enough time to crawl this many links...

v0.3 has been completed this morning. I've done a lot of code cleanup, on both a security and code standpoint. Instead of "assuming" data passed to the script from the database is "valid" for feeds, we run subjects through htmlentities (thus validating code) and we force "int" on fields which should only contain numbers.

I've repaired a problem with the offsets option introduced in 0.2 which allows you to split your one large sitemap into several easier to manage sitemaps (thus allowing you the opportunity to not go over the 50k link limit that some services enforce.

I have also built onto the offsets option and built in the ability to utilize a sitemap index which will list any "valid" sitemaps on from your forum (built using this script only); everything will be done automatically, all you need to do is submit the sitemap index into google (or a search engine which support them) and when you tell them to look at your site, they will grab your list of maps and work on them accordingly.

I have also set the "offsets" option to off by default; when I introduced it in 0.2 I set it to "on" by default.

Coming in 0.4:
I'm looking to add the ability to base "importance" to urls based on their date posted; this will allow the newer data to have a larger importance than the older data, thus allowing it to be crawled more often than older threads.

I'm also looking to allow the maximum thread inclusion to be adapted in the URL as well; however this ability will not be able to go over the scripts hardcoded 50k limit.

Also, if you've got a suggestion for expansion, please let me know, I'm always looking at new ideas.

If you're looking to upgrade, please email me (james[at]virtualnightclub[dot]net) with your licensed url's and I'll send off the new files.

If you're looking to purchase license please feel more than free to email me for more information.

If it could include custom pages, then I would buy it like a shot

Custom pages are out of scope for this (at least at this point), I COULD code it in but cost vs demand... There would have to be a way to add items into the map and since the current manner just auto-pulls data from the database there is no real "need" for the ability to add items (short of posting more content to your forums)

I do however have plans to write a separate sitemap utility for custom/standard pages; this script since i have to hop through hoops to cater to limitations (filesize and total links) it would be difficult to factor in custom pages, the next product will gear just towards your "standard site" items.

You could always try adding them to the sitemap manually (by hacking it in) so long as you're sure you won't bust over the 50k links/page limitation.

Hello Gizmo,
Couple questions:
Can I 'select' which forums I want to include on the sitemap?
Like there are few forums that are more like 'chit-chat' and there is no need for them.

How is your upgrade police for your program versions and for UBB versions? I mean, if you upgrade your product, the registered users get it free? And how about if UBB upgrades?

Thank you

Q. Can I 'select' which forums I want to include on the sitemap?
A. No, it takes any "public" forum and creates a map for posts in it.

There is a valid reason for this; either way google will eventually crawl all data, a sitemap just helps it along; either way the data will be there (sooner or later).

Q. How is your upgrade police for your program versions and for UBB versions
A. Upgrades, at this point, are free of charge to valid license holders; I upgrade as the UBB upgrades (for example, as the page variable was redundant, it was also purged in 0.3). The upgrade pricing may change in the future, but at this point I have no plans of charging for upgrades.

To recieve an upgrade to the UBB.Sitemapper, all you need to is email me after there is one available and I'll email it back; however I do request that when emailing me you also email the site that you've purchased it for so i can do internal auditing (for a possible download manager in the future).

Upgraded and working fine

Cheers

Glad to hear it Mark

Anyone try to make a literal site map for the forums on the board. Like a page that shows the forums, subs, subs in order on a flat summary like table?

Here's what I mean only I mean coming down instead of sideways, but sideways could work. (not my website).

http://www.darkobessions-rpg.com/sitemap.htm