Today I had someone ask me how to keep bots off of their forum. They were seeing a noticed slowdown in the site when it was being indexed. That and they don’t want their “crazy cat lady talk” indexed on Google.
So here goes: if you want to keep Google off of your site - the easiest way is to add a file called robots.txt to the root directory of your website:
An example to disallow all robots would be:
User-agent: *
Disallow: /
With those 2 lines, all robots (including Google) will leave your site entirely alone.
If you wanted to disallow Google, but allow all others (like Yahoo, etc)
User-agent: Google
Disallow: /
User-agent: *
Disallow:
Everything you never wanted to know about the use of the robots.txt file is available at: http://www.robotstxt.org.
The best thing to keep in mind is that the “bad” bots will ignore robots.txt with wild abandon - doing their best to leave spammy comments for shoddy websites. For those bad boys you’ll need to block them by IP address, something we’ll address another day.
February 29, 2008 – 6:57 am
I’ve been very impressed with what I’ve been learning about the recently introduced Intel Modular Server as late. With the ability to house six server blades and fourteen Serial Attached SCSI drives it’s a datacenter in a 6U box (or horizontal pedestal if you prefer).
The management software is intuitive and easy to use, and provides remote, end-to-end management. It works through a web browser (IE and Firefox), yet feels like you’re standing right in front of the rack. The same interface is used for everything from storage, KVM, and power management.
The designers obviously had the middle-market in mind, because this offering sidesteps all of the issues have made implementing a blade solution in a SMB situation problematic. The system runs on either 110 or 240V AC, eliminating the need for a costly 220-240 UPS upgrade. At 70 dB maximum sound output, I wouldn’t want it under my desk, but it wouldn’t overwhelm an office. Running at a fairly cool 75 degrees, it won’t overwhelm the cooling ability of a small equipment room. The availability of the chassis as a pedestal base for situations where there is not a rack available is a first in the blade market, as far as I’m aware.
If the Modular Server had an A/C module, it would truly be a data center in a box, providing everything a small business needed except an Internet connection. I’m interested to see how it fares against HP’s c-Class blade system, or IBM’s BladeCenter S, which have similar capabilities. With it’s significantly lower cost than HP and IBM, it’s sure to hit the sweet spot for many SMB customers.
December 19, 2007 – 10:28 pm
Hello, my name is T.J. and I’m an affiliate program failure.
If you look at my latest earning reports, you can see that I’ve been making slow but steady progress at increasing my websites’ revenue through Google Adsense and eBook sales. I seem to be coming up short in affiliate programs, though. I’ve been running a variety of Commission Junction ads on a couple of our sites for just under a year now. In this time, there have been 222,059 impressions but only 276 clicks - for a CTR of 0.12% which is pathetic in itself. Even worse is that not one of those 276 clicks converted into a referral or sale so I’ve made a grand total of $0 for the entire year on CJ. I’m not sure if the advertisers I’ve been choosing aren’t a good fit for the websites they are on, or if I’m missing even more basic than that. I’m completely at a loss why my site visitors will click on AdSense ads for a topic on my sites, but completely ignore very similar affiliate ads in the same location.
Now you may be thinking - “there’s more affiliate programs than CJ, T.J.”…. Sure there are. Google has an affiliate advertising add-on to Adsense. Can you guess how much I’ve made off of them? If you guessed $0, you’re right! Last month I made I applied for several others: and they all told me the same thing: denied. Even WalMart!
I will admit that I’ve done a little better with WidgetBucks and Amazon averaging about $2.00 and $1.25 a month respectively. With WidgetBucks‘ $25 sign-up bonus, and a $50 minimum payout, I’m only 10 months away from my first payout, and two years after that until my 2nd payout. I think I’d be doing much better if I had a site that meshed better with one of the categories. I’ve got some sites in the works that may bring the monthly total up significantly. Amazon makes it a little easier to get payouts if you’re willing to take it in Amazon credit. The threshold there is a measly $10, meaning I’ll get paid in another 7 months.
Don’t get me wrong - I know there is money to be made in the affiliate business. John Chow made $6,579.67 from it last month, and it was his #2 moneymaker after private ad sales. I just think that the site needs to align better with an in demand big-ticket affiliate product. Web hosting, advertising networks, and expensive hardware and software all seem to lend themselves to the affiliate game than cute babies or gluten free recipes. I would imagine a site about AutoCAD or PhotoShop could make a pretty penny with an Amazon affiliate link to those high-priced products. John Chow obviously gets affiliate bonuses for people who sign up for advertising networks using his affiliate links. I’d love to have a site about Plasma TVs, but that market is a little saturated. I need a great product that is in demand, high-priced, under-represented online and has an affiliate program with high% payouts. I wonder if anyone has an affiliate program for dentures?
December 14, 2007 – 8:08 am
No, not me! I’m quoting my favorite author, Terry Pratchett who recently revealed that he’s suffering from a form of early Alzheimer’s disease. He seems to be taking it pretty well saying:
Frankly, I would prefer it if people kept things cheerful - I think there’s time for at least a few more books yet
I hope he’s right, because:
a) I love taking little “mini vacations” to the Discworld (by reading his books), and without him the place just won’t be the same.
b) I have given my wife the latest Pratchett book as a Christmas present every year for as long as I can remember. My kids will ‘get’ her jewelery and clothes and bath and body lotions, but my official present has been a Discworld book. It says a lot about an author that he’s that reliable in chunking out a fairly large novel every year, and they continue to be hilariously well written. It will kind of be like Old Faithful shutting down once Mr. Pratchett is done writing.
I guess all the fans can do is keep the same positive outlook as their favorite author:
I will, of course, be dead at some future point, as will everybody else. For me, this maybe further off than you think.
I hope you’re right Mr. Pratchett!
December 12, 2007 – 7:20 pm
When I logged on to post an update today, I was greeted with the error message:
Allowed memory size of 8388608 bytes exhausted (tried to allocate 208834 bytes) in /public_html/*
(where * represented the path to any file in the wp-admin portion of one of my blogs). The content/end-user portion of the blogs were showing up just fine, I just couldn’t write any new content or make changes.
My initial research said that I would need to get my hosting company to increase the memory allocation of PHP on the server, or add the line ini_set("memory_limit","16M"); to any offending .php file. Since my hosting company is takes days to get to any problems, and I was having this issue with every .php file in the /wp-admin/ directory, these seemed like non-solutions. When I came across the idea of adding php_value memory_limit 32000000 to my .htaccess file, my hopes of a quick fix were restored. I added the line to the .htaccess file in the root of my website (/public_html) and the problem disappeared instantly, for all of my blogs hosted on this server. Hurray for quick, easy fixes!