Putting it out there can get you into trouble. Not only is there "publisher's remorse" but also the more serious take-down notice that may crash into your client's inbox from time to time claiming that the content of their web page has infringed one of the sender's rights. It's easy enough if the client owns the site to eliminate the offending material or whole pages; that's why delete buttons were made. But Google is not so easily deterred. Having indexed material it may continue to serve up links to that material, if only in its cache; and its bots may continue to worm their way into material that the client thought had been successfully hidden.
What to do? Google's not exactly the corner store, where you can drop in and have a decent chat with the owner, and on their sites you'll notice a distinct lack of email addresses or phone numbers you can use to send them your problems.
Google knows it's a juggernaut (fun etymology in this case: "lord of the universe"). And eventually it provides.
Google's Webmaster Central Blog has released the first in a series of entries showing us how to pull our coals out of the fire: "URL removal explained, Part I: URLs & directories." I'm not going to try to reproduce their instructions here: it's not an occasion for approximation. But I can say that getting them to remove a URL involves essentially two steps: blocking their bots correctly, and then submitting the URL to their removal tool.
And because we sometimes make mistake upon mistake — or, more happily, you were able to defeat the take down notice — the client can undo the removal and re-instate the URL.
To use the Google URL removal feature you can either block the page with robots.txt or remove it. It may take a day or so for the page to be removed from the search index and it will likely continue to be shown by other search engines, however.
Perhaps the best way to play it safe is to add a "noarchive" meta tag before publishing the content. The search engines will still show the page in search results but will not make a cached page available. You can then make the undesired content disappear (from the most common sources at least) the moment you remove the page or change its contents.
Thanks, Lawnix. Best, though, to read the Google page for the exact way to do it.
Thanks for this link. I'll be looking for the sequels (I'm assuming there's at least a Part II).
Still seems like removal is only possible through compliance of the domain owner, which still doesn't address SEO and reputation issues.
What I'm interested in seeing is if a court can compel a domain owner to block bots. Does anyone know if this has happened before?