VisualVision: professional content publishing software - Web site, self installing CD DVD, catalog, protect ebook, autoplay CD brochure, HTML Help, manual, guide, reference, database import CSV

Welcome, Guest. Please Login or Register.
Dec 15th, 2017, 3:12am

(this forum was - and actually still is - here)

Home Home Help Help Search Search Members Members Login Login Register Register
Visual Vision **User's** Forum (the fast) « robots.text file robots.txt »



Français (Google)
software free download
Español (Google)
software free download
Italiano
 
create Website1SITE: create a business or personal Website with ease...
software support
manuals, documentation on CDCDFrontEnd: distribute documentation, CD brochures & CD business cards; protect CD (Fra, Ita)
software free download
create Web CD catalogHyperPublish: create Website, product catalog (also from DB CSV), multimedia CD... (Ita)
software free download
create ebookEBooksWriter: edit, pack and protect smart ebooks... (Esp, Ita)
cd software
create manualPaperKiller: create a manual, hypertext, manage electronic documentation
software free download
software documentationRoboAuthor: make software helpfile and documentation .CHM .HLP HTML Help
software free download
host hostingVisionHost: web space hosting for VV customers
software free download
create a Web siteEasyWebEditor: build your own Web site, create good Internet pages without code or "HTML"
software free download
Français (Google)
Créer une page Internet personnelle ou d'entreprise avec facilité; distribuer de la documentation, des brochures avec CD DVD à démarrage automatique / auto exécutable, protéger CD / DVD; logiciel pour catalogue de produits (database, CSV); modifier, emballer et protéger ebooks; créer un manuel, un hypertexte, gérer la documentation électronique; construire votre site Web, créer des pages...
software free download
Español (Google)
Crear un negocio o sitio web personal con facilidad; distribuir la documentación, manuales, folletos y CD / DVD tarjetas de visita; proteger CD; crear catálogo de productos (database, CSV), editar, empaquetar y proteger ebook; crear un manual, hipertexto, gestión de documentación electrónica y ayuda CHM y HLP; construir su propio sitio Web, crear páginas de Internet...
software free download
Search:  
Advanced search...

Max results:

   Visual Vision **User's** Forum (the fast)
   Our Products
   EasyWebEditor / 1site

   robots.text file robots.txt
« Previous topic | Next topic »
Pages: 1  Reply Reply, or ask/add more Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: robots.text file robots.txt  (Read 2536 times)
Jo-Ann
Guest

Email

robots.text file robots.txt
« on: May 12th, 2010, 1:46pm »
Quote Quote Modify Modify Remove Remove

When I add my site to search engines, one in particular shows this message:
 
This site does NOT provide the robots.txt file or the robots.txt file is not constructed properly. It is now recommended all sites provide this useful search engine file.  
 
What do I do? Is this necessary ? and how do I do this ?  
Your assistance is appreciated. Thank You !
« Last Edit: May 12th, 2010, 2:54pm by Support Staff » Logged
Support Staff
YaBB Administrator
*****



Meow!

   
WWW Email

Gender: male
Posts: 907
Re: robots.text file robots.txt
« Reply #1 on: May 12th, 2010, 3:06pm »
Quote Quote Modify Modify

frankly speaking, robots.txt (not .text, must be .txt) is almost useless nowadays.
 
it is a text file that you can create with any common text (.txt) file editor such as MSWindows Notepad (or similar software; you have Notepad as it is included in all Windows versions) and you may transfer with EWE / 1Site internal "Generic FTP" feature.
 
an example of this file is
 
User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /tmp/
Disallow: /private/
 
this file does tell the search engine robots that they must not go to the folder cgi-bin, and they also must ignore the folder images, tmp, private (this is just an example; you may not have these folders at all).
 
If in your web site, you like that the search engines can go everywhere, you can safely avoid putting a robot.txt file
 
Some robots supports also other directives such as (some examples)
 
Allow: /folder1/myfile.html
Allow: /folder1/foldertobeseen/
(e.g. you disabled folder1 but you want to enable a specific file or subfolder)
 
Crawl-delay: 10
(number of seconds between two successive crawls)
 
Visit-time: 0600-0845
(only visit between 06:00 and 08:45 UTC (GMT))
 
Logged
Jo-Ann
Guest

Email

Re: robots.text file robots.txt
« Reply #2 on: May 13th, 2010, 6:58am »
Quote Quote Modify Modify Remove Remove

Thanks for lesson on this topic. I appreciate it.
It worried me when I got that message ??
Logged
walterjackson
YaBB Newbie
*



I love Visual Vision!

   
WWW Email

Posts: 2
Re: robots.text file robots.txt
« Reply #3 on: Jan 5th, 2011, 11:15pm »
Quote Quote Modify Modify

hi, is there a way if i don't want my site to be crawled by spiders or bots?
Logged

vita mix blender | vitamix reviews
Pages: 1  Reply Reply, or ask/add more Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »


VisualVision main Web site
Easy software tools from Visual Vision:

Visual Vision **User's** Forum (the fast) » Powered by YaBB 1 Gold - SP1!
YaBB © 2000-2001, XNull. All Rights Reserved.
This forum © 2001-2010 VisualVision and VisualVision Users