Author Topic: /robots.txt  (Read 4932 times)

Offline hidden

  • Mr Fred!
  • Junior Member
  • **
  • Posts: 90
/robots.txt
« on: October 18, 2007, 02:32:19 PM »
Anyone found a way to add robots.txt files into a csb page upload?

Offline hidden

  • Sami
  • Administrator
  • Senior Member
  • *****
  • Posts: 5924
  • Not a geek. Just a Nerd.
    • CSB Tutorials
Re: /robots.txt
« Reply #1 on: October 18, 2007, 03:01:10 PM »
robots.txt file MUST MUST MUST be uploaded directly to the MAIN DIRECTORY of your website.

CSB can NOT be used for that. :noshake:
CSB would put the file into a folder on your website which defeats the purpose for having it on your site.
Use FTP or upload feature in your file manager at your webhost control panel.


Come on Fred....FTP is our FRIEND.  Easy and fast.  And free. 

Try COREFTP Lite.  Simple as dragging files from one side of the window to the other! 
Quote
Now you can download Core FTP LE - free Windows software that includes the client FTP features you need.  Features like SFTP (SSH), SSL, TLS, IDN, browser integration, site to site transfers, FTP transfer resume, drag and drop support, file viewing & editing, firewall support, custom commands, FTP URL parsing, command line transfers, filters, and much, much more!

This free, secure FTP client gives you a fast, easy, reliable way to update and maintain your website via FTP.  It also provides a secure method (via SSL, TLS, or SFTP) to upload / download files to and from FTP servers.  Check out the list of features and you'll find almost every feature you need, all in a free ftp program.   There are no popup ads, advertising or spyware and you're never asked or reminded to register.

For more features, you need to purchase but the FREE one will handle just about everything YOU would want.
« Last Edit: October 18, 2007, 03:24:28 PM by Samrc »
-Samantha
TNG: "Sometimes, you can make no mistakes, do everything right, and still lose" - Capt Picard to Data
(:turtle: In memory of Turtle: May 22, 1944 - Nov 24, 2007  GURU, mentor, and really nice guy! :turtleleft: )

Offline hidden

  • Mr Fred!
  • Junior Member
  • **
  • Posts: 90
Re: /robots.txt
« Reply #2 on: October 18, 2007, 03:21:31 PM »
Is there one script that's better than another?

Offline hidden

  • Sami
  • Administrator
  • Senior Member
  • *****
  • Posts: 5924
  • Not a geek. Just a Nerd.
    • CSB Tutorials
Re: /robots.txt
« Reply #3 on: October 18, 2007, 03:25:07 PM »
Script for what???

robots.txt is standard language, no script involved.
-Samantha
TNG: "Sometimes, you can make no mistakes, do everything right, and still lose" - Capt Picard to Data
(:turtle: In memory of Turtle: May 22, 1944 - Nov 24, 2007  GURU, mentor, and really nice guy! :turtleleft: )

Offline hidden

  • Mr Fred!
  • Junior Member
  • **
  • Posts: 90
Re: /robots.txt
« Reply #4 on: October 19, 2007, 03:48:30 PM »
Any idea what it should say to maximize bots scanning a website?

Offline hidden

  • Global Moderator
  • Senior Member
  • *****
  • Posts: 1497
  • I Dare to Dream!
    • InspirationMotivation.com
Re: /robots.txt
« Reply #5 on: October 19, 2007, 04:40:36 PM »
here is a really easy to use robot text generator - you don't even have to think about it. lol mcanerin.com/EN/search-engine/robots-txt.asp
Success is a way of life found moment by moment

Offline hidden

  • Mr Fred!
  • Junior Member
  • **
  • Posts: 90
Re: /robots.txt
« Reply #6 on: October 19, 2007, 04:45:29 PM »
is it put in using insert html? should it be on the index page or all pages?

Thanx.. that's what i needed.

Offline hidden

  • Sami
  • Administrator
  • Senior Member
  • *****
  • Posts: 5924
  • Not a geek. Just a Nerd.
    • CSB Tutorials
Re: /robots.txt
« Reply #7 on: October 19, 2007, 06:56:58 PM »
Quote
is it put in using insert html? should it be on the index page or all pages?

already answered above:
Quote
robots.txt file MUST MUST MUST be uploaded directly to the MAIN DIRECTORY of your website.

CSB can NOT be used for that.
CSB would put the file into a folder on your website which defeats the purpose for having it on your site.
Use FTP or upload feature in your file manager at your webhost control panel.
-Samantha
TNG: "Sometimes, you can make no mistakes, do everything right, and still lose" - Capt Picard to Data
(:turtle: In memory of Turtle: May 22, 1944 - Nov 24, 2007  GURU, mentor, and really nice guy! :turtleleft: )

Offline hidden

  • Global Moderator
  • Senior Member
  • *****
  • Posts: 1497
  • I Dare to Dream!
    • InspirationMotivation.com
Re: /robots.txt
« Reply #8 on: October 19, 2007, 08:53:42 PM »
Pirate you may also be thinking of the robot meta tag that can be used on a page by page basis - really you should use both in my opinion. Here is one of the briefer sites to pick up that tag. Searching robot meta tag will get you tons more. robotstxt.org/wc/meta-user.html
Success is a way of life found moment by moment