Net information reapers

ONE OF the appeals of the Internet is the huge information base that it possesses. After locating the resource a major task for a Net user is to bring the resource to his/her PC. Downloading and storing files from the Internet is a skill that every Internet user could do with. This week we will explore some utilities that help us reap the Net resources efficiently.

Webreaper

A website usually contains many local pages and external links. When we browse a site, what is being viewed is just one of the many pages that constitute the site. To access the whole site, we need to click at the various links displayed on the current page.

If you want to store the site for offline viewing, just saving the individual pages may not be enough as the site files will not be saved with all their proper connections. There is now a useful utility that can fetch all the files and links that constitute a site and stores on our hard disk the complete site files in a directory structure that is exactly similar to the original one. Webreaper is the name of this utility.

You just have to invoke the program and enter the address of the site you want to download in the web address box. Once you enter the address, the system automatically starts downloading the web page, the pages it links to, the graphics and other files related to the site and organises all the files in a proper site folder. The downloaded site can be either kept in the cache or can be stored in a directory specified by you. The smartness of the utility is that the whole downloading activity goes on rather transparently in the background, while you continue to browse the Net.

How to use the utility?

Download the free utility from the site: http://www.otway.com/webreaper/download.html. After installation, you will get the program's quite user-friendly interface. Customise the utility so that you can change some of the parameters in a way that will be suitable to your needs. For example, if you want the site to be stored in the directory 'web' on your hard disk, invoke the configuration menu by clicking at the options button in the file menu. Click at the save button and enable the option Save copies of downloaded files locally.

Type in your directory name (example: C:\web) at the location box.

When you enter the address of a site in the address box, the program reads the initial page first, then the links in the page and finally collects the link pages. Another useful option is the crawl depth, which determines how deep the program is permitted to look for links. A link referred to by a page is called a child link and each child link represents a one level crawl depth.

Zip the website

If you want to zip a web page or an entire site and preserve its graphics and active links, you can use the utility 'zip up the web'. By typing the URL into the zip up the web user interface, you can grab the page directly from the web as an executable file. By zipping pages this way you keep all the related files in one concise file and later you can run this executable file for off line viewing of the site.

Since this utility allows you to pack all the resources that constitute a website (home page, link pages, graphics) into a single executable program file, you can share a complete website with your friend/colleagues by simply sending this file as an attachment. An evaluation version of the package can be downloaded from the site http://www.zipuptheweb.com.

Transparent file downloaders

A major problem faced by dial-up users in downloading files is the disconnection of the lines. Once I was downloading a large file from the Net. After an hour or so about 80 per cent of the file got downloaded before my line was cut off. I had to start the downloading process afresh and get the entire file all over again. Now many programs are available, which help us get around difficulties of this kind. Get Right (http://www.getright.com), Netzip (http://www.netzip.com) and GoZilla (http://www.gozilla.com) are examples of such programs. Most of them are shareware programs.

An illustration with GetRight

Download the GetRight program from the site http://www.getright.com. Once the program has been installed, its icon will be displayed in the system tray icon. When you enter the address of the resource in the browser window or click at the resource link, GetRight immediately takes over and starts the downloading activity.

The program recovers and resumes after many errors (including disconnection) that happen while downloading files. But there is one hitch - the server in which your resource is located should allow a resuming operation. If there is an interuption, using the program you need to get only the remaining part of a file when you resume your downloading operation. So this way even very big files can be downloaded over a period of time without worrying about the line being disconnected.

J. Murali

(The author can be contacted at: murali27@satyam.net.in)

Send this article to Friends by E-Mail

Section  : Business
Previous : Corrosion within SAIL
Next     : Agreements need an update