***************************************************************************
*                                                                         *
*  Project: Networking                                                    *
*                                                                         *
*  Module:  URL                                                           *
*                                                                         *
*  Created: Wed 06-Mar-96      By: Kevin Bracey                           *
*                                                                         *
*  First version: 0.12                                                    *
*                                                                         *
*    Copyright:    (C) 1996, Acorn Computers Ltd., Cambridge, England.    *
*                                                                         *
***************************************************************************

Purpose:
========

Generic fetcher for Universal Resource Locators.

***************************************************************************


Change Log:
===========


---------------------------------------------------------------------------

Version: 0.12 Wed 06-Mar-96        Kevin Bracey                  

Checked in to source filer.

---------------------------------------------------------------------------

Version: 0.13 Tue 19-Mar-96        Paul Wain                     

Fixed the longstanding bug whereby if HTTP or any other proxying
was enabled, the URL module would not forward on unproxied requests
to the requested protocol module. (e.g. enabling http proxying
cause file URLs to be unusable, as the URL module would claim that
the protocol module was unavaliable). 

Also tidied up c.readdata so that the compile time warning about
an unused variable is zapped.

Finally incremented version number in cmhg file.

---------------------------------------------------------------------------

Version: 0.15 Tue 20-May-97        Kevin Bracey                  

Don't ask me, I've forgotten. It was nearly a year ago.

---------------------------------------------------------------------------

Version: 0.16 Thu 05-Jun-97        Carl Elkins                   

Enabled 'GetURL' SWI to return errors from the fetcher to the client.
This enables better handling of bad URL/hostname and retry after autodial.
---------------------------------------------------------------------------

Version: 0.17 Tue 21-Oct-97        Stewart Brodie                

No longer attempts to terminate protocol module descriptive strings at
50 characters.


---------------------------------------------------------------------------

Version: 0.18 Wed 12-Nov-97        Stewart Brodie                

Uses a service call to inform the protocol modules that it is arriving or
dying so that they can re-register automatically if the URL module arrives
after the protocol modules.  (Requires: File 0.29, FTP 0.20, Gopher 0.06,
HTTP 0.57 or later for this auto-registering to work)

Moved all the error mesages to the Messages file in ResourceFS.

SWIs at offsets 32 and 33 now have proper names.


---------------------------------------------------------------------------

Version: 0.19 Thu 13-Nov-97        Stewart Brodie                

Version 0.18 lost the ability to set proxy information for individual
requests.  This is now restored.


---------------------------------------------------------------------------

Version: 0.20 Thu 27-Nov-97        Stewart Brodie                

Contains new enumeration SWI (URL_EnumerateSchemes) and general URL
processing SWI (URL_ParseURL).  The former is thought to be stable, the
latter is definitely beta status currently.  HTTP 0.60 uses this SWI.  Other
modules/applications may do so too soon.

This version contains two main test programs - one to test the enumeration
SWI and the other to test the URL parser.  It is supplied with two scripts
in which the two input URLs (a base and an optional relative) are given, and
then the expected 11 component answers are given.  The test program calls URL
to process the URLs and checks the answers, and will show any failures.

---------------------------------------------------------------------------

Version: 0.21 Mon 01-Dec-97        Stewart Brodie                

Now canonicalises URLs before looking them up in the proxy tables and
checking for handlers.  This is a partial fix for PAN-00988.

This version issues a service call when a protocol module registers or
deregisters, so related interested modules (eg. AcornURI) can keep track of
the status of URL.

Modification to URL_ProtocolRegister to allow a flags word to be given in R5
(if R0:0 is set) indicating URL style for that scheme.

Fix to spelling of E01 message in Messages file (cound => could)


---------------------------------------------------------------------------

Version: 0.22 Thu 04-Dec-97        Stewart Brodie                

Better trace and better code in geturl proxy management code.

Relinked with version 5.06 of the linker (solves ReInit data aborts)
Specific build for December 4 development release.

---------------------------------------------------------------------------

Version: 0.23 Mon 08-Dec-97        Stewart Brodie                

Fix to URL resolver so it doesn't omit query and fragment strings frm
resolved URLs.  (Needed to fetch Dilbert as of today ;)


---------------------------------------------------------------------------

Version: 0.24 Mon 08-Dec-97        Stewart Brodie                

More resolver fixes.  New test cases added to Tests.URLs.

---------------------------------------------------------------------------

Version: 0.25 Wed 10-Dec-97        Stewart Brodie                

Allows "!" in scheme names - required by browser's internal scheme.


---------------------------------------------------------------------------

Version: 0.26 Wed 17-Dec-97        Stewart Brodie                

Multiple proxy settings no longer cause an abort.
Specific build for December 18 development release.
---------------------------------------------------------------------------

Version: 0.27 Mon 05-Jan-98        Stewart Brodie                

Replaced sprintf's with len_strcpy for the case where we are just copying
strings.   This seems to have increased the speed of URL_ParseURL by 7-8%!

No longer requires protocol modules to be loaded in order to determine the
URL format attributes.  (URL 0.26 failed the FileURLs test cases if the
FileFetcher module was not loaded)

New reason code for URL_ParseURL.  This is used exclusively for resolving
relative URLs in one SWI call - caller must provide a buffer in R4, buffer
size in R5 and the URL module will fill in the buffer, and indicate the bytes
used count by subtracting the total length (inc \0 terminator) from R5 on
exit.  In order to be sure that the buffer is long enough, the length
supplied by the client should be "length(base) + length(rel) + 4", where
length(NULL) is zero and length(non-NULL) is the length of the string pointed
to, as would be returned by the standard C function strlen.

Resolution of fully specified relative URL in netloc-less schemes failed as a
consequence of a RFC deviation I'd implemented in order to handle (illegal
but used) stuff like "http:/just/the/path/no/host".  Deviation is retained
but only for protocols which have no <net_loc> fields in them.  (URL knows
which do, and new modules should use R0:1 and R5 to tell URL about any newer
schemes.

Added new speedtest test program.  This compares the speed of the new reason
code mentioned above against the previous way of resolving URLs.  It shows
that using the new code halves the time taken to resolve URLs.

---------------------------------------------------------------------------

Version: 0.28 Thu 08-Jan-98        Stewart Brodie                

Uses a single RMA block for temporary buffers during resolving instead of
calling malloc/free for every little block it needs.  URL resolution is now
performed 35 (yes, THIRTY FIVE) times faster.

Uses new method of communicating proxy information to HTTP.  Requires HTTP
0.68 or later for correct operation.  This makes URL faster, more memory
efficient (both in code size and run-time requirements) and more reliable.

Quite a few more new test cases have been added to Tests.URLs to catch
reported failures which have now been fixed (which we don't want coming back
again).


---------------------------------------------------------------------------

Version: 0.29 Mon 19-Jan-98        Stewart Brodie                

When issuing "No fetch in progress for this client ID" errors, the client ID
that the client quoted is appended to the message (to assist debugging
Browse to find out why it is calling URL after terminating a fetch).

Fix to URL resolver.  More test cases added to the URL test file covering
those cases (specifically too many ../ sequences)

---------------------------------------------------------------------------

Version: 0.30 Thu 29-Jan-98        Stewart Brodie                

I'd "released" version 0.29 with "ADH debug build #1" in the module help
string.  This version is a specific January 30th Expresso beta release.
