TWebCompress v2.0 -- Remote Site Administration Component

(c) 1997-8 South Pacific Information Services Ltd
software@spis.co.nz

=== Purpose:

TWebCompress allows Web site applications to be remotely requested to either:

a) Deploy (decompress) a set of site update files which have been uploaded in one or more archives

OR

b)  Gather (compress) one or more files matching supplied file specifications into a compressed archive, ready for downloading or emailing

During these processes, full verification of archive contents are carried out, and event hooks are provided for other suspend/refresh requirements -- such as closing tables or other files which might be updated or gathered by the process.

=== Other features:

* Can be used to support multiple sites

* Can be remotely controlled via FTP using "signal" and report files (the WebDeply sample application does this)

* Optional automatic renaming of EXE/DLL/OCX files on upload so that they don't clash with running applications with the same names.  The running apps must then be manually shut down, then the file renaming done via FTP before the apps are re-run.

* Each uploaded archive is fully verified for safe expansion (i.e. no transmission errors, no file access problems) BEFORE any files are overwritten. This is vital when a set of files must update the site "as a unit"

* Fully configurable for location of uploads, signal/report files, "gather" archive etc. This configuration can be changed from FTP control files or via the local UI.

* The Gather function can be completely data-driven (e.g. from your app) if desired, and thus be used to build ad-hoc LHA-compatible archives of particular files or sets of files.

* Security is implemented at a per-site level, so that each site is permitted to access its (and only its) directory tree

* Encryption of archives going in either direction is supported

=== Requirements:

* TWebCompress requires a copy of the TCompLHA component for Delphi (http://www.spis.co.nz/compLHA.htm). An unregistered copy can be used for implementation and testing, but a registered copy would be needed for a live site.

* Our  WEBQUEUE application (http://www.spis.co.nz/webcentr/webqueue.htm)
is also highly desirable to have. This application scans your off-site mirror directories and uses "last updated" times to create archives containing the changed files for uploading to the site. It will also perform the FTP upload itself if required, requesting TWebCompress to do the Deploy (or Gather) operation, and reporting on the results.  Instead of using WebQueue, you could craft your own application using TCompLHA or simply use the popular freeware LHA utility to create your archives.

=== Key properties of the TWebCompress component:

* CompLHA -- the TCompLHA component instance on this form which provides the compression services.

* Sites -- a list of site names -- whether WebHub or non-WebHub sites. All upload/download location properties are stored "per site"

* Site -- the name of the currently selected site (must be in the list above)

* UserSignalFileName -- if set, the component will look for this signal file
in all sites' uploadpaths and "fire" the OnUserSignal event if you've hooked it.

* UploadPath -- the location where the archive(s) and signal files (if any) will be uploaded for the current site. Also used for the STATUS.TXT file storage after a Deploy or Gather.

* OriginPath -- if specified, TWebCompress will ensure that all files expanded onto or gathered from this site have paths in or below this path.  For example, of OriginPath is "c:\mysite" then a file  from the archive called  "\here\there\x.txt"  will NOT be expanded, but a file called \mysite\here\there\x.txt" WILL be expanded. Thus, the pathnames stored in the archives need to be properly formed before the files will be expanded.

Just as importantly, when gathering files, the filenames stored in the archive will be relative to OriginPath (e.g. "\mysite\here\there\x.txt" will be stored as "here\there\x.txt"), which is important when expanding the files at the offsite end, especially using WebQueue.

Note: If OriginPath is NOT specified, archives being expanded will have a (TCompLHA) TargetPath set to the path holding the archive (usually the UploadPath).

* Password -- the optional encryption/decryption password for this site. Use this only if you are using WebQueue as an off-site tool and have the same password set for it.

* RenameEXEs -- set True if you want deployed filenames ending in EXE, DLL or OCX renamed to .EX_,  .DL_ and .OC_ respectively to avoid clashing with running applications

* Status -- stringlist of messages describing the last Deploy or Gather operation. This information can be accessed from HTML if desired, e.g. 
%=WebCompress.Status=%. It is also stored in a file called STATUS.TXT in the current upload path.

* GatherList -- a list of file specifications describing the files to be compressed in the next Gather operation for this site. Wild cards and full paths are supported, e.g. d:\ht\data\logs.*  Can also specify in the form "c:\here\*.*/S" to recurse into subdirectories below "here".

* Responsive. Only important if the FTP/signal approach is used to control the component. See the ApplicationIdle event below.

=== Key events of the TWebCompress component:

* OnDeploy -- called with a list of all the archive files found in the UploadPath directory. You can optionally amend the list, even adding archives which are in another path, or removing archives you don't want to process right now. The AppID property can be set and interrogated if necessary to ensure that any operations you do are for the correct site. 

* AfterDeploy -- called with the total number of archives which were deployed (and the list of their filenames again). This is the appropriate place to refresh your application so that it reflects any changed chunks etc.

For example, you might NOT want to refresh your main application after deploying if the archives are in an upload directory which is only for updating the "ABC" application, or another, static, client site.

* OnExpand -- called just before a given archive is decompressed -- set its filename parameter to blank to skip it.

* AfterExpand -- called just after a given archive was decompressed, passing the filename, a list of files decompressed, and the current list of status messages (you can add to it if desired).  Note that each archive file is deleted after it has been successfully Expanded.

* AfterGather -- called just after a Gather operation, with a list of the files which were compressed

* ApplicationIdle -- you would normally want to control the component via FTP
signals, in which case this special event HANDLER should be assigned in your application (see notes below). This will ensure that the WebCompress component is called from time to time  -- say, every 15-30 seconds or so -- PROVIDED the application isn't busy. It will use this event to check for the signal files (see "Controlling WebCompress via FTP signal files"). Note that if the Responsive property is True, the average response time is usually much quicker than 30 seconds, BUT, the CPU usage of the app increases significantly, so it is not recommended.

=== TCompLHA Component Properties

Certain properties of this component are automatically set, as follows:

ArchiveName (always Gather.LZH for Gather operations),  Confirm (False), FilesToProcess, ExceptionOnFileError (True)

You may also wish to consider setting the following after checking their specifications in the TCompLHA help:

* For Deploy:
   MakeDirectories (recommended True)

* For Gather:
    StorePath (STRONGLY recommended to be False unless you are quite sure the resulting you download *won't* be expanded over the top of existing local files which you want to keep!)
    CompressionMethod (recommended colh5, i.e. best/fastest compression)
    ArchiveType (recommended caLHA = LHA compatible)

*  For both:
    TargetPath (recommended '', i.e. empty -- but it is set by WebCompress when     expanding archives)
    Regname/RegNumber (TCompLHA registration information)
    Password (if set, this must be consistent both at the site and offline -- note that it renders the archives non-LHA compatible, as LHA itself doesn't support encryption)

* If the OnCheckFile event of the TCompLHA component is set, it will be called -- AFTER any RenameEXEs-controlled handling is done by the TWebCompress component.

=== Using TWebCompress

1. Drop a TWebCompress and TCompLHA component on a form in your Web application

2. Set the "additional" TCompLHA properties as noted above

3. Set the CompLHA and IniFileName properties of the TWebCompress component as noted above

4. Write handlers for any of the TCompLHA or TWebCompress events you want to hook in order to provide special handling (e.g. emailing of a new archive in the AfterGather event). See the example handlers in the WebDeply (wcsitef.pas) sample application -- most of these have been implemented to ensure seamless updating of the INI file when the form fields are altered, and vice versa.

6. Edit the supplied WebDeply.ini file so that it refers to the component name
you are using (e.g. WebCompress1 instead of WebCompress) and your site
information in the relevant fields. Ensure it has the same name/path as the
WebCompress.IniFileName property you've set.

7. Since you almost certainly want to control this component via FTP signal files:
a) In FormCreate, set Application.OnIdle := WebCompress1.ApplicationIdle;
b) Call WebCompress1.Update to get it initialised to start with
c) See "Controlling WebCompress via FTP signal files"

8. IF you want to use the Deploy function:

a) Optionally set the RenameEXEs property to True

b) Place one or more archives named using the form "yy-mm-dd.0nn" in an upload path (e.g. 97-08-22.001). This naming convention is important because it allows a "multiple archive" case to be nicely handled -- they will be sorted and processed in the correct order (oldest first).

c) Upload (or create) a file called "Deploy.sig" in the upload path -- this
is recognised by the application which will then deploy the archive(s). If
you want to invoke a deploy manually, call WebCompress1.Deploy in
a buttonclick event.

d) View the results by looking at the contents of Status.txt in the upload path (or the component's Status property) after the deploy.

Deploying is usually a very quick operation.

9. IF you want to use the Gather function:

a) Edit the GatherList section for the current site in the INI file to specify a set of filenames to be processed, e.g.
1=c:\sites\yoursitet\*.log/S       (the /S means search subdirectories too)
2=d:\data\*.db    (etc)

c) Upload (or create) a file called "Gather.sig" in the upload path -- this
is recognised by the application which will then gather the files. If
you want to invoke a gather manually, call WebCompress1.Gather in
a buttonclick event.

d) View the results by looking at the contents of Status.txt in the upload path (or the component's Status property) after the gather, and download or view the Gather.LZH archive created in the upload path.

10. IF you want to change the properties for a given site, do this:

a) Upload an amended copy of the INI file to its usual location
b) Upload a file called Refresh.sig to any site's Upload Path  -- once it has
been noticed, WebCompress will use the new settings from the INI file.

=== Sample Application

The WebDeply application is designed to show you the kind of logic you
may wish to use to implement an application with TWebCompress.
It has quite a lot of logic which is mainly related to user interface handling.
If you just want to control your application remotely, you can dispense with
most of it and just set the OnIdle hook and other essential properties.

See WebDeply.txt for operating instructions. To compile the application:

a) Make sure TWebCompress and TCompLHA are both on your Delphi palette (see installation instructions in README.TXT)

b) Load WebDeply.dpr and ensure WebDeply.ini is correct, per the operating instructions

c) Run the app and test property setting via GUI or FTP control

e) Keep the app running, and control it with WebQueue via an uploads directory path which is accessible to your FTP server

=== Controlling WebCompress via FTP signal files:

If FTP-based signalling is in effect, place one of the following files in an UploadPath to invoke either a Deploy or Gather operation:

-- DEPLOY.SIG  for Deploy
-- GATHER.SIG  for Gather
also: -- REFRESH.SIG for Refresh (reload the INI file settings)

The signal file will be deleted as soon as the operation starts, and a status report (called STATUS.TXT) will be placed there when the operation is completed.

The GATHER signal file can optionally contain revised GatherList information as a simple list of file specifications. These will override existing GatherList settings for the component for this Site (and will rewrite the settings stored in the application's INI file). If the Gather signal file is empty, the current/default settings are used.

Note that WebQUEUE uses the above signalling/reporting approach to automate its interaction with any site which has a WebCompress-enabled application running. Thus, you don't need to bother with any of the above details if you are using WebQueue. However, you may find it interesting to watch the changing upload directory contents "as it happens".


Licensing restrictions:
The full source and documentation of TWebCompress (and its sample
application) may be used and adapted  provided: 
*  It is not on-sold in source, DLL or object form to other Web developers without explicit permission from  SPIS Ltd. 
*  It is only deployed for operational use with a registered version of our TCompLHA component.
* If you choose to use our SPIS WebQueue application for off-site operations (e.g. providing it to your  clients), you or your client(s) register each copy of WebQueue which is used for such purposes. 
