TWHCompress -- Remote Administration WebAction component for WebHub 1.10+

(c) 1997-8 Peter Hyde, South Pacific Information Services Ltd

=== Purpose:

TWHCompress allows WebHub applications to be remotely requested to either:

a) Deploy (decompress) a set of site update files which have been uploaded in one or more archives

OR

b)  Gather (compress) one or more files matching supplied file specifications into a compressed archive, ready for downloading or emailing

During these processes, WebHub applications have their page handling suspended, and event hooks are provided for other suspend/refresh requirements -- such as closing tables or other files which might be updated or gathered by the process.

=== Other features:

* Can be used to support multiple sites, both WebHub and non-WebHub

* Can be remotely controlled and checked via Web pages (as a Webaction component) or via FTP using "signal" and report files (the WebQueue application does this)

* Optional automatic renaming of EXE/DLL/OCX files on upload so that they don't clash with running applications with the same names.  The running apps must then be manually shut down, then the file renaming done via FTP before the apps are re-run.

* Each uploaded archive is fully verified for safe expansion (i.e. no transmission errors, no file access problems) BEFORE any files are overwritten. This is vital when a set of files must update the site "as a unit"

* Fully configurable for location of uploads, signal/report files, "gather" archive etc. This configuration can be changed from HTML (optionally), from FTP control files, or via the local UI.

* The Gather function can be completely data-driven (e.g. from a Web page or your app) if desired, and thus be used to build ad-hoc LHA-compatible archives of particular files or sets of files.

* Security is implemented at a per-site level, so that each site is permitted to access its (and only its) directory tree

* Key control properties are also configurable via WebHub-style INI runtime property editing

* Encryption of archives going in either direction is supported

* Can be tested at design time using its right-click menu

* (Subject to improvements in WebHub:) A future version will temporarily deactivate ALL web applications on the server, not just the current one, during Deploy or Gather processes

=== Requirements:

* TWHCompress requires a copy of the TCompLHA component for Delphi (http://www.spis.co.nz/compLHA.htm). An unregistered copy can be used for implementation and testing, but a registered copy would be needed for a live site.

* Our  WEBQUEUE application (http://www.spis.co.nz/webcentr/webqueue.htm) 
is also highly desirable to have. This application scans your off-site mirror directories and uses "last updated" times to create archives containing the changed files for uploading to the site. It will also perform the FTP upload itself if required, requesting TWHCompress to do the Deploy (or Gather) operation, and reporting on the results.  Instead of using WebQueue, you could craft your own application using TCompLHA or simply use the popular freeware LHA utility to create your archives.

=== Key properties of the TWHCompress component:

* WebCommandLine -- used to suspend and then reactivate handling of page requests. May also be used in a future version to suspend other applications.

* CompLHA -- the TCompLHA component instance on this form which provides the compression services.

* Sites -- a list of site names -- whether WebHub or non-WebHub sites. All upload/download location properties are stored "per site"

* Site -- the name of the currently selected site (must be in the list above)

* UserSignalFileName -- if set, the component will look for this signal file
in all sites' uploadpaths and "fire" the OnUserSignal event if you've hooked it.

* UploadPath -- the location where the archive(s) and signal files (if any) will be uploaded for the current site. Also used for the STATUS.TXT file storage after a Deploy or Gather.

* OriginPath -- if specified, TWHCompress will ensure that all files expanded onto or gathered from this site have paths in or below this path.  For example, of OriginPath is "c:\mysite" then a file  from the archive called  "\here\there\x.txt"  will NOT be expanded, but a file called \mysite\here\there\x.txt" WILL be expanded. Thus, the pathnames stored in the archives need to be properly formed before the files will be expanded.

Just as importantly, when gathering files, the filenames stored in the archive will be relative to OriginPath (e.g. "\mysite\here\there\x.txt" will be stored as "here\there\x.txt"), which is important when expanding the files at the offsite end, especially using WebQueue.

V1.5+: If OriginPath is NOT specified, you will not be able to use AllowWebAdmin properly, and archives being expanded will have a (TCompLHA) TargetPath set to the path containing the archive itself.

* AppID -- the optional AppID for this site.  No function at present but in future WebHub versions, any AppIDs set here will be notified/suspended while files are deployed.

* Password -- the optional encryption/decryption password for this site. Use this only if you are using WebQueue as an off-site tool and have the same password set for it.

* RenameEXEs -- set True if you want deployed filenames ending in EXE, DLL or OCX renamed to .EX_,  .DL_ and .OC_ respectively to avoid clashing with running applications

* AllowWebAdmin -- set to true to allow the Sites, AppID, UploadPath, OriginPath, Password or GatherList properties to be changed from HTML. Since this presents a security hole, it is not advisable to do this without careful control of who can use this feature (or limiting which fields you expose on the Web page).

* Status -- stringlist of messages describing the last Deploy or Gather operation. This information can be accessed from HTML if desired, e.g. 
%=WHCompress.Status=%. It is also stored in a file called STATUS.TXT in the current upload path.

* GatherList -- a list of file specifications describing the files to be compressed in the next Gather operation for this site. Wild cards and full paths are supported, e.g. d:\ht\data\logs.*  Can also specify in the form "c:\here\*.*/S" to recurse into subdirectories below "here".

* Responsive. Only important if the FTP/signal approach is used to control the component. See the ApplicationIdle event below.

=== Key events of the TWHCompress component:

* OnDeploy -- called with a list of all the archive files found in the UploadPath directory. You can optionally amend the list, even adding archives which are in another path, or removing archives you don't want to process right now. The AppID property can be interrogated if necessary to ensure that any operations you do are for the correct site or WebApp. 

* AfterDeploy -- called with the total number of archives which were deployed (and the list of their filenames again). This is the appropriate place to refresh your application so that it reflects any changed chunks etc.

For example, you might NOT want to refresh your main application after deploying if the archives are in an upload directory which is only for updating the "ABC" application, or another, static, client site.

* OnExpand -- called just before a given archive is decompressed -- set its filename parameter to blank to skip it.

* AfterExpand -- called just after a given archive was decompressed, passing the filename, a list of files decompressed, and the current list of status messages (you can add to it if desired).  Note that each archive file is deleted after it has been successfully Expanded.

* AfterGather -- called just after a Gather operation, with a list of the files which were compressed

Note: If calling from HTML, you can also use the component's standard OnExecute and AfterExecute events to hook control if required.

* ApplicationIdle -- if you want to control the component via FTP signals, this special event HANDLER should be assigned in your application (see notes below). This will ensure that the WHCompress component is called from time to time  -- say, every 15-30 seconds or so -- PROVIDED the application isn't busy. It will use this event to check for the signal files (see "Controlling WHCompress via FTP signal files"). Note that if the Responsive property is True, the average response time is usually much quicker than 30 seconds, BUT, the CPU usage of the app increases significantly, so it is not recommended.

=== TCompLHA Component Properties

Certain properties of this component are automatically set, as follows:

ArchiveName (always Gather.LZH for Gather operations),  Confirm (False), FilesToProcess, ExceptionOnFileError (True)

You may also wish to consider setting the following after checking their specifications in the TCompLHA help:

* For Deploy:
   MakeDirectories (recommended True)

* For Gather:
    StorePath (STRONGLY recommended to be False unless you are quite sure the resulting you download *won't* be expanded over the top of existing local files which you want to keep!)
    CompressionMethod (recommended colh5, i.e. best/fastest compression)
    ArchiveType (recommended caLHA = LHA compatible)

*  For both:
    TargetPath (recommended '', i.e. empty -- but it is set by WHCompress when     expanding archives)
    Regname/RegNumber (TCompLHA registration information)
    Password (if set, this must be consistent both at the site and offline -- note that it renders the archives non-LHA compatible, as LHA itself doesn't support encryption)

* If the OnCheckFile event of the TCompLHA component is set, it will be called -- AFTER any RenameEXEs-controlled handling is done by the TWHCompress component.

=== Using TWHCompress

1. Drop a TWHCompress and TCompLHA component on your Webapp's form (or custom panel)

2. Set the "additional" TCompLHA properties as noted above

3. Set the CompLHA and WebCommandLine properties of the TWHCompress component as noted above (also set the WebApp property manually if necessary)

4. Make sure the Webapp, and then the component both update ok (double-click)

5. Write handlers for any of the TCompLHA or TWHCompress events you want to hook in order to provide special handling (e.g. emailing of a new archive in the AfterGather event). See the commented-out handlers in the WCPANEL sample panel.

6. Edit the Sites property so it contains least ONE site name (the Key column should be a number, Value column a name). Then set the Site property to the name of the site (from the Sites list) which you want to set up. Now set the UploadPath, AppID, Password and OriginPath properties for that site. Set AllowWebAdmin to True if you want to change these properties from HTML.

7. IF you want to control this component via FTP signal files:

a) Drop a TtpApplication component and a TtpAppRole component (both Tpack2 palette) on the form

b) In your form's OnCreate event, add:
  if tpAppRole1.InitialInstance then { test is optional  -- see below }
     tpApplication1.OnIdle := WHCompress1ApplicationIdle;

NB: You can omit the TtpAppRole component and its test if you are only ever going to run ONE copy of this application -- the test simply prevents having the second and subsequent copies all competing to check for FTP signals, which would be awkward.

c) See "Controlling WHCompress via FTP signal files"

8. IF you want to use the Deploy function:

a) Optionally set the RenameEXEs property to True

b) Place one or more archives named using the form "yy-mm-dd.0nn" in an upload path (e.g. 97-08-22.001). This naming convention is important because it allows a "multiple archive" case to be nicely handled -- they will be sorted and processed in the correct order (oldest first).

c) In HTML, execute the component via 
%=WHCompress1.execute|Deploy|SiteName=%
(SiteName MUST be specified and MUST be valid)

d) To view the results in HTML display them with 
%=WHCompress1.Status=%.

Note that if you are deploying many, large files, this display might need to be on a separate page, accessed later, in order to allow time for the deploying to finish. However, deploying is usually a very quick operation.

9. IF you want to use the Gather function:

a) Edit GatherList for the current site to specify a set of filenames to be processed (use numbers in the Key cell and filespecs in the Value cell, e.g.   1=d:\ht\*.txt
2=d:\data\*.db    (etc)

b) In HTML, execute the component via 
%=WHCompress1.execute|Gather|SiteName
(SiteName MUST be specified and MUST be valid)

c) View the results in HTML using the Status property as noted above for Deploy, and look at the Gather.LZH archive created in the site's UploadPath

10. IF you want to change the properties for a given site, do this:

a) start the page with:
%=WHCompress1.SelectSite|SiteName=% 
(SiteName can be a macro or literal reference) followed by:

b) For text fields, use Input fields with Names in the format Name="WHCompress1.propertyName"
and Values in the format Value="%=WHCompress1.propertyname=%", e.g.

 <INPUT TYPE=TEXT NAME="WHCompress1.UploadPath"
 Value="%=WHCompress1.UploadPath=%">

Supported text properties are OriginPath, UploadPath, Password and AppID. Note that password is the only field which can be CLEARED via HTML, by placing a "-" in the field.  Blanking the other text fields don't result in them being cleared (but the lists will).

c) For list properties (GatherList and Sites), use this format:

<TEXTAREA rows=8 cols=30  name="txtWHCompress1GatherList">%=NoBr|WHCompress1.GatherList=%</TEXTAREA>

NoBR is a Webaction on the sample WHCompress panel which outputs the list items with no <BR>'s at the end -- they bother some browsers if present in text areas.

=== Sample Panel 

The WCPANEL panel is designed to be incorporated into any existing Webapp (see installation information in README.TXT). It has sample HTML in WC.HTM, and enough user interface on the form to allow you to set relevant properties and see the results of Deploy or Gather operations . To use it:

a) Make sure TWHCompress and TCompLHA are both on your D3 palette (see installation instructions in README.TXT)

b) Add the form to an existing project per the notes in README.TXT.

c) Add WC.HTM to the app's list of Files

d) Run the app and test property setting via GUI or HTML control

e) Keep the app running, and control it with WebQueue via an uploads directory path which is accessible to your FTP server

=== Controlling WHCompress via FTP signal files:

If FTP-based signalling is in effect, place one of the following files in an UploadPath to invoke either a Deploy or Gather operation:

-- DEPLOY.SIG  for Deploy
-- GATHER.SIG  for Gather
-- REFRESH.SIG to re-read the application INI file, on the assumption that
the Webcompress-related entries in it have been manually changed

The signal file will be deleted as soon as the operation starts, and a status report (called STATUS.TXT) will be placed there when the operation is completed.

The GATHER signal file can optionally contain revised GatherList information as a simple list of file specifications. These will override existing GatherList settings for the component for this Site (and will rewrite the settings stored in the application's INI file). If the Gather signal file is empty, the current/default settings are used.

Note that WebQUEUE uses the above signalling/reporting approach to automate its interaction with any site which has a WHCompress-enabled application running. Thus, you don't need to bother with any of the above details if you are using WebQueue. However, you may find it interesting to watch the changing upload directory contents "as it happens".


Licensing restrictions:
The full source and documentation of TWHCompress (and its drop-in panel)
may be used and adapted  provided: 
*  It is not on-sold in source, DLL or object form to other Web developers without explicit permission from  SPIS Ltd. 
*  It is only deployed for operational use with a registered version of our TCompLHA component.
* If you choose to use our SPIS WebQueue application for off-site operations (e.g. providing it to your clients), you or your client(s) register each copy of WebQueue which is used for such purposes. 
