Software Spotlight: Drive Compression tools
The early 1990s were an interesting time. CPU processor speeds were increasing almost every day, applications making use of that speed got larger, and used larger data sets. Yet hard disk sizes lagged behind. A 486 laptop might only have a 120 megabyte hard drive, for example. 20, 40, and 80mb hard drives were still common.
A savvy user could manage their files using compression tools, like PKZIP, and offline floppy disk storage.
There were even professional off-line management tools such as Sytron's Never Ending Disk (NED):
https://winworldpc.com/product/neverending-disk/2x
But a more seamless solution was possible: Drive compression
A number of utilities evolved that could perform seamless, operating system level, real-time disk compression.
These utilities would create a large hidden file on the normal hard drive, and then present that file to applications as a drive letter. Applications could access and run from the drive just like any other DOS drive, but everything inside that drive was automatically and transparently compressed and decompressed.
Surprisingly, this was quite practical. As a single-tasking OS, fast DOS systems normally wasted time sitting and waiting for disk I/O. Instead of idling, depending on the program and data, doing drive compression could even speed applications up! Some drive compression programs could use hardware co-processor cards, but their speed quickly fell shy of newer CPUs.
Most "drive" compression programs work by creating a hidden file on the regular hard drive
that is then presented to the system as an extra drive letter. Files copied in to this drive are transparently compressed and decompressed. Applications do not have to take any special action or include any specific support.
There were a number of different competing products:
DoubleDisk
https://winworldpc.com/product/doubledisk/2.5
DoubleDisk was first released in 1989 by Vertisoft Systems, Inc. and was the first real-time drive compression product for DOS. It was a fairly no-frills program. It was relatively low cost compared to its later competitors. It used a pure software based drive compression.
DoubleDisk is notable as its compression technology was licensed by
Microsoft for use in MS-DOS 6.0's DoubleSpace.
Stacker
https://winworldpc.com/product/stacker
Stacker was an extremely popular disk compression product. It boasted high performance and better compression over competitors. Stacker included a number of utilities that aided in volume migration and maintenance.
It was originally intended to operate with the use of a dedicated hardware co-processor card, similar to their earlier tape based products. But highly optimized assembler code and newer CPUs enabled Stacker to run almost just as fast without. A hardware co-processor version was available, however it was mainly useful on slower computers.
Later versions of IBM PC-DOS, Novell DOS, and DR-DOS bundled Stacker instead of SuperStor.
SuperStor
https://winworldpc.com/product/superstor
SuperStor, from Addstor, was a feature-rich drive compression program, comparable in speed and ability to Stacker. It had a number of advanced configuration features.
SuperStor was bundled with IBM PC DOS 6.x and DR DOS 6.0.
I'm not sure of the exact timeline, but it sounds like they were the first to go out of business. Hence, why stacker is bundled with later PC DOS and DR DOS.
Expanz! Plus
Wanted: Software for this.
Expanz! Plus, from InfoChip Systems is a lesser known, yet widely reviewed, product. It requires a hardware co-processor card, yet its performance is less than the other software based competitors. It worked a little differently as it compressed an entire partition rather than using an uncompressed host partition. I had never even heard of this until I looked these up :P
There was one on ebay recently. And sold for $41 plus shipping: https://www.ebay.com/itm/312144066736
XtraDrive
Wanted: XtraDrive
XtraDrive - From Integrated Information Technology Inc. Another oddball in that it supposedly works at the BIOS level intercepting INT13 calls. Also works with floppy drives.
In practice, average users found that drive compression could give them more storage space and even speed things up. Windows 3.x and Windows application binaries compressed quite well, as did word processor documents, desktop databases, windows bitmaps, and other common document formats of the time.
Users that already dealt with compressed file formats would see less or no benefit.
Because the compression was implemented at the OS level, there were very few application compatibility problems. The main compatibility issues stemmed from applications checking the amount for free disk space and then getting confused when it filled up faster than expected.
Over all, millions of units were sold.
The market for third party drive compression software dried up almost overnight when MS-DOS 6.0 was released.
MS-DOS 6.0 included Microsoft's "DoubleSpace" drive compression, using technology licensed from DoubleDisk. It was heavily criticized as buggy by the media, primarily due to file corruption issues with DoubleSpace and the lack of tools to repair such issues.
MS-DOS 6.2 (They skipped 6.1) was made available as a free/inexpensive 6.0 -> 6.2 "step up kit". This update corrected some major bugs in DoubleSpace, introduced ScanDisk, added automounting of compressed floppies, and a tool to uncompress DoubleSpace compressed drives.
In 1993, Stac sued Microsoft for infringing on two of its compression related patents, and won. Microsoft had seen some of the code while considering which vendor's utility they would include with MS-DOS. Stac obtained a preliminary injunction stopping the sales of all MS-DOS products that included DoubleSpace. By this time Microsoft was shipping 6.21 that removed DoubleSpace. Microsoft also pulled compression functionality from a number of unrelated products including Windows for Workgroups 3.11, Excel, PowerPoint, Windows NT, Visual C++, and Fortran PowerStation.
MS-DOS 6.21 was specifically released to REMOVE DoubleSpace due to the legal injunction. Their replacement "DriveSpace" was not yet ready.
6.22 adds DriveSpace, a replacement for DOS 6.20's DoubleSpace drive compression that was removed in 6.21.
DriveSpace and a 32-bit driver was also included in Windows 95 and 98. A crippled version limited to removable media was included with Windows ME.
Windows NT's NTFS file system included seamless file-level compression. It could be used as full drive compression by compressing the root folder, or on just specific large or seldom used files or folders. This eliminates the need for multiple drive letters, and avoids wasting time on commonly used files. NT Workstation users could even specify to compress their files on remote NT server NTFS file shares.
However, by around 1995 or so, hard drive sizes and performance began to increase in leaps and bounds much in the same way CPU speed had. More applications began to integrate their own file compression, so data files or documents saw less benefit. OSes like Windows 95 and NT demanded more performance and did not waste as much idle time during disk I/O.
As such, the need for drive compression slowly died off.
Current Windows NTFS still provides the option to use seamless file-level compression, and there are even more advanced options available such as de-duplication.
Comments
Wikipedia devotes a page to the program that caused the failure of Addstor. https://en.wikipedia.org/wiki/Double_Tools_for_DoubleSpace Oddly, they don't have any page for either the company or its other projects. The wisdom of creating a utility to support a minor feature of an OS could be questioned but it wasn't released until after the Stacker lawsuit occurred which eliminated its value. The discount bins of CompUSA were filled with boxes of Double Tools.
Impressive, yet hard to imagine how computers of the day were able to handle on-the-fly compression of the whole harddisk access, with a nice compression ratio, if you remember how poorly they performed when working with ZIP files. And if an error occured, everything was lost. A very uncomfortable thought, DoubleSpace felt like black magick to many.
SoftWitch said:
"And if an error occured, everything was lost."
Yes. That bothered me also. My GW2000 386 had a max of 528mb. I had a primary HDD of 400mb and a secondary of 220mb. I used Stacker from v2.0 to v4.0 and was always very happy with the performance. It came close to doubling my available HDD space. I had a small partition for uncompressed programs that did not like compression. But most programs ran on the Stacker drives with no problem. There was a bit of a learning curve, but I eventually was using it even on my 1.44m floppies (mostly for games).
There was as I recall a repair utility, but what if it failed? I think I used it successfully once or twice.
My solution to this was an addition of a 4GB SCSI drive for backup. I don't recall now how many partitions were required. But I was able to copy not only the compressed images, I also copied the drives in an uncompressed state. So if the compressed image was completely messed up, I could read all the files in the uncompressed backup. This worked out quite well, although I never had to use the backups. But having the backups was reassurring.
Don't forget that shorter reads means faster disk IO. If you have static data sets, compressing them let's you access them much faster, as CPU time to decompress is trivial to waiting for spinning disks.
There are modern incarnations of these tools. I have worked on a couple. Check www.zipmagic.co for one which emphasizes maximum disk space, and check www.diskzip.com for one which emphasizes maximum performance. The DiskZIP page also indicates that there are speedup benefits even today. UIs are modeled after Stacker, even borrowing some of their original design elements. We finally have disk compression on par with Stacker's legendary compression, two decades later, for modern Windows releases!
I also remember the physical hardware cards that a lot of times were incorporated into a controller card, but they cost so much more so you spent either way--on a larger drive or on this controller.
We evaluated several of the software solutions when we finally got our hands on them and didn't like them for the risk/reward since drives were coming down in price. In the end, we never used them. I think the failure of these solutions just gave the software industry a license to bloat as much as they want, and here we are today needing 20GB for just an OS install, lol.