Reformat some NOTES and README files

Formatting is still very mixed in the NOTES and README files.
This commit tries to make formatting more consistent with the one
introduced in pull request #10545.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14042)
This commit is contained in:
Dr. Matthias St. Pierre 2021-02-01 18:53:29 +01:00
parent 9f1fe6a950
commit dc589daec8
4 changed files with 522 additions and 494 deletions

View File

@ -1,125 +1,133 @@
TOC
===
Notes on Perl
=============
- Notes on Perl
- Notes on Perl on Windows
- Notes on Perl modules we use
- Notes on installing a perl module
- [General Notes](#general-notes)
- [Perl on Windows](#perl-on-windows)
- [Perl on VMS](#perl-on-vms)
- [Required Perl modules](#required-perl-modules)
- [Notes on installing a Perl module](#notes-on-installing-a-perl-module])
Notes on Perl
-------------
For our scripts, we rely quite a bit on Perl, and increasingly on
some core Perl modules. These Perl modules are part of the Perl
source, so if you build Perl on your own, you should be set.
General Notes
-------------
However, if you install Perl as binary packages, the outcome might
differ, and you may have to check that you do get the core modules
installed properly. We do not claim to know them all, but experience
has told us the following:
For our scripts, we rely quite a bit on Perl, and increasingly on
some core Perl modules. These Perl modules are part of the Perl
source, so if you build Perl on your own, you should be set.
However, if you install Perl as binary packages, the outcome might
differ, and you may have to check that you do get the core modules
installed properly. We do not claim to know them all, but experience
has told us the following:
- on Linux distributions based on Debian, the package `perl` will
install the core Perl modules as well, so you will be fine.
- on Linux distributions based on RPMs, you will need to install
`perl-core` rather than just `perl`.
You MUST have at least Perl version 5.10.0 installed. This minimum
requirement is due to our use of regexp backslash sequence \R among
other features that didn't exist in core Perl before that version.
You MUST have at least Perl version 5.10.0 installed. This minimum
requirement is due to our use of regexp backslash sequence \R among
other features that didn't exist in core Perl before that version.
Notes on Perl on Windows
------------------------
Perl on Windows
---------------
There are a number of build targets that can be viewed as "Windows".
Indeed, there are `VC-*` configs targeting VisualStudio C, as well as
MinGW and Cygwin. The key recommendation is to use "matching" Perl,
one that matches build environment. For example, if you will build
on Cygwin be sure to use the Cygwin package manager to install Perl.
For MSYS builds use the MSYS provided Perl.
For VC-* builds we recommend Strawberry Perl, from <http://strawberryperl.com>.
An alternative is ActiveState Perl, from <http://www.activestate.com/ActivePerl>
for which you may need to explicitly select the Perl module Win32/Console.pm
available via <https://platform.activestate.com/ActiveState>.
There are a number of build targets that can be viewed as "Windows".
Indeed, there are `VC-*` configs targeting VisualStudio C, as well as
MinGW and Cygwin. The key recommendation is to use a Perl installation
that matches the build environment. For example, if you will build
on Cygwin be sure to use the Cygwin package manager to install Perl.
For MSYS builds use the MSYS provided Perl.
For VC-* builds we recommend Strawberry Perl, from <http://strawberryperl.com>.
An alternative is ActiveState Perl, from <http://www.activestate.com/ActivePerl>
for which you may need to explicitly select the Perl module Win32/Console.pm
available via <https://platform.activestate.com/ActiveState>.
Notes on Perl on VMS
--------------------
Perl on VMS
-----------
You will need to install Perl separately. One way to do so is to
download the source from <http://perl.org/>, unpacking it, reading
`README-VMS.md` and follow the instructions. Another way is to download a
`.PCSI` file from <http://www.vmsperl.com/> and install it using the
POLYCENTER install tool.
You will need to install Perl separately. One way to do so is to
download the source from <http://perl.org/>, unpacking it, reading
`README-VMS.md` and follow the instructions. Another way is to download a
`.PCSI` file from <http://www.vmsperl.com/> and install it using the
POLYCENTER install tool.
Notes on Perl modules we use
----------------------------
Required Perl modules
---------------------
We make increasing use of Perl modules, and do our best to limit
ourselves to core Perl modules to keep the requirements down. There
are just a few exceptions:
We do our best to limit ourselves to core Perl modules to keep the
requirements down. There are just a few exceptions.
* `Test::More`
We require the minimum version to be 0.96, which
appeared in Perl 5.13.4, because that version was
the first to have all the features we're using.
This module is required for testing only!
If you don't plan on running the tests,
you don't need to bother with this one.
## For Building
* `Text::Template`
* `Text::Template`
This module is not part of the core Perl modules.
As a matter of fact, the core Perl modules do not
include any templating module to date.
This module is absolutely needed,
configuration depends on it.
This module is not part of the core Perl modules.
As a matter of fact, the core Perl modules do not
include any templating module to date.
This module is absolutely needed,
configuration depends on it.
To avoid unnecessary initial hurdles, we have bundled a copy of the
following modules in our source. They will work as fallbacks if
these modules aren't already installed on the system.
## For Testing
Text::Template
* `Test::More`
Notes on installing a perl module
---------------------------------
We require the minimum version to be 0.96, which
appeared in Perl 5.13.4, because that version was
the first to have all the features we're using.
This module is required for testing only!
If you don't plan on running the tests,
you don't need to bother with this one.
There are a number of ways to install a perl module. In all
descriptions below, `Text::Template` will serve as an example.
1. for Linux users, the easiest is to install with the use of your
favorite package manager. Usually, all you need to do is search
for the module name and to install the package that comes up.
On Debian based Linux distributions, it would go like this:
To avoid unnecessary initial hurdles, we have bundled a copy of the
following modules in our source. They will work as fallbacks if
these modules aren't already installed on the system.
$ apt-cache search Text::Template
...
libtext-template-perl - perl module to process text templates
$ sudo apt-get install libtext-template-perl
Text::Template
Perl modules in Debian based distributions use package names like
the name of the module in question, with "lib" prepended and
"-perl" appended.
Notes on installing a Perl module
---------------------------------
2. Install using CPAN. This is very easy, but usually requires root
access:
There are a number of ways to install a perl module. In all
descriptions below, `Text::Template` will serve as an example.
$ cpan -i Text::Template
1. for Linux users, the easiest is to install with the use of your
favorite package manager. Usually, all you need to do is search
for the module name and to install the package that comes up.
Note that this runs all the tests that the module to be installed
comes with. This is usually a smooth operation, but there are
platforms where a failure is indicated even though the actual tests
were successful. Should that happen, you can force an
installation regardless (that should be safe since you've already
seen the tests succeed!):
On Debian based Linux distributions, it would go like this:
$ cpan -f -i Text::Template
$ apt-cache search Text::Template
...
libtext-template-perl - perl module to process text templates
$ sudo apt-get install libtext-template-perl
Note: on VMS, you must quote any argument that contains upper case
characters, so the lines above would be:
Perl modules in Debian based distributions use package names like
the name of the module in question, with "lib" prepended and
"-perl" appended.
$ cpan -i "Text::Template"
2. Install using CPAN. This is very easy, but usually requires root
access:
and:
$ cpan -i Text::Template
$ cpan -f -i "Text::Template"
Note that this runs all the tests that the module to be installed
comes with. This is usually a smooth operation, but there are
platforms where a failure is indicated even though the actual tests
were successful. Should that happen, you can force an
installation regardless (that should be safe since you've already
seen the tests succeed!):
$ cpan -f -i Text::Template
Note: on VMS, you must quote any argument that contains upper case
characters, so the lines above would be:
$ cpan -i "Text::Template"
and:
$ cpan -f -i "Text::Template"

View File

@ -1,102 +1,110 @@
NOTES FOR THE OPENVMS PLATFORM
Notes for the OpenVMS platform
==============================
Requirement details
-------------------
- [Requirement details](#requirement-details)
- [About ANSI C compiler](#about-ansi-c-compiler)
- [About ODS-5 directory names and Perl](#about-ods-5-directory-names-and-perl)
- [About MMS and DCL](#about-mms-and-dcl)
- [About debugging](#about-debugging)
- [Checking the distribution](#checking-the-distribution)
In addition to the requirements and instructions listed
in [INSTALL.md](INSTALL.md), this are required as well:
Requirement details
-------------------
In addition to the requirements and instructions listed
in [INSTALL.md](INSTALL.md), this are required as well:
* At least ODS-5 disk organization for source and build.
Installation can be done on any existing disk organization.
About ANSI C compiler
---------------------
About ANSI C compiler
---------------------
An ANSI C compiled is needed among other things. This means that
VAX C is not and will not be supported.
An ANSI C compiled is needed among other things. This means that
VAX C is not and will not be supported.
We have only tested with DEC C (aka HP VMS C / VSI C) and require
version 7.1 or later. Compiling with a different ANSI C compiler may
require some work.
We have only tested with DEC C (aka HP VMS C / VSI C) and require
version 7.1 or later. Compiling with a different ANSI C compiler may
require some work.
Please avoid using C RTL feature logical names `DECC$*` when building
and testing OpenSSL. Most of all, they can be disruptive when
running the tests, as they affect the Perl interpreter.
Please avoid using C RTL feature logical names `DECC$*` when building
and testing OpenSSL. Most of all, they can be disruptive when
running the tests, as they affect the Perl interpreter.
About ODS-5 directory names and Perl
------------------------------------
About ODS-5 directory names and Perl
------------------------------------
It seems that the perl function canonpath() in the `File::Spec` module
doesn't treat file specifications where the last directory name
contains periods very well. Unfortunately, some versions of VMS tar
will keep the periods in the OpenSSL source directory instead of
converting them to underscore, thereby leaving your source in
something like `[.openssl-1^.1^.0]`. This will lead to issues when
configuring and building OpenSSL.
It seems that the perl function canonpath() in the `File::Spec` module
doesn't treat file specifications where the last directory name
contains periods very well. Unfortunately, some versions of VMS tar
will keep the periods in the OpenSSL source directory instead of
converting them to underscore, thereby leaving your source in
something like `[.openssl-1^.1^.0]`. This will lead to issues when
configuring and building OpenSSL.
We have no replacement for Perl's canonpath(), so the best workaround
for now is to rename the OpenSSL source directory, as follows (please
adjust for the actual source directory name you have):
We have no replacement for Perl's canonpath(), so the best workaround
for now is to rename the OpenSSL source directory, as follows (please
adjust for the actual source directory name you have):
$ rename openssl-1^.1^.0.DIR openssl-1_1_0.DIR
About MMS and DCL
-----------------
About MMS and DCL
-----------------
MMS has certain limitations when it comes to line length, and DCL has
certain limitations when it comes to total command length. We do
what we can to mitigate, but there is the possibility that it's not
enough. Should you run into issues, a very simple solution is to set
yourself up a few logical names for the directory trees you're going
to use.
MMS has certain limitations when it comes to line length, and DCL has
certain limitations when it comes to total command length. We do
what we can to mitigate, but there is the possibility that it's not
enough. Should you run into issues, a very simple solution is to set
yourself up a few logical names for the directory trees you're going
to use.
About debugging
---------------
About debugging
---------------
If you build for debugging, the default on VMS is that image
activation starts the debugger automatically, giving you a debug
prompt. Unfortunately, this disrupts all other uses, such as running
test programs in the test framework.
If you build for debugging, the default on VMS is that image
activation starts the debugger automatically, giving you a debug
prompt. Unfortunately, this disrupts all other uses, such as running
test programs in the test framework.
Generally speaking, if you build for debugging, only use the programs
directly for debugging. Do not try to use them from a script, such
as running the test suite.
Generally speaking, if you build for debugging, only use the programs
directly for debugging. Do not try to use them from a script, such
as running the test suite.
### The following is not available on Alpha
### The following is not available on Alpha
As a compromise, we're turning off the flag that makes the debugger
start automatically. If there is a program that you need to debug,
you need to turn that flag back on first, for example:
As a compromise, we're turning off the flag that makes the debugger
start automatically. If there is a program that you need to debug,
you need to turn that flag back on first, for example:
$ set image /flag=call_debug [.test]evp_test.exe
Then just run it and you will find yourself in a debugging session.
When done, we recommend that you turn that flag back off:
Then just run it and you will find yourself in a debugging session.
When done, we recommend that you turn that flag back off:
$ set image /flag=nocall_debug [.test]evp_test.exe
Checking the distribution
-------------------------
Checking the distribution
-------------------------
There have been reports of places where the distribution didn't quite
get through, for example if you've copied the tree from a NFS-mounted
Unix mount point.
There have been reports of places where the distribution didn't quite
get through, for example if you've copied the tree from a NFS-mounted
Unix mount point.
The easiest way to check if everything got through as it should is to
check that this file exists:
The easiest way to check if everything got through as it should is to
check that this file exists:
[.include.openssl]configuration^.h.in
The best way to get a correct distribution is to download the gzipped
tar file from ftp://ftp.openssl.org/source/, use `GZIP -d` to uncompress
it and `VMSTAR` to unpack the resulting tar file.
The best way to get a correct distribution is to download the gzipped
tar file from ftp://ftp.openssl.org/source/, use `GZIP -d` to uncompress
it and `VMSTAR` to unpack the resulting tar file.
Gzip and VMSTAR are available here:
Gzip and VMSTAR are available here:
<http://antinode.info/dec/index.html#Software>
Should you need it, you can find UnZip for VMS here:
Should you need it, you can find UnZip for VMS here:
<http://www.info-zip.org/UnZip.html>

View File

@ -1,46 +1,57 @@
NOTES FOR WINDOWS PLATFORMS
Notes for Windows platforms
===========================
There are various options to build and run OpenSSL on the Windows platforms.
- [Native builds using Visual C++](#native-builds-using-visual-c++)
- [Native builds using MinGW](#native-builds-using-mingw)
- [Linking native applications](#linking-native-applications)
- [Hosted builds using Cygwin](#hosted-builds-using-cygwin)
"Native" OpenSSL uses the Windows APIs directly at run time.
To build a native OpenSSL you can either use:
Microsoft Visual C++ (MSVC) C compiler on the command line
or
MinGW cross compiler
run on the GNU-like development environment MSYS2
or run on Linux or Cygwin
There are various options to build and run OpenSSL on the Windows platforms.
"Hosted" OpenSSL relies on an external POSIX compatibility layer
for building (using GNU/Unix shell, compiler, and tools) and at run time.
For this option you can use Cygwin.
"Native" OpenSSL uses the Windows APIs directly at run time.
To build a native OpenSSL you can either use:
Visual C++ native builds, aka VC-*
=====================================
Microsoft Visual C++ (MSVC) C compiler on the command line
or
MinGW cross compiler
run on the GNU-like development environment MSYS2
or run on Linux or Cygwin
Requirement details
-------------------
"Hosted" OpenSSL relies on an external POSIX compatibility layer
for building (using GNU/Unix shell, compiler, and tools) and at run time.
For this option you can use Cygwin.
In addition to the requirements and instructions listed in INSTALL.md,
these are required as well:
Native builds using Visual C++
==============================
- Perl.
We recommend Strawberry Perl, available from <http://strawberryperl.com/>
Please read NOTES.PERL for more information, including the use of CPAN.
An alternative is ActiveState Perl, <https://www.activestate.com/ActivePerl>
for which you may need to explicitly build the Perl module Win32/Console.pm
via <https://platform.activestate.com/ActiveState> and then download it.
The native builds using Visual C++ have a VC-* prefix.
- Microsoft Visual C compiler.
Since these are proprietary and ever-changing we cannot test them all.
Older versions may not work. Use a recent version wherever possible.
Requirement details
-------------------
- Netwide Assembler (NASM), available from <https://www.nasm.us>
Note that NASM is the only supported assembler.
In addition to the requirements and instructions listed in INSTALL.md,
these are required as well:
Quick start
-----------
### Perl
We recommend Strawberry Perl, available from <http://strawberryperl.com/>
Please read NOTES.PERL for more information, including the use of CPAN.
An alternative is ActiveState Perl, <https://www.activestate.com/ActivePerl>
for which you may need to explicitly build the Perl module Win32/Console.pm
via <https://platform.activestate.com/ActiveState> and then download it.
### Microsoft Visual C compiler.
Since these are proprietary and ever-changing we cannot test them all.
Older versions may not work. Use a recent version wherever possible.
### Netwide Assembler (NASM)
NASM is the only supported assembler. It is available from <https://www.nasm.us>.
Quick start
-----------
1. Install Perl
@ -67,40 +78,40 @@ NOTES FOR WINDOWS PLATFORMS
8. nmake install
For the full installation instructions, or if anything goes wrong at any stage,
check the INSTALL.md file.
For the full installation instructions, or if anything goes wrong at any stage,
check the INSTALL.md file.
Installation directories
------------------------
Installation directories
------------------------
The default installation directories are derived from environment
variables.
The default installation directories are derived from environment
variables.
For VC-WIN32, the following defaults are use:
For VC-WIN32, the following defaults are use:
PREFIX: %ProgramFiles(86)%\OpenSSL
OPENSSLDIR: %CommonProgramFiles(86)%\SSL
PREFIX: %ProgramFiles(86)%\OpenSSL
OPENSSLDIR: %CommonProgramFiles(86)%\SSL
For VC-WIN64, the following defaults are use:
For VC-WIN64, the following defaults are use:
PREFIX: %ProgramW6432%\OpenSSL
OPENSSLDIR: %CommonProgramW6432%\SSL
PREFIX: %ProgramW6432%\OpenSSL
OPENSSLDIR: %CommonProgramW6432%\SSL
Should those environment variables not exist (on a pure Win32
installation for examples), these fallbacks are used:
Should those environment variables not exist (on a pure Win32
installation for examples), these fallbacks are used:
PREFIX: %ProgramFiles%\OpenSSL
OPENSSLDIR: %CommonProgramFiles%\SSL
PREFIX: %ProgramFiles%\OpenSSL
OPENSSLDIR: %CommonProgramFiles%\SSL
ALSO NOTE that those directories are usually write protected, even if
your account is in the Administrators group. To work around that,
start the command prompt by right-clicking on it and choosing "Run as
Administrator" before running 'nmake install'. The other solution
is, of course, to choose a different set of directories by using
--prefix and --openssldir when configuring.
ALSO NOTE that those directories are usually write protected, even if
your account is in the Administrators group. To work around that,
start the command prompt by right-clicking on it and choosing "Run as
Administrator" before running 'nmake install'. The other solution
is, of course, to choose a different set of directories by using
--prefix and --openssldir when configuring.
Special notes for Universal Windows Platform builds, aka VC-*-UWP
--------------------------------------------------------------------
Special notes for Universal Windows Platform builds, aka VC-*-UWP
--------------------------------------------------------------------
- UWP targets only support building the static and dynamic libraries.
@ -108,10 +119,10 @@ NOTES FOR WINDOWS PLATFORMS
"vcvarsall.bat" before you compile. For example, if you want to build
"arm64" builds, you should run "vcvarsall.bat x86_arm64 uwp".
Native OpenSSL built using MinGW
================================
Native builds using MinGW
=========================
MinGW offers an alternative way to build native OpenSSL, by cross compilation.
MinGW offers an alternative way to build native OpenSSL, by cross compilation.
* Usually the build is done on Windows in a GNU-like environment called MSYS2.
@ -135,11 +146,12 @@ NOTES FOR WINDOWS PLATFORMS
In the MSYS2 shell do the configuration depending on the target architecture:
./Configure mingw ...
./Configure mingw ...
or
./Configure mingw64 ...
./Configure mingw64 ...
or
./Configure ...
./Configure ...
for the default architecture.
Apart from that, follow the Unix / Linux instructions in INSTALL.md.
@ -149,55 +161,55 @@ NOTES FOR WINDOWS PLATFORMS
In this case configure with the corresponding --cross-compile-prefix= option.
For example
./Configure mingw --cross-compile-prefix=i686-w64-mingw32- ...
./Configure mingw --cross-compile-prefix=i686-w64-mingw32- ...
or
./Configure mingw64 --cross-compile-prefix=x86_64-w64-mingw32- ...
./Configure mingw64 --cross-compile-prefix=x86_64-w64-mingw32- ...
This requires that you've installed the necessary add-on packages for
mingw[64] cross compilation.
Linking your application
========================
Linking native applications
===========================
This section applies to all "native" builds.
This section applies to all native builds.
If you link with static OpenSSL libraries then you're expected to
additionally link your application with WS2_32.LIB, GDI32.LIB,
ADVAPI32.LIB, CRYPT32.LIB and USER32.LIB. Those developing
non-interactive service applications might feel concerned about
linking with GDI32.LIB and USER32.LIB, as they are justly associated
with interactive desktop, which is not available to service
processes. The toolkit is designed to detect in which context it's
currently executed, GUI, console app or service, and act accordingly,
namely whether or not to actually make GUI calls. Additionally those
who wish to /DELAYLOAD:GDI32.DLL and /DELAYLOAD:USER32.DLL and
actually keep them off service process should consider implementing
and exporting from .exe image in question own _OPENSSL_isservice not
relying on USER32.DLL. E.g., on Windows Vista and later you could:
If you link with static OpenSSL libraries then you're expected to
additionally link your application with WS2_32.LIB, GDI32.LIB,
ADVAPI32.LIB, CRYPT32.LIB and USER32.LIB. Those developing
non-interactive service applications might feel concerned about
linking with GDI32.LIB and USER32.LIB, as they are justly associated
with interactive desktop, which is not available to service
processes. The toolkit is designed to detect in which context it's
currently executed, GUI, console app or service, and act accordingly,
namely whether or not to actually make GUI calls. Additionally those
who wish to /DELAYLOAD:GDI32.DLL and /DELAYLOAD:USER32.DLL and
actually keep them off service process should consider implementing
and exporting from .exe image in question own _OPENSSL_isservice not
relying on USER32.DLL. E.g., on Windows Vista and later you could:
__declspec(dllexport) __cdecl BOOL _OPENSSL_isservice(void)
{
DWORD sess;
__declspec(dllexport) __cdecl BOOL _OPENSSL_isservice(void)
{
DWORD sess;
if (ProcessIdToSessionId(GetCurrentProcessId(), &sess))
return sess == 0;
return FALSE;
}
if (ProcessIdToSessionId(GetCurrentProcessId(), &sess))
return sess == 0;
return FALSE;
}
If you link with OpenSSL .DLLs, then you're expected to include into
your application code a small "shim" snippet, which provides
the glue between the OpenSSL BIO layer and your compiler run-time.
See also the OPENSSL_Applink manual page.
If you link with OpenSSL .DLLs, then you're expected to include into
your application code a small "shim" snippet, which provides
the glue between the OpenSSL BIO layer and your compiler run-time.
See also the OPENSSL_Applink manual page.
Hosted OpenSSL built using Cygwin
=================================
Hosted builds using Cygwin
==========================
Cygwin implements a POSIX/Unix runtime system (cygwin1.dll) on top of the
Windows subsystem and provides a Bash shell and GNU tools environment.
Consequently, a build of OpenSSL with Cygwin is virtually identical to the
Unix procedure.
Cygwin implements a POSIX/Unix runtime system (cygwin1.dll) on top of the
Windows subsystem and provides a Bash shell and GNU tools environment.
Consequently, a build of OpenSSL with Cygwin is virtually identical to the
Unix procedure.
To build OpenSSL using Cygwin, you need to:
To build OpenSSL using Cygwin, you need to:
* Install Cygwin, see <https://cygwin.com/>
@ -206,9 +218,9 @@ NOTES FOR WINDOWS PLATFORMS
* Run the Cygwin Bash shell
Apart from that, follow the Unix / Linux instructions in INSTALL.md.
Apart from that, follow the Unix / Linux instructions in INSTALL.md.
NOTE: "make test" and normal file operations may fail in directories
mounted as text (i.e. mount -t c:\somewhere /home) due to Cygwin
stripping of carriage returns. To avoid this ensure that a binary
mount is used, e.g. mount -b c:\somewhere /home.
NOTE: "make test" and normal file operations may fail in directories
mounted as text (i.e. mount -t c:\somewhere /home) due to Cygwin
stripping of carriage returns. To avoid this ensure that a binary
mount is used, e.g. mount -b c:\somewhere /home.

View File

@ -1,308 +1,308 @@
ENGINES
=======
With OpenSSL 0.9.6, a new component was added to support alternative
cryptography implementations, most commonly for interfacing with external
crypto devices (eg. accelerator cards). This component is called ENGINE,
and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
caused a little confusion as 0.9.6** releases were rolled in two
versions, a "standard" and an "engine" version. In development for 0.9.7,
the ENGINE code has been merged into the main branch and will be present
in the standard releases from 0.9.7 forwards.
With OpenSSL 0.9.6, a new component was added to support alternative
cryptography implementations, most commonly for interfacing with external
crypto devices (eg. accelerator cards). This component is called ENGINE,
and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
caused a little confusion as 0.9.6** releases were rolled in two
versions, a "standard" and an "engine" version. In development for 0.9.7,
the ENGINE code has been merged into the main branch and will be present
in the standard releases from 0.9.7 forwards.
There are currently built-in ENGINE implementations for the following
crypto devices:
There are currently built-in ENGINE implementations for the following
crypto devices:
* Microsoft CryptoAPI
* VIA Padlock
* nCipher CHIL
In addition, dynamic binding to external ENGINE implementations is now
provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
section below for details.
In addition, dynamic binding to external ENGINE implementations is now
provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
section below for details.
At this stage, a number of things are still needed and are being worked on:
At this stage, a number of things are still needed and are being worked on:
1. Integration of EVP support.
2. Configuration support.
3. Documentation!
Integration of EVP support
--------------------------
Integration of EVP support
--------------------------
With respect to EVP, this relates to support for ciphers and digests in
the ENGINE model so that alternative implementations of existing
algorithms/modes (or previously unimplemented ones) can be provided by
ENGINE implementations.
With respect to EVP, this relates to support for ciphers and digests in
the ENGINE model so that alternative implementations of existing
algorithms/modes (or previously unimplemented ones) can be provided by
ENGINE implementations.
Configuration support
---------------------
Configuration support
---------------------
Configuration support currently exists in the ENGINE API itself, in the
form of "control commands". These allow an application to expose to the
user/admin the set of commands and parameter types a given ENGINE
implementation supports, and for an application to directly feed string
based input to those ENGINEs, in the form of name-value pairs. This is an
extensible way for ENGINEs to define their own "configuration" mechanisms
that are specific to a given ENGINE (eg. for a particular hardware
device) but that should be consistent across *all* OpenSSL-based
applications when they use that ENGINE. Work is in progress (or at least
in planning) for supporting these control commands from the CONF (or
NCONF) code so that applications using OpenSSL's existing configuration
file format can have ENGINE settings specified in much the same way.
Presently however, applications must use the ENGINE API itself to provide
such functionality. To see first hand the types of commands available
with the various compiled-in ENGINEs (see further down for dynamic
ENGINEs), use the "engine" openssl utility with full verbosity, i.e.:
Configuration support currently exists in the ENGINE API itself, in the
form of "control commands". These allow an application to expose to the
user/admin the set of commands and parameter types a given ENGINE
implementation supports, and for an application to directly feed string
based input to those ENGINEs, in the form of name-value pairs. This is an
extensible way for ENGINEs to define their own "configuration" mechanisms
that are specific to a given ENGINE (eg. for a particular hardware
device) but that should be consistent across *all* OpenSSL-based
applications when they use that ENGINE. Work is in progress (or at least
in planning) for supporting these control commands from the CONF (or
NCONF) code so that applications using OpenSSL's existing configuration
file format can have ENGINE settings specified in much the same way.
Presently however, applications must use the ENGINE API itself to provide
such functionality. To see first hand the types of commands available
with the various compiled-in ENGINEs (see further down for dynamic
ENGINEs), use the "engine" openssl utility with full verbosity, i.e.:
openssl engine -vvvv
openssl engine -vvvv
Documentation
-------------
Documentation
-------------
Documentation? Volunteers welcome! The source code is reasonably well
self-documenting, but some summaries and usage instructions are needed -
moreover, they are needed in the same POD format the existing OpenSSL
documentation is provided in. Any complete or incomplete contributions
would help make this happen.
Documentation? Volunteers welcome! The source code is reasonably well
self-documenting, but some summaries and usage instructions are needed -
moreover, they are needed in the same POD format the existing OpenSSL
documentation is provided in. Any complete or incomplete contributions
would help make this happen.
STABILITY & BUG-REPORTS
=======================
STABILITY & BUG-REPORTS
=======================
What already exists is fairly stable as far as it has been tested, but
the test base has been a bit small most of the time. For the most part,
the vendors of the devices these ENGINEs support have contributed to the
development and/or testing of the implementations, and *usually* (with no
guarantees) have experience in using the ENGINE support to drive their
devices from common OpenSSL-based applications. Bugs and/or inexplicable
behaviour in using a specific ENGINE implementation should be sent to the
author of that implementation (if it is mentioned in the corresponding C
file), and in the case of implementations for commercial hardware
devices, also through whatever vendor support channels are available. If
none of this is possible, or the problem seems to be something about the
ENGINE API itself (ie. not necessarily specific to a particular ENGINE
implementation) then you should mail complete details to the relevant
OpenSSL mailing list. For a definition of "complete details", refer to
the OpenSSL "README" file. As for which list to send it to:
What already exists is fairly stable as far as it has been tested, but
the test base has been a bit small most of the time. For the most part,
the vendors of the devices these ENGINEs support have contributed to the
development and/or testing of the implementations, and *usually* (with no
guarantees) have experience in using the ENGINE support to drive their
devices from common OpenSSL-based applications. Bugs and/or inexplicable
behaviour in using a specific ENGINE implementation should be sent to the
author of that implementation (if it is mentioned in the corresponding C
file), and in the case of implementations for commercial hardware
devices, also through whatever vendor support channels are available. If
none of this is possible, or the problem seems to be something about the
ENGINE API itself (ie. not necessarily specific to a particular ENGINE
implementation) then you should mail complete details to the relevant
OpenSSL mailing list. For a definition of "complete details", refer to
the OpenSSL "README" file. As for which list to send it to:
* openssl-users: if you are *using* the ENGINE abstraction, either in an
pre-compiled application or in your own application code.
* openssl-dev: if you are discussing problems with OpenSSL source code.
USAGE
=====
USAGE
=====
The default "openssl" ENGINE is always chosen when performing crypto
operations unless you specify otherwise. You must actively tell the
openssl utility commands to use anything else through a new command line
switch called "-engine". Also, if you want to use the ENGINE support in
your own code to do something similar, you must likewise explicitly
select the ENGINE implementation you want.
The default "openssl" ENGINE is always chosen when performing crypto
operations unless you specify otherwise. You must actively tell the
openssl utility commands to use anything else through a new command line
switch called "-engine". Also, if you want to use the ENGINE support in
your own code to do something similar, you must likewise explicitly
select the ENGINE implementation you want.
Depending on the type of hardware, system, and configuration, "settings"
may need to be applied to an ENGINE for it to function as expected/hoped.
The recommended way of doing this is for the application to support
ENGINE "control commands" so that each ENGINE implementation can provide
whatever configuration primitives it might require and the application
can allow the user/admin (and thus the hardware vendor's support desk
also) to provide any such input directly to the ENGINE implementation.
This way, applications do not need to know anything specific to any
device, they only need to provide the means to carry such user/admin
input through to the ENGINE in question. Ie. this connects *you* (and
your helpdesk) to the specific ENGINE implementation (and device), and
allows application authors to not get buried in hassle supporting
arbitrary devices they know (and care) nothing about.
Depending on the type of hardware, system, and configuration, "settings"
may need to be applied to an ENGINE for it to function as expected/hoped.
The recommended way of doing this is for the application to support
ENGINE "control commands" so that each ENGINE implementation can provide
whatever configuration primitives it might require and the application
can allow the user/admin (and thus the hardware vendor's support desk
also) to provide any such input directly to the ENGINE implementation.
This way, applications do not need to know anything specific to any
device, they only need to provide the means to carry such user/admin
input through to the ENGINE in question. Ie. this connects *you* (and
your helpdesk) to the specific ENGINE implementation (and device), and
allows application authors to not get buried in hassle supporting
arbitrary devices they know (and care) nothing about.
A new "openssl" utility, "openssl engine", has been added in that allows
for testing and examination of ENGINE implementations. Basic usage
instructions are available by specifying the "-?" command line switch.
A new "openssl" utility, "openssl engine", has been added in that allows
for testing and examination of ENGINE implementations. Basic usage
instructions are available by specifying the "-?" command line switch.
DYNAMIC ENGINES
===============
DYNAMIC ENGINES
===============
The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
implementations that aren't pre-compiled and linked into OpenSSL-based
applications. This could be because existing compiled-in implementations
have known problems and you wish to use a newer version with an existing
application. It could equally be because the application (or OpenSSL
library) you are using simply doesn't have support for the ENGINE you
wish to use, and the ENGINE provider (eg. hardware vendor) is providing
you with a self-contained implementation in the form of a shared-library.
The other use-case for "dynamic" is with applications that wish to
maintain the smallest foot-print possible and so do not link in various
ENGINE implementations from OpenSSL, but instead leaves you to provide
them, if you want them, in the form of "dynamic"-loadable
shared-libraries. It should be possible for hardware vendors to provide
their own shared-libraries to support arbitrary hardware to work with
applications based on OpenSSL 0.9.7 or later. If you're using an
application based on 0.9.7 (or later) and the support you desire is only
announced for versions later than the one you need, ask the vendor to
backport their ENGINE to the version you need.
The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
implementations that aren't pre-compiled and linked into OpenSSL-based
applications. This could be because existing compiled-in implementations
have known problems and you wish to use a newer version with an existing
application. It could equally be because the application (or OpenSSL
library) you are using simply doesn't have support for the ENGINE you
wish to use, and the ENGINE provider (eg. hardware vendor) is providing
you with a self-contained implementation in the form of a shared-library.
The other use-case for "dynamic" is with applications that wish to
maintain the smallest foot-print possible and so do not link in various
ENGINE implementations from OpenSSL, but instead leaves you to provide
them, if you want them, in the form of "dynamic"-loadable
shared-libraries. It should be possible for hardware vendors to provide
their own shared-libraries to support arbitrary hardware to work with
applications based on OpenSSL 0.9.7 or later. If you're using an
application based on 0.9.7 (or later) and the support you desire is only
announced for versions later than the one you need, ask the vendor to
backport their ENGINE to the version you need.
How does "dynamic" work?
------------------------
How does "dynamic" work?
------------------------
The dynamic ENGINE has a special flag in its implementation such that
every time application code asks for the 'dynamic' ENGINE, it in fact
gets its own copy of it. As such, multi-threaded code (or code that
multiplexes multiple uses of 'dynamic' in a single application in any
way at all) does not get confused by 'dynamic' being used to do many
independent things. Other ENGINEs typically don't do this so there is
only ever 1 ENGINE structure of its type (and reference counts are used
to keep order). The dynamic ENGINE itself provides absolutely no
cryptographic functionality, and any attempt to "initialise" the ENGINE
automatically fails. All it does provide are a few "control commands"
that can be used to control how it will load an external ENGINE
implementation from a shared-library. To see these control commands,
use the command-line;
The dynamic ENGINE has a special flag in its implementation such that
every time application code asks for the 'dynamic' ENGINE, it in fact
gets its own copy of it. As such, multi-threaded code (or code that
multiplexes multiple uses of 'dynamic' in a single application in any
way at all) does not get confused by 'dynamic' being used to do many
independent things. Other ENGINEs typically don't do this so there is
only ever 1 ENGINE structure of its type (and reference counts are used
to keep order). The dynamic ENGINE itself provides absolutely no
cryptographic functionality, and any attempt to "initialise" the ENGINE
automatically fails. All it does provide are a few "control commands"
that can be used to control how it will load an external ENGINE
implementation from a shared-library. To see these control commands,
use the command-line;
openssl engine -vvvv dynamic
The "SO_PATH" control command should be used to identify the
shared-library that contains the ENGINE implementation, and "NO_VCHECK"
might possibly be useful if there is a minor version conflict and you
(or a vendor helpdesk) is convinced you can safely ignore it.
"ID" is probably only needed if a shared-library implements
multiple ENGINEs, but if you know the engine id you expect to be using,
it doesn't hurt to specify it (and this provides a sanity check if
nothing else). "LIST_ADD" is only required if you actually wish the
loaded ENGINE to be discoverable by application code later on using the
ENGINE's "id". For most applications, this isn't necessary - but some
application authors may have nifty reasons for using it. The "LOAD"
command is the only one that takes no parameters and is the command
that uses the settings from any previous commands to actually *load*
the shared-library ENGINE implementation. If this command succeeds, the
(copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
that has been loaded from the shared-library. As such, any control
commands supported by the loaded ENGINE could then be executed as per
normal. Eg. if ENGINE "foo" is implemented in the shared-library
"libfoo.so" and it supports some special control command "CMD_FOO", the
following code would load and use it (NB: obviously this code has no
error checking);
The "SO_PATH" control command should be used to identify the
shared-library that contains the ENGINE implementation, and "NO_VCHECK"
might possibly be useful if there is a minor version conflict and you
(or a vendor helpdesk) is convinced you can safely ignore it.
"ID" is probably only needed if a shared-library implements
multiple ENGINEs, but if you know the engine id you expect to be using,
it doesn't hurt to specify it (and this provides a sanity check if
nothing else). "LIST_ADD" is only required if you actually wish the
loaded ENGINE to be discoverable by application code later on using the
ENGINE's "id". For most applications, this isn't necessary - but some
application authors may have nifty reasons for using it. The "LOAD"
command is the only one that takes no parameters and is the command
that uses the settings from any previous commands to actually *load*
the shared-library ENGINE implementation. If this command succeeds, the
(copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
that has been loaded from the shared-library. As such, any control
commands supported by the loaded ENGINE could then be executed as per
normal. Eg. if ENGINE "foo" is implemented in the shared-library
"libfoo.so" and it supports some special control command "CMD_FOO", the
following code would load and use it (NB: obviously this code has no
error checking);
ENGINE *e = ENGINE_by_id("dynamic");
ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
ENGINE *e = ENGINE_by_id("dynamic");
ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
For testing, the "openssl engine" utility can be useful for this sort
of thing. For example the above code excerpt would achieve much the
same result as;
For testing, the "openssl engine" utility can be useful for this sort
of thing. For example the above code excerpt would achieve much the
same result as;
openssl engine dynamic \
-pre SO_PATH:/lib/libfoo.so \
-pre ID:foo \
-pre LOAD \
-pre "CMD_FOO:some input data"
openssl engine dynamic \
-pre SO_PATH:/lib/libfoo.so \
-pre ID:foo \
-pre LOAD \
-pre "CMD_FOO:some input data"
Or to simply see the list of commands supported by the "foo" ENGINE;
Or to simply see the list of commands supported by the "foo" ENGINE;
openssl engine -vvvv dynamic \
-pre SO_PATH:/lib/libfoo.so \
-pre ID:foo \
-pre LOAD
openssl engine -vvvv dynamic \
-pre SO_PATH:/lib/libfoo.so \
-pre ID:foo \
-pre LOAD
Applications that support the ENGINE API and more specifically, the
"control commands" mechanism, will provide some way for you to pass
such commands through to ENGINEs. As such, you would select "dynamic"
as the ENGINE to use, and the parameters/commands you pass would
control the *actual* ENGINE used. Each command is actually a name-value
pair and the value can sometimes be omitted (eg. the "LOAD" command).
Whilst the syntax demonstrated in "openssl engine" uses a colon to
separate the command name from the value, applications may provide
their own syntax for making that separation (eg. a win32 registry
key-value pair may be used by some applications). The reason for the
"-pre" syntax in the "openssl engine" utility is that some commands
might be issued to an ENGINE *after* it has been initialised for use.
Eg. if an ENGINE implementation requires a smart-card to be inserted
during initialisation (or a PIN to be typed, or whatever), there may be
a control command you can issue afterwards to "forget" the smart-card
so that additional initialisation is no longer possible. In
applications such as web-servers, where potentially volatile code may
run on the same host system, this may provide some arguable security
value. In such a case, the command would be passed to the ENGINE after
it has been initialised for use, and so the "-post" switch would be
used instead. Applications may provide a different syntax for
supporting this distinction, and some may simply not provide it at all
("-pre" is almost always what you're after, in reality).
Applications that support the ENGINE API and more specifically, the
"control commands" mechanism, will provide some way for you to pass
such commands through to ENGINEs. As such, you would select "dynamic"
as the ENGINE to use, and the parameters/commands you pass would
control the *actual* ENGINE used. Each command is actually a name-value
pair and the value can sometimes be omitted (eg. the "LOAD" command).
Whilst the syntax demonstrated in "openssl engine" uses a colon to
separate the command name from the value, applications may provide
their own syntax for making that separation (eg. a win32 registry
key-value pair may be used by some applications). The reason for the
"-pre" syntax in the "openssl engine" utility is that some commands
might be issued to an ENGINE *after* it has been initialised for use.
Eg. if an ENGINE implementation requires a smart-card to be inserted
during initialisation (or a PIN to be typed, or whatever), there may be
a control command you can issue afterwards to "forget" the smart-card
so that additional initialisation is no longer possible. In
applications such as web-servers, where potentially volatile code may
run on the same host system, this may provide some arguable security
value. In such a case, the command would be passed to the ENGINE after
it has been initialised for use, and so the "-post" switch would be
used instead. Applications may provide a different syntax for
supporting this distinction, and some may simply not provide it at all
("-pre" is almost always what you're after, in reality).
How do I build a "dynamic" ENGINE?
----------------------------------
How do I build a "dynamic" ENGINE?
----------------------------------
This question is trickier - currently OpenSSL bundles various ENGINE
implementations that are statically built in, and any application that
calls the "ENGINE_load_builtin_engines()" function will automatically
have all such ENGINEs available (and occupying memory). Applications
that don't call that function have no ENGINEs available like that and
would have to use "dynamic" to load any such ENGINE - but on the other
hand such applications would only have the memory footprint of any
ENGINEs explicitly loaded using user/admin provided control commands.
The main advantage of not statically linking ENGINEs and only using
"dynamic" for hardware support is that any installation using no
"external" ENGINE suffers no unnecessary memory footprint from unused
ENGINEs. Likewise, installations that do require an ENGINE incur the
overheads from only *that* ENGINE once it has been loaded.
This question is trickier - currently OpenSSL bundles various ENGINE
implementations that are statically built in, and any application that
calls the "ENGINE_load_builtin_engines()" function will automatically
have all such ENGINEs available (and occupying memory). Applications
that don't call that function have no ENGINEs available like that and
would have to use "dynamic" to load any such ENGINE - but on the other
hand such applications would only have the memory footprint of any
ENGINEs explicitly loaded using user/admin provided control commands.
The main advantage of not statically linking ENGINEs and only using
"dynamic" for hardware support is that any installation using no
"external" ENGINE suffers no unnecessary memory footprint from unused
ENGINEs. Likewise, installations that do require an ENGINE incur the
overheads from only *that* ENGINE once it has been loaded.
Sounds good? Maybe, but currently building an ENGINE implementation as
a shared-library that can be loaded by "dynamic" isn't automated in
OpenSSL's build process. It can be done manually quite easily however.
Such a shared-library can either be built with any OpenSSL code it
needs statically linked in, or it can link dynamically against OpenSSL
if OpenSSL itself is built as a shared library. The instructions are
the same in each case, but in the former (statically linked any
dependencies on OpenSSL) you must ensure OpenSSL is built with
position-independent code ("PIC"). The default OpenSSL compilation may
already specify the relevant flags to do this, but you should consult
with your compiler documentation if you are in any doubt.
Sounds good? Maybe, but currently building an ENGINE implementation as
a shared-library that can be loaded by "dynamic" isn't automated in
OpenSSL's build process. It can be done manually quite easily however.
Such a shared-library can either be built with any OpenSSL code it
needs statically linked in, or it can link dynamically against OpenSSL
if OpenSSL itself is built as a shared library. The instructions are
the same in each case, but in the former (statically linked any
dependencies on OpenSSL) you must ensure OpenSSL is built with
position-independent code ("PIC"). The default OpenSSL compilation may
already specify the relevant flags to do this, but you should consult
with your compiler documentation if you are in any doubt.
This example will show building the "atalla" ENGINE in the
crypto/engine/ directory as a shared-library for use via the "dynamic"
ENGINE.
This example will show building the "atalla" ENGINE in the
crypto/engine/ directory as a shared-library for use via the "dynamic"
ENGINE.
1. "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL
source tree.
source tree.
2. Recompile at least one source file so you can see all the compiler
flags (and syntax) being used to build normally. Eg;
flags (and syntax) being used to build normally. Eg;
touch hw_atalla.c ; make
touch hw_atalla.c ; make
will rebuild "hw_atalla.o" using all such flags.
will rebuild "hw_atalla.o" using all such flags.
3. Manually enter the same compilation line to compile the
"hw_atalla.c" file but with the following two changes;
* add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
* change the output file from "hw_atalla.o" to something new,
eg. "tmp_atalla.o"
"hw_atalla.c" file but with the following two changes;
* add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
* change the output file from "hw_atalla.o" to something new,
eg. "tmp_atalla.o"
4. Link "tmp_atalla.o" into a shared-library using the top-level
OpenSSL libraries to resolve any dependencies. The syntax for doing
this depends heavily on your system/compiler and is a nightmare
known well to anyone who has worked with shared-library portability
before. 'gcc' on Linux, for example, would use the following syntax;
OpenSSL libraries to resolve any dependencies. The syntax for doing
this depends heavily on your system/compiler and is a nightmare
known well to anyone who has worked with shared-library portability
before. 'gcc' on Linux, for example, would use the following syntax;
gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
5. Test your shared library using "openssl engine" as explained in the
previous section. Eg. from the top-level directory, you might try
previous section. Eg. from the top-level directory, you might try
apps/openssl engine -vvvv dynamic \
-pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
apps/openssl engine -vvvv dynamic \
-pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
If the shared-library loads successfully, you will see both "-pre"
commands marked as "SUCCESS" and the list of control commands
displayed (because of "-vvvv") will be the control commands for the
*atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
the "-t" switch to the utility if you want it to try and initialise
the atalla ENGINE for use to test any possible hardware/driver issues.
If the shared-library loads successfully, you will see both "-pre"
commands marked as "SUCCESS" and the list of control commands
displayed (because of "-vvvv") will be the control commands for the
*atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
the "-t" switch to the utility if you want it to try and initialise
the atalla ENGINE for use to test any possible hardware/driver issues.
PROBLEMS
========
PROBLEMS
========
It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
A quick test done right before the release showed that trying "openssl speed
-engine cswift" generated errors. If the DSO gets enabled, an attempt is made
to write at memory address 0x00000002.
It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
A quick test done right before the release showed that trying "openssl speed
-engine cswift" generated errors. If the DSO gets enabled, an attempt is made
to write at memory address 0x00000002.