build.info: Introduce special syntax for dependencies on script modules

The DEPEND statement, when applied on files generated with GENERATE, may
be used to specify script modules that the template to be generated from
depends on.  In short, this sort of depend:

    DEPEND[generated]=util/perl/OpenSSL/something.pm

... would generate a perl run that has the inclusion directory
'util/perl/OpenSSL' and 'something' as the module to be loaded.  However,
the package name for this module is 'OpenSSL::something', so to load it the
way it's expected, the inclusion directory should be 'util/perl', and the
module to be loaded should be specified as 'OpenSSL/something' (to be
massaged into a proper module name by the build file template).

To allow this, we introduce a file syntax, where a single '|' is used as a
directory separator, to delineate what part should be used as the inclustion
directory, and which part the module name to be loaded should be derived
from:

    DEPEND[generated]=util/perl|OpenSSL/something.pm

Fixes #21112

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21117)
This commit is contained in:
Richard Levitte 2023-06-02 14:32:07 +02:00
parent b8fa5be550
commit b684ee2ce4
6 changed files with 196 additions and 33 deletions

View File

@ -1056,16 +1056,48 @@ EOF
my $dofile = abs2rel(rel2abs(catfile($config{sourcedir},
"util", "dofile.pl")),
rel2abs($config{builddir}));
my @perlmodules = ( 'configdata.pm',
grep { $_ =~ m|\.pm$| } @{$args{deps}} );
my %perlmoduleincs = map { '"-I'.dirname($_).'"' => 1 } @perlmodules;
my @perlmodules = ();
my %perlmoduleincs = ();
my %perlmoduledeps = ();
foreach my $x (('configdata.pm', @{$args{deps}})) {
# Compute (i)nclusion directory, (m)odule name and (d)ependency
my $i, $m, $d;
if ($x =~ /\|/) {
$i = $`;
$d = $';
# Massage the module part to become a real perl module spec
$m = $d;
$m =~ s|\.pm$||;
# Directory specs are :: in perl package names
$m =~ s|/|::|g;
# Full file name of the dependency
$d = catfile($i, $d) if $i;
} elsif ($x =~ /\.pm$/) {
$i = dirname($x);
$m = basename($x, '.pm');
$d = $x;
} else {
# All other dependencies are simply collected
$d = $x;
}
push @perlmodules, '"-M'.$m.'"' if $m;
$perlmoduledeps{$d} = 1;
$perlmoduleincs{'"-I'.$i.'"'} = 1 if $i;
}
my @decc_include_data
= make_decc_include_files(dirname($args{src}), dirname($gen0));
my $decc_include_scripture = pop @decc_include_data;
$deps = join(' ', $deps, @decc_include_data,
compute_platform_depends(@perlmodules));
@perlmodules = map { '"-M'.basename($_, '.pm').'"' } @perlmodules;
my $perlmodules = join(' ', '', sort keys %perlmoduleincs, @perlmodules);
# Because of the special treatment of dependencies, we need to
# recompute $deps completely
my $deps
= join(" ", @decc_include_data,
compute_platform_depends(@{$args{generator_deps}},
sort keys %perlmoduledeps));
my $perlmodules = join(' ', '', ( sort keys %perlmoduleincs ), @perlmodules);
return <<"EOF";
$args{src} : $gen0 $deps

View File

@ -1641,12 +1641,44 @@ EOF
my $dofile = abs2rel(rel2abs(catfile($config{sourcedir},
"util", "dofile.pl")),
rel2abs($config{builddir}));
my @perlmodules = ( 'configdata.pm',
grep { $_ =~ m|\.pm$| } @{$args{deps}} );
my %perlmoduleincs = map { '"-I'.dirname($_).'"' => 1 } @perlmodules;
$deps = join(' ', $deps, compute_platform_depends(@perlmodules));
@perlmodules = map { "-M".basename($_, '.pm') } @perlmodules;
my $perlmodules = join(' ', '', sort keys %perlmoduleincs, @perlmodules);
my @perlmodules = ();
my %perlmoduleincs = ();
my %perlmoduledeps = ();
foreach my $x (('configdata.pm', @{$args{deps}})) {
# Compute (i)nclusion directory, (m)odule name and (d)ependency
my $i, $m, $d;
if ($x =~ /\|/) {
$i = $`;
$d = $';
# Massage the module part to become a real perl module spec
$m = $d;
$m =~ s|\.pm$||;
# Directory specs are :: in perl package names
$m =~ s|/|::|g;
# Full file name of the dependency
$d = catfile($i, $d) if $i;
} elsif ($x =~ /\.pm$/) {
$i = dirname($x);
$m = basename($x, '.pm');
$d = $x;
} else {
# All other dependencies are simply collected
$d = $x;
}
push @perlmodules, '"-M'.$m.'"' if $m;
$perlmoduledeps{$d} = 1;
$perlmoduleincs{'"-I'.$i.'"'} = 1 if $i;
}
# Because of the special treatment of dependencies, we need to
# recompute $deps completely
my $deps
= join(" ", compute_platform_depends(@{$args{generator_deps}},
sort keys %perlmoduledeps));
my $perlmodules = join(' ', '', ( sort keys %perlmoduleincs ), @perlmodules);
return <<"EOF";
$args{src}: $gen0 $deps
\$(PERL)$perlmodules "$dofile" "-o$target{build_file}" $gen0$gen_args > \$@

View File

@ -790,12 +790,44 @@ EOF
my $dofile = abs2rel(rel2abs(catfile($config{sourcedir},
"util", "dofile.pl")),
rel2abs($config{builddir}));
my @perlmodules = ( 'configdata.pm',
grep { $_ =~ m|\.pm$| } @{$args{deps}} );
my %perlmoduleincs = map { '"-I'.dirname($_).'"' => 1 } @perlmodules;
$deps = join(' ', $deps, compute_platform_depends(@perlmodules));
@perlmodules = map { "-M".basename($_, '.pm') } @perlmodules;
my $perlmodules = join(' ', '', sort keys %perlmoduleincs, @perlmodules);
my @perlmodules = ();
my %perlmoduleincs = ();
my %perlmoduledeps = ();
foreach my $x (('configdata.pm', @{$args{deps}})) {
# Compute (i)nclusion directory, (m)odule name and (d)ependency
my $i, $m, $d;
if ($x =~ /\|/) {
$i = $`;
$d = $';
# Massage the module part to become a real perl module spec
$m = $d;
$m =~ s|\.pm$||;
# Directory specs are :: in perl package names
$m =~ s|/|::|g;
# Full file name of the dependency
$d = catfile($i, $d) if $i;
} elsif ($x =~ /\.pm$/) {
$i = dirname($x);
$m = basename($x, '.pm');
$d = $x;
} else {
# All other dependencies are simply collected
$d = $x;
}
push @perlmodules, '"-M'.$m.'"' if $m;
$perlmoduledeps{$d} = 1;
$perlmoduleincs{'"-I'.$i.'"'} = 1 if $i;
}
# Because of the special treatment of dependencies, we need to
# recompute $deps completely
my $deps
= join(" ", compute_platform_depends(@{$args{generator_deps}},
sort keys %perlmoduledeps));
my $perlmodules = join(' ', '', ( sort keys %perlmoduleincs ), @perlmodules);
return <<"EOF";
$args{src}: "$gen0" $deps
"\$(PERL)"$perlmodules "$dofile" "-o$target{build_file}" "$gen0"$gen_args > \$@

View File

@ -2442,17 +2442,39 @@ EOF
} elsif ($dest eq '') {
$ddest = '';
} else {
$ddest = cleanfile($sourced, $_, $blddir);
$ddest = cleanfile($sourced, $dest, $blddir);
# If the destination doesn't exist in source, it can only be
# a generated file in the build tree.
if ($ddest eq $src_configdata || ! -f $ddest) {
$ddest = cleanfile($buildd, $_, $blddir);
$ddest = cleanfile($buildd, $dest, $blddir);
}
}
foreach (@{$depends{$dest}}) {
my $d = cleanfile($sourced, $_, $blddir);
my $d2 = cleanfile($buildd, $_, $blddir);
foreach my $f (@{$depends{$dest}}) {
# If the dependency destination is generated, dependencies
# may have an extra syntax to separate the intended inclusion
# directory from the module to be loaded: a | instead of a
# / as directory separator.
# Do note that this has to be handled in the build file
# template as well.
# $i = inclusion path in source directory
# $i2 = inclusion path in build directory
# $m = module path (within the inclusion path)
# $i = full module path in source directory
# $i2 = full module path in build directory
my $i; my $i2; my $m; my $d; my $d2;
if ($unified_info{generate}->{$ddest}
&& $f =~ m/^(.*?)\|(.*)$/) {
$i = $1;
$m = $2;
$i = cleanfile($sourced, $i, $blddir);
$i2 = cleanfile($buildd, $i, $blddir);
$d = cleanfile($sourced, "$i/$m", $blddir);
$d2 = cleanfile($buildd, "$i/$m", $blddir);
} else {
$d = cleanfile($sourced, $f, $blddir);
$d2 = cleanfile($buildd, $f, $blddir);
}
# If we know it's generated, or assume it is because we can't
# find it in the source tree, we set file we depend on to be
@ -2462,13 +2484,20 @@ EOF
keys %{$unified_info{generate}})
|| ! -f $d) {
$d = $d2;
$i = $i2;
}
if ($i) {
# Put together the computed inclusion dir with the
# original module name. Do note that we conserve the
# Unixly path syntax for the module path.
$d = "$i|$m";
}
$unified_info{depends}->{$ddest}->{$d} = 1;
# Fix up associated attributes
$unified_info{attributes}->{depends}->{$ddest}->{$d} =
$attributes{depends}->{$dest}->{$_}
if defined $attributes{depends}->{$dest}->{$_};
$attributes{depends}->{$dest}->{$f}
if defined $attributes{depends}->{$dest}->{$f};
}
}
@ -2638,7 +2667,9 @@ EOF
next if $dest eq "";
foreach my $d (keys %{$unified_info{depends}->{$dest}}) {
next unless $d =~ /\.(h|pm)$/;
my $i = dirname($d);
# Take into account when a dependency uses the inclusion|module
# syntax
my $i = $d =~ m/\|/ ? $` : dirname($d);
my $spot =
$d eq "configdata.pm" || defined($unified_info{generate}->{$d})
? 'build' : 'source';

View File

@ -75,6 +75,9 @@ GENERATE[include/openssl/x509_vfy.h]=include/openssl/x509_vfy.h.in
GENERATE[include/crypto/bn_conf.h]=include/crypto/bn_conf.h.in
GENERATE[include/crypto/dso_conf.h]=include/crypto/dso_conf.h.in
DEPEND[crypto/params_idx.c \
include/internal/param_names.h \
include/openssl/core_names.h]=util/perl|OpenSSL/paramnames.pm
GENERATE[crypto/params_idx.c]=crypto/params_idx.c.in
GENERATE[include/internal/param_names.h]=include/internal/param_names.h.in
GENERATE[include/openssl/core_names.h]=include/openssl/core_names.h.in

View File

@ -461,18 +461,15 @@ C<libmandatory.a> is strong, while the dependency between C<libfoo.a>
and C<libbar.a> and C<libcookie.a> is weak. See the description of
B<weak> in L</Known attributes> for more information.
B<DEPEND> is a bit more involving when used with I<item>s that are
generated with B<GENERATE>. This is described more in depth below.
=item B<GENERATE[>I<item>B<]> B<=> I<generator> I<generator-arg> ...
This specifies that the I<item> is generated using the I<generator>
with the I<generator-arg>s as arguments, plus the name of the output
file as last argument.
For I<generator>s where this is applicable, any B<INCLUDE> statement
for the same I<item> will be given to the I<generator> as its
inclusion directories. Likewise, any B<DEPEND> statement for the same
I<item> will be given to the I<generator> as an extra file or module
to load, where this is applicable.
The build file generators must be able to recognise the I<generator>.
Currently, they at least recognise files ending in C<.pl>, and will
execute them to generate the I<item>, and files ending in C<.in>,
@ -480,6 +477,42 @@ which will be used as input for L<OpenSSL::Template> to generate
I<item> (in other words, we use the exact same style of
L</Perl nuggets> mechanism that is used to read F<build.info> files).
For I<generator>s where this is applicable, any B<INCLUDE> statement
for the same I<item> will be given to the I<generator> as its
inclusion directories.
Likewise, For I<generator>s where this is applicable, any B<DEPEND>
statement for the same I<item> will be given to the I<generator> as an
extra file or module to load, where this is applicable.
=over 4
=item The B<DEPEND> statement may be problematic:
Depending on what generator is used, a B<DEPEND> statement also acts
as an B<INCLUDE> statement for the directory where the I<file> is
located. In some cases, that's not quite feasible, because a module
isn't meant to be loaded by filename only and may require a nondefault
separation between the implied inclusion directory and the intended module
name.
=item ... but there is a solution:
To enable that sort of separation, B<DEPEND> can use a slightly
different I<file> syntax, that looks like this:
B<DEPEND[>I<items>B<]> B<=> I<dir>|I<module>
The I<module> must be specified in a way that makes sense for the generator.
For example, when the generator implies perl (ends with C<.in>) and depends
on the module F<OpenSSL::foo> - a.k.a. F<OpenSSL/foo.pm> - which lives in
F<util/perl>, it feasible to have something like this:
GENERATE[something.c]=something.c.in
DEPEND[something.c]=util/perl|OpenSSL/foo.pm
=back
=item B<SOURCE[>I<item>B<]> B<=> I<file> ...
Collects filenames that will be used as source files for I<item>.