NAME

Module::ScanDeps::Static - a cleanup of rpmbuild's perl.req

SYNOPSIS

scandeps-static [options] Module

If "Module" is not provided, the script will read from STDIN.

my $scanner = Module::ScanDeps::Static->new({ path => 'myfile.pl' });
$scanner->parse;
print $scanner->format_text;

DESCRIPTION

This module is a mashup (and cleanup) of the /usr/lib/rpm/perl.req file found in the rpm build tools library (see "LICENSE") below.

Successful identification of the required Perl modules for a module or script is the subject of more than one project on CPAN. While each approach has its pros and cons I have yet to find a better scanner than the simple parser that Ken Estes wrote for the rpm build tools package.

Module::ScanDeps::Static is a simple static scanner that essentially uses regular expressions to locate use, require, parent, and base in all of their disguised forms inside your Perl script or module. It's not perfect and the regular expressions could use some polishing, but it works on a broad enough set of situations as to be useful.

Only direct dependencies are returned by this module. If you want a recursive search for dependencies, use find-requires included in this distribution.

Comparison to Other Scanners

Two other CPAN scanners cover similar ground: Perl::PrereqScanner, built on PPI, and Perl::PrereqScanner::NotQuiteLite (here, "NQLite"), which uses its own lexer. The comparison below reflects hands-on testing against both, not just a reading of their documentation.

Speed. This module is regex/line-based rather than a true tokenizer, and that's a deliberate trade-off, not an oversight - it's the source of most of the speed difference. On a representative 414-line file, this module scans in roughly 1.7ms; NQLite takes roughly 6ms; PPI-based Perl::PrereqScanner takes roughly 45ms. PPI's own documentation acknowledges this cost directly, describing itself as "painful... with very large files." The trade-off cuts the other way too: a real tokenizer is structurally immune to a class of edge case (an unbalanced brace inside a string or heredoc, for example) that this module's regex approach can, in principle, still be fooled by, even though no such failure has been found in testing against real-world code.

Dependency classification. All three tools distinguish a hard requirement from something merely conditional or optional, but this module is alone in giving the developer an explicit way to state that judgment rather than have it inferred. NQLite decides recommends vs. suggests purely from code structure - specifically, whether the call is a bare eval or one of two specific, by-name-recognized CPAN modules (Module::Runtime and Class::Load) - with no way for a developer to say "no, that one's actually important" short of restructuring their code to match what the scanner looks for. This module instead classifies a guarded dependency as suggests by default (or recommends project-wide via eval_recommends), and lets a ## scandeps: recommends or ## scandeps: suggests comment immediately after the guarding statement override that default per-instance. Neither other tool has an equivalent escape hatch.

This module also warns - rather than silently picking a winner - when a module is both a hard requirement in one place and merely recommended or suggested in another, since that's a real contradiction in the source worth surfacing to the developer, not something a scanner should resolve on its own.

Version numbers. Both Perl::PrereqScanner and NQLite report every dependency's version as a fixed 0 placeholder - not "unknown", literally the string 0 regardless of what's actually installed or required. This module performs a real version lookup (see --add-version) and reports what it actually finds.

Reporting a version number or 0 both have their advantages and disadvantages. Users of these scanners can always edit the output and modify the version number requirements, however this scanner opted to choose the currently installed version. This assumes you are developing your module against a known environment, and the idea is that while other versions of a module might work, the version in your working environment is known to work with your module. Selecting 0 means any version will do - but consider a module like List::Util running under Perl 5.10. While List::Util has been in core since 5.7.3, its feature list has grown significantly. You may be using features that were not present in that version of List::Util. Shipping with the minimum version of List::Util set to 0 is almost certainly going to fail on earlier versions of Perl.

The true minimum - the exact release in which every feature your code actually relies on first appeared - would in principle be the most accurate answer. But determining that reliably would mean downgrading each dependency in turn and re-running your test suite against every prior release, which presumes you have test coverage specifically exercising the features you're relying on, for every dependency you use. That's not a reasonable expectation in practice - reporting the currently installed version is a deliberate trade-off in favor of a version known to work over one merely presumed to.

Self-referential modules. Scanning a whole project can turn a sibling module into a false positive - file A's use of file B's package isn't an external dependency at all. NQLite handles this with --private, a manually maintained list of module names to exclude. This module's --filter does the same job automatically, by detecting package declarations across the batch being scanned, at the cost of only working when the sibling file is actually part of the same scan.

What this module doesn't attempt. NQLite can infer a minimum Perl version from newer syntax (signatures, say, and similar) appearing in the source without an explicit use v5.x declaration; this module does not attempt that. NQLite also has an opt-in CPAN::Common::Index integration that can deduplicate module names belonging to the same distribution - deliberately not pursued here, since it requires a network round-trip (or a locally maintained mirror) per module found at scan time, in exchange for a purely cosmetic shortening of the output that changes nothing about what actually gets installed. Finally, no static scanner - this one included - can discover a dependency loaded through a genuinely dynamic mechanism (a module name computed at runtime and passed to something like Module::Load); that's a fundamental limit of static analysis, not a gap specific to any one tool. See "Dynamic Module Loading" below.

OPTIONS

--add-version, -a        add version numbers to output
--no-add-version         don't add version numbers to output
--core                   include core modules (default)
--no-core                don't include core modules
--cpanfile-file PATH     write a cpanfile combining all three tiers to PATH
--eval-recommends        classify unannotated eval-wrapped deps as recommends, not suggests
--file-list, -L PATH     scan a batch of files listed one per line in PATH
--filter, -f             exclude modules that are this project's own packages
--help, -h               help
--include-require, -i    include 'require'd modules
--no-include-require     don't include required modules
--json, -j               output JSON formatted list
--min-core-version, -m   minimum version of perl to consider core
--path, -p PATH          file to scan (alternative to the positional argument)
--raw, -r                raw output
--recommend-require, -R  classify indented (non-eval) conditional requires as recommends
--recommends-file PATH   write the recommends tier to PATH
--requires-file PATH     write the requires tier to PATH
--separator, -s          separator for output (default: =>)
--suggests-file PATH     write the suggests tier to PATH
--text, -t               output as text (default)
--version, -v            version

Examples

scandeps-static --no-core lib/Some/Module.pm

scandeps-static --json lib/Some/Module.pm

Use the find-requires script included in this distribution to recurse directories and create dependency files like cpanfile.

OPTION DETAILS

WHAT IS A DEPENDENCY?

For the purposes of this module, dependencies are identified by looking for Perl modules and other Perl artifacts declared using use, require, parent, or base. The script will also consider Moo/Role::Tiny modules included using with, and Moose inheritance declared using extends - both are treated identically to use parent/use base.

If the module contains a require statement, by default the require must be flush up against the left edge of your script without any whitespace between it and beginning of the line. This is the default behavior to avoid identifying require statements that are embedded in if statements. If you want to include all of the targets of require statements as dependencies, set the include-require option to a true value.

Dependency Tiers

Every dependency found is classified into one of three tiers, matching the requires/recommends/suggests relationship types defined by CPAN::Meta::Spec:

This module makes no attempt to infer which of recommends or suggests is correct by analyzing what a guarded dependency's failure path actually does (for example, whether it dies or degrades gracefully). That was a deliberate decision, not an oversight: whether a dependency is merely nice-to-have or genuinely important is a judgment about a project's own design, not a fact recoverable from code structure - the same code shape (eval { require Foo }) is written identically by an author who considers Foo essential and one who considers it optional. Guessing at that judgment and presenting the guess as if it were derived from parsing would be dishonest about what static analysis can actually know. Use the annotation below to state it explicitly instead.

Structural Classification

Only a genuinely unconditional use/require is ever classified as requires. Anything the author has structurally guarded is pulled into one of the soft tiers:

The ## scandeps: Annotation

Because the recommends vs. suggests distinction is a judgment call this module deliberately does not try to infer (see "Dependency Tiers"), an author can state that judgment explicitly with a comment:

eval { require Foo::Bar; };  ## scandeps: recommends

eval { require Foo::Bar; 1; } or die $@;  ## scandeps: suggests

eval {
    require Foo::Bar;
    1;
} or do {
    warn "Foo::Bar unavailable, continuing without it\n";
};  ## scandeps: recommends

The annotation must appear immediately after the semicolon that ends the whole statement - not the eval block's own closing brace, but wherever the statement as a whole actually terminates, which may be after a trailing or die ...; or or do { ... };. An explicit annotation always overrides both this module's structural default and the --eval-recommends setting.

This annotation is only recognized on the brace form of eval (eval { ... }). It is not supported on the string form (eval "...") - anyone still writing string eval for this purpose is not expected to participate in this feature.

Conflicting Classifications

A single module can legitimately be a hard requirement in one file and guarded in another - both facts can be true of the same project at once. When that happens, this module does not silently pick a winner: it emits a warning to STDERR naming the conflicting module, and leaves require, recommends, and suggests exactly as found. Resolving the contradiction belongs in the project's own source, not in this tool - either the guarded usage no longer needs to hedge, since the module is guaranteed to be present anyway because of the other, unconditional usage, or the unconditional usage should really be conditional too.

The same warning fires if a module appears in both recommends and suggests at once.

Self-Referential Modules

When scanning a whole project (typically via --file-list), a module in one file may use a sibling module declared with package in another file in the same project - not a real external dependency at all. --filter excludes any module name that appears anywhere in the batch as a package declaration; see "--filter, -f" for its (context-dependent) default.

Dynamic Module Loading

This module performs static analysis: it reads source as text and looks for recognizable patterns. It cannot execute your code, and so cannot know the value of a variable at runtime. A module loaded via a computed name - for example Module::Load::load($module) where $module is a variable - is invisible to this scanner and will never appear in require, recommends, or suggests, no matter how the code is structured.

This is a real and legitimate use case for modules like Module::Load, Module::Runtime, and Class::Load: loading a plugin or driver whose name genuinely isn't known until the program runs. For that use case, invisibility to any static scanner is an unavoidable trade-off, not a defect in this module.

It becomes an antipattern, though, when one of these modules is called with a name that is known at write time:

Module::Load::load('Term::ANSIColor');

The module name here is a literal string, not runtime data - a plain

eval { require Term::ANSIColor; };

would behave identically at runtime, while remaining visible to this scanner and annotatable with ## scandeps:. Reach for Module::Load and similar modules only when the module name is genuinely computed at runtime - for loading plugins or other modules that are already known to be part of your project's own architecture - not as a general-purpose substitute for require.

Do not expect this scanner to understand dynamic module loading. If a project genuinely depends on a module that can only be discovered at runtime, that dependency needs to be declared by hand (for example, in a cpanfile or buildspec.yml) - this module cannot discover it.

MINOR IMPROVEMENTS TO perl.req

CAVEATS

There are still many situations (including multi-line statements) that may prevent this module from properly identifying a dependency. As always, YMMV.

METHODS AND SUBROUTINES

new

new(options)

Returns a Module::ScanDeps::Static object.

Options

get_require

After calling the parse() method, call this method to retrieve a hash containing the dependencies and (potentially) their version numbers.

$scanner->parse;
my $requires = $scanner->get_require;

get_perlreq

Returns a hash ref of Perl version requirements discovered while parsing (keyed by 'perl'). Populated for use 5.010; / require 5.010; style statements. Pair with get_require.

$scanner->parse;
my $perlreq = $scanner->get_perlreq;  # { perl => '5.010', ... }

parse

Scans the specified input and returns a list of Perl module dependencies.

Use the get_dependencies method to retrieve the dependencies as a formatted string or as a list of dependency objects. Use the get_require and get_perlreq methods to retrieve dependencies as a list of hash refs.

my $scanner = Module::ScanDeps::Static->new({ path => 'my-script.pl' });
my @dependencies = $scanner->parse;

get_dependencies

Returns a formatted list of dependencies or a list of dependency objects.

As JSON:

print $scanner->get_dependencies( format => 'json' )

[
  {
   "name" : "Module::Name",
   "version" : "version"
  },
  ...
]

..or as text:

print $scanner->get_dependencies( format => 'text' )

Module::Name => version
...

In scalar context in the absence of an argument returns a JSON formatted string. In list context will return a list of hashes that contain the keys "name" and "version" for each dependency.

Note: this context-sensitivity only applies when none of json, text, or raw is set (or when format => 'json' / format => 'text' is passed explicitly). If the json option is true, get_dependencies always returns a scalar JSON string, even when called in list context.

format_text

$scanner->parse;
print $scanner->format_text;

Returns the dependency list as a formatted text string, one module per line, honoring the separator and raw options. Core modules are omitted when core is false.

format_json

my $json     = $scanner->format_json;   # scalar context
my @requires = $scanner->format_json;   # list context

In scalar context returns a pretty-printed JSON string; in list context returns a list of hash refs of the form { name => ..., version => ... }. Core modules are omitted when core is false. Any arguments are treated as a seed list and prepended to the results.

format_cpanfile

$scanner->parse;
print $scanner->format_cpanfile;

Returns a single cpanfile (see Module::CPANfile) combining all three tiers, using native requires/recommends/suggests DSL syntax. Versions are included only where actually detected (see add_version); otherwise the unversioned form is written. Applies the same core/filter filtering as format_json, since it's built on top of it rather than duplicating that logic.

is_core

my $bool = $scanner->is_core($module);
my $bool = $scanner->is_core("$module $version");

Returns true if $module is considered core. A module is core when Module::CoreList reports its first release at or before min_core_version and the module has never been removed from core at any point in its history.

A module that was core at some earlier Perl but has since been removed is always treated as non-core, regardless of how min_core_version compares to the version it was removed at. There is no way to know which Perl an end user actually has installed - a version comparison against a single reference point only protects against removals that happen to fall on one particular side of it, so the only safe answer once a module has ever been removed is to always require it explicitly.

min_core_version

my $numified = $scanner->min_core_version;

Returns the min_core_version option numified via version (e.g. 5.008009) for comparison inside is_core. Note this is distinct from the generated get_min_core_version accessor, which returns the raw stored value.

get_module_version

my $info = $scanner->get_module_version($module, @include_path);

Returns a hash ref describing $module:

{ module => ..., version => ..., path => ..., file => ... }

Searches @include_path (defaulting to @INC) for the module and extracts its version via ExtUtils::MM-parse_version>. If $module already carries a version ("Foo::Bar 1.23"), that version is returned without a filesystem lookup.

add_require

$scanner->add_require($module);
$scanner->add_require($module, $version);

Registers $module as a dependency, optionally with $version. When no version is supplied and the add_version option is true, the installed version is looked up. Retains the higher of two versions if the module is added more than once. Returns $self.

to_rpm

my $deps = $scanner->to_rpm;

Returns the dependency list as RPM-style requirement expressions (perl(Module) >= version, plus perl >= version for any Perl version requirement). Core modules are omitted when core is false.

VERSION

This documentation refers to version 1.9.3

AUTHOR

This module is largely a lift and drop of Ken Este's perl.req script lifted from rpm build tools.

Ken Estes Mail.com kestes@staff.mail.com

The method parse is a cleaned up version of process_file from the same script.

Rob Lauer - bigfoot@cpan.org

LICENSE

This statement was lifted directly from perl.req...

The entire code base may be distributed under the terms of the GNU General Public License (GPL), which appears immediately below. Alternatively, all of the source code in the lib subdirectory of the RPM source code distribution as well as any code derived from that code may instead be distributed under the GNU Library General Public License (LGPL), at the choice of the distributor. The complete text of the LGPL appears at the bottom of this file.

This alternatively is allowed to enable applications to be linked against the RPM library (commonly called librpm) without forcing such applications to be distributed under the GPL.

Any questions regarding the licensing of RPM should be addressed to Erik Troan <ewt@redhat.com.>