OrePAN2::S3 - Manage a DarkPAN CPAN mirror on Amazon S3
# via the bash wrapper (recommended)
orepan2-s3 add My-Dist-1.0.tar.gz
orepan2-s3 index
# or directly via the modulino
orepan2-s3-index create --upload
# add and index a new distribution
orepan2-s3-index inject My-App-1.0.0.tar.gz
# upload artifacts listed in config file
orepan2-s3-index upload-artifacts
This class is used to add distributions to your own DarkPAN repository housed on Amazon's S3 storage. It leverages OrePAN2 to create and maintain your own DarkPAN repository. You can read more about setting up a DarkPAN on Amazon using S3 + Cloudfront here.
You can read more about creating a secure static website using Amazon S3 here.
orepan2-s3 options command
Perl script for maintaining a DarkPAN mirror using S3 + CloudFront.
NOTE: orepan2-s3 is the bash script that calls this Perl class
that doubles as a modulino. The documentation here refers to the
script (not the class).
-h, --help Display this help message
-b, --bucket-name Overrides bucket in configuration
-c, --config-file Name of the configuration file (default: ~/.orepan2-s3.json)
-o, --output Name of the output file
-p, --profile Your AWS profile if not provided in configuration
-n, --profile-name Name of a profile inside the config file
-t, --template Name of a template that will be used as the index.html page
-d, --distribution Path to distribution tarball
-u, --upload Upload files after processing (for create-index, create-docs)
-U, --url Cloudfront URL
index.html from the mirror's manifest file.02packages.details.txt.gz).index: files: section of the config file.The preferred way of using this utility is through the bash wrapper.
The following commands are available only through the orepan2-s3
bash wrapper, not the modulino directly:
The configuration file for orepan2-s3 is a JSON file that can
contain multiple profiles (or none). Each profile represents a DarkPAN
S3 repository. The format should look something like this:
{
"default" : "bedrock",
"tbc" : {
"author_path": "D/DU/DUMMY",
"AWS": {
"profile" : "prod",
"region" : "us-east-1",
"bucket" : "tbc-cpan-mirror",
"prefix" : "orepan2"
},
"CloudFront" : {
"DistributionId" : "E2ABCDEFGHIJK"
}
},
"bedrock" : {
"author_path": "D/DU/DUMMY",
"index" : {
"template" : "/path/to/template",
"files": {
"src" : "dest"
}
},
"AWS": {
"profile" : "prod",
"region" : "us-east-1",
"bucket" : "cpan.openbedrock.net",
"prefix" : "orepan2"
},
"CloudFront" : {
"DistributionId" : "E2JKLMNOPQRXYZ",
"InvalidationPaths" : [],
"url" : "https://cpan.openbedrock.net/orepan2"
}
}
}
Each profile can contain the keys described below. If you only have one profile you don't need to place it in a 'default' section.
The value for the 'default' key can be the name of a profile or a hash of the profile.
author_path
Overrides default D/DU/DUMMY. For a personal DarkPAN you should
place all modules in one path.
index
This section allows you to create custom template for the DarkPAN home page.
template
The name of a template file that will be parsed and uploaded as
/index.html. If you do not provide a template file a default
template is used. The default template is a Template::Toolkit style
template. To see the default template use the dump-template command
to the orepan2-s3-index script.
orepan2-s3-index dump-template
The templating process is provided with these variables:
utils
A blessed reference to an object with one method (module_name) that
returns a version of the module name suitable for use as unique CSS id.
repo
A hash of key value pairs where the key is the name of a DarkPAN distribution and value is an array or arrays. Each array is of the form:
[0] => Perl module name
[1] => Module version
localtime
The current time and date as a string.
pod_links
A hash where the keys are distribution names and the values are links to the POD for a module.
readme_links
A hash where the keys are distribution names and the values are links to a README for a module.
NOTE: Sometimes the README and the POD will contain the same information.
files
A hash of source/destination pairs that specify additional files you want uploaded to your S3 bucket.
Example:
"files": {
"/home/rlauer/git/some-project/foo.css" : "/css/foo.css",
"/home/rlauer/git/some-project/foo.js" : "/javascript/foo.js"
}
AWS
profile
The IAM profile that allows access to the S3 bucket and CloudFront.
region
AWS region. Default: us-east-1
bucket
S3 bucket name
prefix
The prefix where the CPAN distribution files will be stored. Default: orepan2.
CloudFront
NOTE: Your profile must have the ability to invalidate the CloudFront cache!
DistributionId
CloudFront distribution id
InvalidationPaths
OrePAN2::S3 is designed to work with CloudFront + Amazon
S3. CloudFront is a CDN and will read content from your S3 bucket when
clients make HTTP requests. CloudFront will cache content to avoid
costly reads to the S3 bucket. If you change some of the static assets
(like the index page) you may want to invalidate the cache to see
your new assets. You could wait until the cache is updated (the
default time is 24 hours)...but why? The script will automatically
invalidate the cache for you if you tell it what assets to invalidate
when you add a new distribution.
An array of additional paths to invalidate when adding new distributions.
Note: There is no additional charge for adding additional paths. Each invalidation batch is considered as one billing unit by AWS. However, keep in mind you get 1000 invalidation paths for free each month. Thereafter each path costs $0.005 per path.
custom_sections
This section contains key/value pairs where the key is the name of a variable that will be exposed to your template and the values are a two element array that contains a regular expression and possible regexp flags. The script will use the regexp to filter your distributions and add them to a hash whose name is the key you provided.
The purpose of this section is to allow you to possibly organize your distributions under possible HTML headings.
Example:
"custom_sections" : {
"plugins" : ["^BLM\-(?!Startup)", "xsm"],
"app_plugins" : ["^BLM\-Startup", "xsm"],
}
...then in your template:
<h1>Application Plugin Index</h1>
[% FOREACH distribution = app_plugins.sort %]
<h2>
<span class="collapse-section-icon">▼</span>
[% distribution %]
[% IF readme_links.$distribution %]
<a title="README" class='doc-link' href="[% readme_links.$distribution %]"><span class="material-symbols-outlined">docs</span></a>
[% END %]
[% IF pod_links.$distribution %]
<a title="pod" class='doc-link' href="[% pod_links.$distribution %]"><span class="material-symbols-outlined">docs</span></a>
[% END %]
</h2>
<ul class="collapsable" id="[% utils.module_name(distribution) %]">
[% FOREACH module IN app_plugins.$distribution %]
<li>[% module.0 %]</li>
[% END %]
</ul>
[% END %]
<hr>
Rob Lauer - rlauer@treasurersbriefcase.com
OrePAN2, Amazon::S3::Lite, DarkPAN::Utils, CLI::Simple, Template
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.