The packages-pagelet is a small re-usable component that you could use to
display information about any npm package. Private, public it doesn't matter as
long as it's in a npm registry.
The packages-pagelet is build upon the Pagelet interface from the BigPipe
framework which allows you to compose pages from small fragments or components
which is you can re-use and remix in any way you want allowing for a fully
modular and high performance front-end development experience.
This module is distributed through the public npm registry and can therefor be installed using:
npm install --save packages-pagelet
The --save flag automatically tells npm to add this dependency and it's
installed version in to your package.json. If you do not have the BigPipe
framework installed, run:
npm install --save bigpipe packages-pagelet
To also install and save BigPipe in one go.
The packages-pagelet was designed with flexibility and developer freedom in
mind that why there are a lot of (important) options that can be changed and
fine tuned. So take a few minutes to read through all options to ensure you get
the most out of this module. In order to configure this module you need to
extend the module and override the desired properties. The extending of the
module is done using extend method that is exposed on the module. The extend
method follows the same schematics as in the backbone library:
'use strict';
var Packages = require('packages-pagelet');
//
// configure the module by extending it with new properties.
//
module.exports = Packages.extend({
registry: 'https://registry.nodejitsu.com/'
});The following options (keys) can be configured:
The registry property allows two different values, it can either be a string
with a trailing slash which points to the location of The npm Registry you want
to use (which is useful for a private registry) or it can be set to a
pre-configured npm-registry instance which would be preferred. The reason for
this is that the npm-registry client uses the githulk module to make API
requests to GitHub repositories to retrieve addition data about packages. As the
API of GitHub is rate limited it's vital that you use authenticated API requests
to GitHub where ever possible. See the [npm-registry] for more detailed
information about the options that can be configured.
Packages.extend({
registry: 'https://registry.nodejitsu.com/'
});Or
var Registry = require('npm-registry');
Packages.extend({
registry: new Registry({
registry: 'https://registry.nodejitsu.com/'
});
});
This might be one of the most important properties that is configurable on this module. The cache allows you to cache the result of the all gathered data per module. We support 2 different cache patterns, an async cache and an sync cache. The sync cache would be ideal during development as you can just use a simple LRU cache for this. In a production environment it might be worth to go with an async, distributed cache. Once you want to scale out to multiple node processes you still want to share this cache.
The supplied cache only needs 2 different methods:
get(key): A method that returns a JSON object or null for a given key.set(key, obj): A method that stores a JSON object for a given key.
When you are using an async cache each method will receive a callback as last argument in addition to that, the cache will be set using fire and forget as we will not wait until the supplied callback is answered before we rendering the content.
var store = {};
Packages.extend({
cache: {
get: function (key) {
return store['.prefix@'+ key];
},
set: function (key, value) {
return store['.prefix@'+ key] = value;
}
}
});Please be aware that this cache is not automatically invalidated. So it might make sense to use either a memcached or redis cache with an expire value for this.
The packages-pagelet uses a lot of GitHub API end points to get the data
retrieval as accurate and complete as possible and to render README's correctly.
As the GitHub API is rate limited to 60 calls per hour for unauthorized and 5000
calls for authorized connections it's really important to use an authorized
githulk instance within your pagelet. The GitHulk module has 2 ways of
creating an authorized connection, either by supplying it with an token option
which is your OAuth token or by supplying the user name and password for your
account so it can use basic auth. For small and private pages it might enough to
have 5000 API calls per hour, but if you want to host the packages page in the
public you would probably need as much as possible. That's why the githulk
implements a request cache so it can do conditional requests to the GitHub API
based on the returned Etag headers from the API.
The caching is implemented in mana which is a framework for writing high available API clients. The cache API that it requires follows the same schematics as the cache that you can implement for this pagelet as it supports both an sync and async interface for retrieving and storing cache.
You don't need to manually invalidate this cache as it's automatically overridden when the Etag is no longer accepted by the GitHub API.
var GitHulk = require('githulk')
var hulk = new GitHulk({
token: 'my secret oh.. auth token',
cache: {
get: function (key) {},
set: function (key, value) {}
}
});The githulk instance that you've created can also be passed in to your custom
npm-registry client so it can also re-use the same instance and credentials
for when it does a lookup.
When we cache the data for the pagelet we automatically run the name of the
module through the key method which prefixes the key so it can be used as
cache key. We are currently prefixing the key with the major version of this
module. So when we make a backwards incompatible change to the data structure
your page wouldn't die because it had an incorrect data structure.
But if you want absolute control over the process, you can just create your own key prefixer:
Packages.extend({
key: function key(name, version) {
return 'foo-bar-prefixed-key:'+ name +':'+ version;
}
});This sets the expiration values for the cache. There are 2 properties can be
configured on this object, the latest which expires the latest version number
for a given package and data which expires the resolved data for the resolved
latest version number. The values should be set in miliseconds.
Packages.extend({
expire: {
latest: 1000,
data: 1000 * 1000 * 1000
}
});So a fully configured and customized packages-pagelet should look something
like this:
'use strict';
var GitHulk = require('githulk')
, Registry = require('npm-registry')
, Packages = require('packages-pagelet')
, DistributedJSONCache = require('no-existing-module-implement-it-yourself');
var githulk = new GitHulk({
cache: new DistributedJSONCache(),
token: 'your oauth token'
})
, registry = new Registry({
githulk: githulk,
registry: 'https://registry.nodejitsu.com/'
});
module.exports = Packages.extend({
cache: new DistributedJSONCache(),
registry: registry,
githulk: githulk
});Correctly resolving a single package requires a lot of HTTP requests and computation. This pagelet does support caching, but this is of course only useful if you start of with a completely cached result set as packages are only cached when they are accessed for the first time (depending on how you implemented your own cache of course).
To make the population of cache a bit easier we've exposed our internal resolve method so it can be used out side of the pagelet as well.
var resolve = require('packages-pagelet').resolve;
[
/* assume a list of modules you want to pre-cache */
].forEach(function (name) {
resolve(name, function (err, data) {
// Cache the data.
});
});The resolve method accepts 3 arguments: