🦐 Ebi: GitHub repositories contents search

Searches files within GitHub repositories. It can be used as a command line tool or a library.

Ebi (えび) is Japanese for prawn/shrimp, and intends to be a small little tool to crawl through your sea of code on GitHub, finding you nuggets of information.

Command Line Usage

Global installation (recommmended)

npm install --global ebi

When you run the tool, it will automatically notify you if there is a newer version of it available for you to update to.

You can disable notifications if you'd prefer not to be notified about updates.

No installation

npx ebi

The npx command lets you use this tool without installing it. However, each time you use npx it downloads the whole package from the npm registry, which takes a while. That's why global installation is reccommended.

Note: If this tool is globally installed, npx ebi will use that globally installed version rather than downloading.

Usage

1. Set up a GitHub personal access token (with all repo scopes) assigned to the GITHUB_PERSONAL_ACCESS_TOKEN environment variable

2. Pass in the list of space-separated repositories as arguments:

   ebi <command> Financial-Times/ebi Financial-Times/tako
   

3. If ebi is silently failing you can turn on --verbose to see more logging

Examples

Show help

ebi --help

Input the repositories to the ebi command either via stdin or args. Determine whether a repo has a Procfile

$ echo -e "Financial-Times/ebi" | ebi contents Procfile

$ ebi contents Procfile Financial-Times/ebi

Find all the node engines and their versions in package.json

$ cat repositories.txt | ebi package:engines

For more examples see Usage Examples.

JSON output

To output as JSON, you can use the --json flag eg, ebi package:engines --json.

The output format of the JSON is

{
    type,
    repository,
    filepath,
    fileContents,
    [search],
    [regex],
    [error]
}

Field	Values	Description
type	match, no-match, error	Type of result
repository	Financial-Times/ebi	The full repository path
filepath	package.json	The filepath searched for
fileContents	{\n \"name\": \"ebi\",\n ... }	The file contents serialized as a string
search	name	[optional] The search term
regex	no.*	[optional] The regex used for search (ie, --regex)
error	404 ERROR: ...	[optional] The error message if the result is of type error

Library Usage

To use ebi as a library in a NodeJS project:

npm install ebi

Require ebi, and run a search:

const {
  contentsSearch,
  packageSearch,
  packageEnginesSearch
} = require('ebi');

// Get a repository list
const repoList = [
  'Financial-Times/ebi'
];

const { getResults, resultsAsync } = await contentsSearch({
  filepath: 'package.json',
  search,       // Optional
  token,        // Optional
  regex,        // Optional
  limit         // Optional
})(repoList);

// Get results synchronously
const {
    allResults,
    searchMatches,
    searchNoMatches,
    searchErrors
} = await getResults();

// Get results asynchronously
const allAsyncResults = await Promise.all(
    resultsAsync.map(promise => {
        // Need to handle errors eg, if file is not found
        return promise.catch(e => e);
    })
);

Similarly:

const { getResults, resultsAsync } = await packageSearch({
  search: 'ebi',  // Optional
  token,          // Optional
  regex,          // Optional
  limit           // Optional
})(repoList);

const { getResults, resultsAsync } = await packageEnginesSearch({
  search: 'node'  // Optional
  token,          // Optional
  regex,          // Optional
  limit           // Optional
})(repoList);

See JSDoc comments for descriptions of the parameters. VS Code also has JSDoc support in the editor. To turn it on, either put // @ts-check on the top of a file or enable the checkJS compiler option.

See examples folder for more usage examples.

Setting up your GitHub personal access token

This tool requires a GitHub personal access token with all repo scopes. This is very powerful as it has access to modify a repository's settings, so it is strongly recommended that you store this token securely.

1. Create a new GitHub personal access token with all repo scopes
2. Store the token in the GITHUB_PERSONAL_ACCESS_TOKEN environment variable. You should avoid passing your GitHub personal access token directly to any CLI arguments as it will be visible in your shell history. There are a few options to do this:
   1. If you work at Financial Times, you can follow the GitHub personal access token docs
   2. Use your operating system's password management system (e.g. Keychain on macOS) to store and retrieve GITHUB_PERSONAL_ACCESS_TOKEN in your shell's rcfile (e.g. ~/.bashrc), then restart your terminal
   3. If all else fails, you can set it in your terminal with GITHUB_PERSONAL_ACCESS_TOKEN=[github-token]
   4. If you want use a different token, you can pass in --token=$GITHUB_PERSONAL_ACCESS_TOKEN when you run the commands

Development

1. Install nvm and use the correct node version

   nvm use
   

2. Install dependencies

   npm install
   

3. Run with:

   ./bin/ebi.js <command>
   

Testing

To run linting and tests

npm test

To just run linting

npm run lint

To fix linting issues

npm run lint-fix

To just run unit tests

npm run unit-test

To watch files and run unit tests

npm run unit-test:watch

To watch individual files and run unit tests

npm run unit-test:watch -- [file...]
# eg,
npm run unit-test:watch -- test/lib/get-repositories.test.js

Code formatting with Prettier

This repo uses prettier for code formatting. To make the most of this when working locally:

• Install the prettier-vscode extension in the extension side bar
• Update your settings to format files on save. This will check your file meets the prettier guidelines and will fix it each time you save. You can update the setting at Code --> Preferences --> Settings --> update "editor.formatOnSave": true

To make sure no eslint rules conflict with the prettier config, we have eslint-config-prettier. This can be run with:

npm run eslint-check

Publishing a release

CircleCI is set up to publish a release to npm. To release:

1. Create a new release from GitHub
   1. Tag it with a semver range and a v prefix eg, v1.2.3 or v1.4.5-beta.3
   2. Create a title and description
   3. Publish release
2. Wait for CircleCI to finish building the tag release, and once done, it will be appear at npmjs.com/package/ebi