Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Update API to reflect changes to CLI interaction #203

Merged
merged 1 commit into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ a set of sample data and behaviour.

## Usage

Simply running `dev/build` will take care of most sample data setup.
Simply running `sake db:build` will take care of most sample data setup.

In order to use any of the optional test behaviour targeted at modules,
install the module and remove the `_manifest_exclude` file from the relevant folder.
Expand All @@ -26,7 +26,7 @@ For example, to test the tagfield module, remove the `frameworktest/code/tagfiel
The module creates some default pages for different CMS behaviours.
The CMS is intended to be perform well with a couple of thousand pages.
If you want to test the CMS behaviour for a large and nested tree,
the module includes a simple generator task: `dev/tasks/FTPageMakerTask`.
the module includes a simple generator task: `sake tasks:FTPageMakerTask`.
It will create 3^5 pages by default, so takes a while to run through.

## Configuring the amount of data
Expand All @@ -35,9 +35,9 @@ Both `FTPageMagerTask` and `FTFileMakerTask` allow the amount of generated conte
To do this, pass a comma-seprarated list of integers representing the amount of records to create at each
depth.

`$ vendor/bin/sake dev/tasks/FTPageMakerTask pageCounts=10,200,5,5`
`$ vendor/bin/sake tasks:FTPageMakerTask --pageCounts=10,200,5,5`

`$ vendor/bin/sake dev/tasks/FTFileMakerTask fileCounts=5,300,55,5 folderCounts=1,5,5,5`
`$ vendor/bin/sake tasks:FTFileMakerTask --fileCounts=5,300,55,5 --folderCounts=1,5,5,5`

## Guaranteed unique images

Expand All @@ -64,8 +64,8 @@ Usage:

```
# Generate some sample files to associate with blocks
sake dev/tasks/FTFileMakerTask
sake dev/tasks/FTPageMakerTask withBlocks=true
sake tasks:FTFileMakerTask
sake tasks:FTPageMakerTask withBlocks=true
```

## Requirements
Expand Down
6 changes: 3 additions & 3 deletions _config/extensions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,17 @@ SilverStripe\FrameworkTest\Fields\NestedGridField\NonRelationalData:
extensions:
- SilverStripe\FrameworkTest\Extension\TestDataObjectExtension

SilverStripe\ORM\DatabaseAdmin:
SilverStripe\Dev\Command\DbBuild:
extensions:
- SilverStripe\FrameworkTest\GridFieldArbitraryData\DatabaseBuildExtension
- SilverStripe\FrameworkTest\GridFieldArbitraryData\DbBuildExtension

---
Only:
moduleexists: 'silverstripe/testsession'
---
SilverStripe\TestSession\TestSessionEnvironment:
extensions:
- SilverStripe\FrameworkTest\GridFieldArbitraryData\DatabaseBuildExtension
- SilverStripe\FrameworkTest\GridFieldArbitraryData\DbBuildExtension

---
Only:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

use SilverStripe\Control\Director;
use SilverStripe\Core\Extension;
use SilverStripe\ORM\DatabaseAdmin;
use SilverStripe\Dev\Command\DbBuild;
use SilverStripe\PolyExecution\PolyOutput;
use SilverStripe\ORM\DataObject;
use SilverStripe\ORM\DB;
use SilverStripe\ORM\FieldType\DBDatetime;
Expand All @@ -13,87 +14,66 @@
/**
* Builds the table and adds default records for the ArbitraryDataModel.
*/
class DatabaseBuildExtension extends Extension
class DbBuildExtension extends Extension
{
/**
* This extension hook is on TestSessionEnvironment, which is used by behat but not by phpunit.
* For whatever reason, behat doesn't use dev/build, so we can't rely on the below onAfterbuild
* For whatever reason, behat doesn't build the db, so we can't rely on the below onAfterbuild
* being run in that scenario.
*/
protected function onAfterStartTestSession()
{
$this->buildTable(true);
$output = PolyOutput::create(
Director::is_cli() ? PolyOutput::FORMAT_ANSI : PolyOutput::FORMAT_HTML,
PolyOutput::VERBOSITY_QUIET
);
$output->startList();
$this->buildTable($output);
$output->stopList();
$this->populateData();
}

/**
* This extension hook is on DatabaseAdmin, after dev/build has finished building the database.
* This extension hook is on DbBuild, after building the database.
*/
protected function onAfterBuild(bool $quiet, bool $populate, bool $testMode): void
protected function onAfterBuild(PolyOutput $output, bool $populate, bool $testMode): void
{
if ($testMode) {
return;
}

if (!$quiet) {
if (Director::is_cli()) {
echo "\nCREATING TABLE FOR FRAMEWORKTEST ARBITRARY DATA\n\n";
} else {
echo "\n<p><b>Creating table for frameworktest arbitrary data</b></p><ul>\n\n";
}
}

$this->buildTable($quiet);

if (!$quiet && !Director::is_cli()) {
echo '</ul>';
}

$output->writeln('<options=bold>Creating table for frameworktest arbitrary data</>');
$output->startList();
$this->buildTable($output);
$output->stopList();
if ($populate) {
if (!$quiet) {
if (Director::is_cli()) {
echo "\nCREATING DATABASE RECORDS FOR FRAMEWORKTEST ARBITRARY DATA\n\n";
} else {
echo "\n<p><b>Creating database records arbitrary data</b></p><ul>\n\n";
}
}

$output->writeln('<options=bold>Creating database records arbitrary data</>');
$output->startList();
$this->populateData();

if (!$quiet && !Director::is_cli()) {
echo '</ul>';
}
}

if (!$quiet) {
echo (Director::is_cli()) ? "\n Frameworktest database build completed!\n\n" : '<p>Frameworktest database build completed!</p>';
$output->stopList();
}
$output->writeln(['<options=bold>Frameworktest database build completed!</>', '']);
}

private function buildTable(bool $quiet): void
private function buildTable(PolyOutput $output): void
{
$tableName = ArbitraryDataModel::TABLE_NAME;

// Log data
if (!$quiet) {
$showRecordCounts = DatabaseAdmin::config()->get('show_record_counts');
if ($showRecordCounts && DB::get_schema()->hasTable($tableName)) {
try {
$count = SQLSelect::create()->setFrom($tableName)->count();
$countSuffix = " ($count records)";
} catch (\Exception $e) {
$countSuffix = ' (error getting record count)';
}
} else {
$countSuffix = "";
}

if (Director::is_cli()) {
echo " * $tableName$countSuffix\n";
} else {
echo "<li>$tableName$countSuffix</li>\n";
$showRecordCounts = DbBuild::config()->get('show_record_counts');
if ($showRecordCounts && DB::get_schema()->hasTable($tableName)) {
try {
$count = SQLSelect::create()->setFrom($tableName)->count();
$countSuffix = " ($count records)";
} catch (\Exception $e) {
$countSuffix = ' (error getting record count)';
}
} else {
$countSuffix = "";
}
// We're adding one list item, but we need to do it this way for consistency
// with the rest of the db build output
$output->writeListItem($tableName . $countSuffix);

// Get field schema
$fields = [
Expand Down
69 changes: 44 additions & 25 deletions code/tasks/FTFileMakerTask.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@
use SilverStripe\Security\Security;
use SilverStripe\Core\Path;
use SilverStripe\Core\Manifest\ModuleResourceLoader;
use SilverStripe\PolyExecution\PolyOutput;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;

/**
* Creates sample folder and file structure, useful to test performance,
Expand All @@ -24,10 +28,6 @@
* recursively delete any generated ones through the following bash command in `assets/`:
* `find . -name '*Resampled*' -print0 | xargs -0 rm`
*
* Parameters:
* - reset=1: Optionally truncate ALL files and folders in the database, plus delete
* the entire `assets/` directory.
*
*
* Configuration
*
Expand Down Expand Up @@ -58,7 +58,7 @@
* - 0
*
* Flush and run:
* /dev/tasks/FTFileMakerTask?flush&reset=1
* sake tasks:FTFileMakerTask --flush --reset
*
* @todo Automatically retrieve file listing from S3
* @todo Handle HTTP errors from S3
Expand Down Expand Up @@ -180,8 +180,6 @@ class FTFileMakerTask extends BuildTask
'video.m4v' => 'SilverStripe\Assets\File',
];

protected $lineBreak = "\n<br>";

/** @var Member */
protected $anonymousMember = null;

Expand All @@ -197,7 +195,7 @@ class FTFileMakerTask extends BuildTask
*/
protected $folderCounts = [];

public function run($request)
protected function execute(InputInterface $input, PolyOutput $output): int
{
set_time_limit(0);

Expand All @@ -213,15 +211,11 @@ public function run($request)
}
Security::setCurrentUser($this->anonymousMember);

if (php_sapi_name() == "cli") {
$this->lineBreak = "\n";
}

if ($request->getVar('reset')) {
$this->reset();
if ($input->getOption('reset')) {
$this->reset($output);
}

$fileCounts = $request->getVar('fileCounts');
$fileCounts = $input->getOption('fileCounts');
if ($fileCounts) {
$counts = explode(',', $fileCounts ?? '');
$this->fileCounts = array_map(function ($int) {
Expand All @@ -231,7 +225,7 @@ public function run($request)
$this->fileCounts = FTFileMakerTask::config()->get('fileCountByDepth');
}

$folderCounts = $request->getVar('folderCounts');
$folderCounts = $input->getOption('folderCounts');
if ($folderCounts) {
$counts = explode(',', $folderCounts ?? '');
$this->folderCounts = array_map(function ($int) {
Expand All @@ -241,26 +235,27 @@ public function run($request)
$this->folderCounts = FTFileMakerTask::config()->get('folderCountByDepth');
}

echo "Downloading fixtures" . $this->lineBreak;
$fixtureFilePaths = $this->downloadFixtureFiles();
$output->writeln('Downloading fixtures');
$fixtureFilePaths = $this->downloadFixtureFiles($output);

if (!FTFileMakerTask::config()->get('documentsOnly')) {
echo "Generate thumbnails" . $this->lineBreak;
$output->writeln('Generate thumbnails');
$this->generateThumbnails($fixtureFilePaths);
}

echo "Generate files" . $this->lineBreak;
$output->writeln('Generate files');
$this->generateFiles($fixtureFilePaths);

if (!FTFileMakerTask::config()->get('doPutProtectedFilesInPublicStore')) {
echo "Incorrectly putting protected files into public asset store on purpose" . $this->lineBreak;
$output->writeln('Incorrectly putting protected files into public asset store on purpose');
$this->putProtectedFilesInPublicAssetStore();
}
return Command::SUCCESS;
}

protected function reset()
protected function reset(PolyOutput $output)
{
echo "Resetting assets" . $this->lineBreak;
$output->writeln('Resetting assets');

DB::query('TRUNCATE "File"');
DB::query('TRUNCATE "File_Live"');
Expand All @@ -271,7 +266,7 @@ protected function reset()
}
}

protected function downloadFixtureFiles()
protected function downloadFixtureFiles(PolyOutput $output)
{
$client = new Client(['base_uri' => $this->fixtureFileBaseUrl]);

Expand All @@ -293,7 +288,7 @@ protected function downloadFixtureFiles()
$promises[$filename] = $client->getAsync($filename, [
'sink' => $path
]);
echo "Downloading $url" . $this->lineBreak;
$output->writeln("Downloading $url");
}
}

Expand Down Expand Up @@ -497,4 +492,28 @@ protected function watermarkImage(string $stampPath, string $targetPath): ?strin
return $targetPath;
}

public function getOptions(): array
{
return [
new InputOption(
'reset',
null,
InputOption::VALUE_NONE,
'Optionally truncate ALL files and folders in the database,'
. ' plus delete the entire `assets/` directory',
),
new InputOption(
'fileCounts',
null,
InputOption::VALUE_REQUIRED,
'Comma separated string'
),
new InputOption(
'folderCounts',
null,
InputOption::VALUE_REQUIRED,
'Comma separated string'
),
];
}
}
Loading