Skip to content

Commit

Permalink
fix: destructure CrawlerRunOptions before passing them to `addReque…
Browse files Browse the repository at this point in the history
…sts` (#2803)

Allows users to actually use `await crawler.run([...], {
purgeRequestQueue: false });`, which is a hard(?) requirement for
stateful crawler use-cases.

Closes #2802
  • Loading branch information
barjin authored Jan 10, 2025
1 parent afbd864 commit 02a598c
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions packages/basic-crawler/src/internals/basic-crawler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@ export class BasicCrawler<Context extends CrawlingContext = BasicCrawlingContext
);
}

const purgeRequestQueue = options?.purgeRequestQueue ?? true;
const { purgeRequestQueue = true, ...addRequestsOptions } = options ?? {};

if (this.hasFinishedBefore) {
// When executing the run method for the second time explicitly,
Expand All @@ -896,7 +896,7 @@ export class BasicCrawler<Context extends CrawlingContext = BasicCrawlingContext
await purgeDefaultStorages({ onlyPurgeOnce: true });

if (requests) {
await this.addRequests(requests, options);
await this.addRequests(requests, addRequestsOptions);
}

await this._init();
Expand Down

0 comments on commit 02a598c

Please sign in to comment.