Skip to content

Test against the latest Crawlee release and without optional extras #1150

Description

@vdusek

Crawlee 1.10.1 made extra_name a required keyword-only argument of crawlee._utils.try_import (apify/crawlee-python#2208), so import apify.scrapy failed for anyone who resolved that release:

TypeError: try_import() missing 1 required keyword-only argument: 'extra_name'

We excluded 1.10.1 in #1136 and Crawlee 1.10.2 restored compatibility (apify/crawlee-python#2247), but our CI never saw it. Tests run only against the Crawlee version pinned in uv.lock, and always with every extra installed (uv sync --all-extras). The SDK imports many crawlee._* internals, so a Crawlee patch release can break it the same way again.

Possible checks:

  1. Run the unit tests against the latest Crawlee release (or Crawlee master) on a schedule. Running the same job in Crawlee's CI before a release would catch it before users do.
  2. Run them against the lower bound, crawlee==1.8.0.
  3. Run a job without the scrapy extra: import apify and import apify.scrapy should succeed, and using a Scrapy symbol should raise the install hint.

✍️ Drafted by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions