This sets the expected API that clients (the Application) will use:

- Creating a fetch-service is done in setup();
- Sessions are created with create_session(), which receives a managed
  instance because we'll need it to figure out the network gateway;
- Sessions are destroyed with teardown_session();
- Stopping the fetch-service is done with shutdown().

Add a set of utility functions so that FetchService.setup() will spawn a
fetch-service if necessary. These utility functions include a function to
get the service's (json) status, and a function to tell whether the service
is up.

Use the $SNAP_USER_COMMON location for the fetch-service snap as a base for
the 'config' and 'spool' subdirectories.

Session credentials are stored to be used (in the near future) when setting up
the http/https proxies. When closing the session, the report obtained from the
fetch-service is discarded (for now, until we figure out what to do with it).

This configuration happens when creating the session. Currently we:

- Install the local certificate (added to this repo)
- Configure pip, snapd and apt
- Refresh apt's package listings to ensure all installations are traceable
  by the fetch-service

This initial implementation *always* refreshes apt listings, even if the
project build won't use them (e.g. no stage- or build- packages). In the
future we'll be smarter about this. The implementation also only supports
LXD instances, just because we need to figure out the instance's gateway
address and only the LXD work for that has been done so far. In the future
we can support all instance types with a cleaner craft-providers API.

This variable points to the local-ca.crt self-signed certificate, effectively
telling requests to trust that certificate when negotiating https requests.

Recent versions of the fetch-service not longer come with a fixed, "built-in"
certificate. Instead, clients are expected to generate their own. To do this,
we use a location that is shared among all Craft applications - the key and
certificate are generated if not found, and are otherwise shared by all
instances of all applications.

Fixes #21

This lets cargo allow our self-signed certificate when making https requests
to its registry.

Fixes #20

Now that the fetch-service's bug is fixed we can revert our temporary
change

This is temporary - in the near future we'll use the in-memory report to
generate a 'proper' manifest, but for now this lets us at least have something
to inspect/validate.

The strict mode is not yet implemented fully, so until further notice we should
create permissive sessions.

The snap's current confinement does not let it access the user's home files,
so <home>/.local/... is inaccessible.

There seems to be a bug currently with creating consecutive sessions, so for
now let's always shut down the fetch-service after the managed run is done.

With an open_stream() the output shows up in a single, updating line in brief
mode. This output is useful because the call takes a while.

This implementation checks the presence of the fetch-service snap through the
hardcoded path (/snap/bin/fetch-service). If, in the future, we need a more
sophisticated approach we can copy craft-provider's logic for checking the
lxd snap through snapd's api.

Fixes #36

With this commit the fetch-service service is disabled by default; the way
to enable it is through a new "--use-fetch-service" command line parameter.

This new parameter is initiallly only available to the "pack" command, as
the cache-clearing that the service needs precludes the use of other
lifecycle commands, and the manifest that the application will create based
on the fetch-service's session report is only produced during packing.

Fixes #15

This commit redirects the output of the fetch-service to a file. Since we plan
to have the fetch-service outlive the application that spawned it, we need to
use bash to redirect the fetch-service's output to a file in a way that
persists after the application ends.

Fixes #51

Test is failing due to canonical/fetch-service#208;
once that's fixed we can revert this commit. Note that only one of the ports
is failing; the proxy port is still giving the expected error output.

Signed-off-by: Callahan Kovacs <callahankovacs@gmail.com>

chore: merge release 4.1.3 into main

This allows applications to override the inner run logic, with the end
goal of capturing application specific exceptions and raise
appropriate ones for Craft Application to handle.

Without the logic split, overriding run to capture exceptions is
virtually impossible as the run method holds a generic Exception
handler.

Signed-off-by: Sergio Schvezov <sergio.schvezov@canonical.com>

This new parameter to ProviderService.instance() defaults to False and, if
True, will cause the service to clean a pre-existing instance and create a
new one.

This is necessary because the fetch-service needs visibility on *all* assets
downloaded during a build, so pre-existing instances that might already have
existing items cannot be re-used.

Fixes #40

Set it to shutdown after 5 minutes of no live sessions.

* feat: update partitions docs & fix autobuild path

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Michael DuBelko <michael.dubelko@gmail.com>

* fix(docs): use Michael's suggested text

Co-authored-by: Michael DuBelko <michael.dubelko@gmail.com>

* Update docs/howto/partitions.rst

Co-authored-by: Sergio Schvezov <sergio.schvezov@canonical.com>

* docs: update other reference w/ Sergio's suggestion

Co-authored-by: Sergio Schvezov <sergio.schvezov@canonical.com>

---------

Co-authored-by: Michael DuBelko <michael.dubelko@gmail.com>
Co-authored-by: Sergio Schvezov <sergio.schvezov@canonical.com>

There are three new models:

- ProjectManifest, representing the metadata from the project itself and its
  built artifact;
- SessionArtifactManifest, representing the metadata for a single asset that
  was downloaded during the fetch-service session;
- CraftManifest, that ties the first two together and provides the full view
  of assets downloaded and generated by a build.

---------

Co-authored-by: Alex Lowe <alex.lowe@canonical.com>

Add a create_project_manifest() method to the FetchService that, when running
in managed mode, creates the yaml for the project metadata in a path that will
be later accessed by the host-side service to create the "full" craft manifest.

When running with the fetch-service integration enabled, the FetchService
service will generate a manifest file containing both metadata about the
packed artifact (from the project and the artifact itself) and about the
dependencies that were downloaded during the build (from the fetch-service
session report).

This brings 'clean' closer to parity with the other lifecycle commands, and is
useful for cases where we want to wipe one of the managed instances, but not
all of them.

Fixes #425

The value makes Go download directly from version-controlled repositories,
instead of module proxies like the default https://proxy.golang.org. It's
currently a requirement for the Go inspector.

'--use-fetch-service' now takes a mandatory argument defining the type of
session to use: "strict" or "permissive".

Fixes #487

By "mark" we mean:

- hide the command-line option from pack's help;
- if enabled, warn that this integration is still in development.

Fixes #488

Also warns about some pending deprecations.

This replaces the diataxis grid with a list-table.
https://docutils.sourceforge.io/docs/ref/rst/directives.html#list-table

Co-authored-by: Alex Lowe <alex.lowe@canonical.com>