Skip to content

Commit

Permalink
[3.13] gh-126180: Remove getopt and optparse deprecation notices (GH-…
Browse files Browse the repository at this point in the history
…128191)

* Remove getopt and optparse deprecation notices
* Add new docs sections for command line app helper libraries
* Add guidance on choosing a CLI parsing library to the optparse docs
* Link to the new guidance from the argparse and getopt docs
* Reword intro in docs section for superseded stdlib modules
* Reframe the optparse->argparse guide as a migration guide
  rather than as an upgrade guide

---------
(cherry picked from commit 831b6de)

Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
  • Loading branch information
3 people authored Dec 23, 2024
1 parent 09d15aa commit 6f3c2c8
Show file tree
Hide file tree
Showing 12 changed files with 266 additions and 65 deletions.
36 changes: 23 additions & 13 deletions Doc/howto/argparse-optparse.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,14 @@
.. currentmodule:: argparse

.. _upgrading-optparse-code:
.. _migrating-optparse-code:

==========================
Upgrading optparse code
==========================
============================================
Migrating ``optparse`` code to ``argparse``
============================================

Originally, the :mod:`argparse` module had attempted to maintain compatibility
with :mod:`optparse`. However, :mod:`optparse` was difficult to extend
transparently, particularly with the changes required to support
``nargs=`` specifiers and better usage messages. When most everything in
:mod:`optparse` had either been copy-pasted over or monkey-patched, it no
longer seemed practical to try to maintain the backwards compatibility.

The :mod:`argparse` module improves on the :mod:`optparse`
module in a number of ways including:
The :mod:`argparse` module offers several higher level features not natively
provided by the :mod:`optparse` module, including:

* Handling positional arguments.
* Supporting subcommands.
Expand All @@ -23,7 +17,23 @@ module in a number of ways including:
* Producing more informative usage messages.
* Providing a much simpler interface for custom ``type`` and ``action``.

A partial upgrade path from :mod:`optparse` to :mod:`argparse`:
Originally, the :mod:`argparse` module attempted to maintain compatibility
with :mod:`optparse`. However, the fundamental design differences between
supporting declarative command line option processing (while leaving positional
argument processing to application code), and supporting both named options
and positional arguments in the declarative interface mean that the
API has diverged from that of ``optparse`` over time.

As described in :ref:`choosing-an-argument-parser`, applications that are
currently using :mod:`optparse` and are happy with the way it works can
just continue to use ``optparse``.

Application developers that are considering migrating should also review
the list of intrinsic behavioural differences described in that section
before deciding whether or not migration is desirable.

For applications that do choose to migrate from :mod:`optparse` to :mod:`argparse`,
the following suggestions should be helpful:

* Replace all :meth:`optparse.OptionParser.add_option` calls with
:meth:`ArgumentParser.add_argument` calls.
Expand Down
15 changes: 10 additions & 5 deletions Doc/howto/argparse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,16 @@ recommended command-line parsing module in the Python standard library.

.. note::

There are two other modules that fulfill the same task, namely
:mod:`getopt` (an equivalent for ``getopt()`` from the C
language) and the deprecated :mod:`optparse`.
Note also that :mod:`argparse` is based on :mod:`optparse`,
and therefore very similar in terms of usage.
The standard library includes two other libraries directly related
to command-line parameter processing: the lower level :mod:`optparse`
module (which may require more code to configure for a given application,
but also allows an application to request behaviors that ``argparse``
doesn't support), and the very low level :mod:`getopt` (which specifically
serves as an equivalent to the :c:func:`!getopt` family of functions
available to C programmers).
While neither of those modules is covered directly in this guide, many of
the core concepts in ``argparse`` first originated in ``optparse``, so
some aspects of this tutorial will also be relevant to ``optparse`` users.


Concepts
Expand Down
5 changes: 0 additions & 5 deletions Doc/library/allos.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,9 @@ but they are available on most other systems as well. Here's an overview:
os.rst
io.rst
time.rst
argparse.rst
logging.rst
logging.config.rst
logging.handlers.rst
getpass.rst
curses.rst
curses.ascii.rst
curses.panel.rst
platform.rst
errno.rst
ctypes.rst
12 changes: 12 additions & 0 deletions Doc/library/argparse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,18 @@

**Source code:** :source:`Lib/argparse.py`

.. note::

While :mod:`argparse` is the default recommended standard library module
for implementing basic command line applications, authors with more
exacting requirements for exactly how their command line applications
behave may find it doesn't provide the necessary level of control.
Refer to :ref:`choosing-an-argument-parser` for alternatives to
consider when ``argparse`` doesn't support behaviors that the application
requires (such as entirely disabling support for interspersed options and
positional arguments, or accepting option parameter values that start
with ``-`` even when they correspond to another defined option).

--------------

.. sidebar:: Tutorial
Expand Down
21 changes: 21 additions & 0 deletions Doc/library/cmdlinelibs.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
.. _cmdlinelibs:

********************************
Command Line Interface Libraries
********************************

The modules described in this chapter assist with implementing
command line and terminal interfaces for applications.

Here's an overview:

.. toctree::
:maxdepth: 1

argparse.rst
optparse.rst
getpass.rst
fileinput.rst
curses.rst
curses.ascii.rst
curses.panel.rst
1 change: 0 additions & 1 deletion Doc/library/filesys.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ in this chapter is:

pathlib.rst
os.path.rst
fileinput.rst
stat.rst
filecmp.rst
tempfile.rst
Expand Down
54 changes: 38 additions & 16 deletions Doc/library/getopt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,13 @@

**Source code:** :source:`Lib/getopt.py`

.. deprecated:: 3.13
The :mod:`getopt` module is :term:`soft deprecated` and will not be
developed further; development will continue with the :mod:`argparse`
module.

.. note::

The :mod:`getopt` module is a parser for command line options whose API is
designed to be familiar to users of the C :c:func:`!getopt` function. Users who
are unfamiliar with the C :c:func:`!getopt` function or who would like to write
less code and get better help and error messages should consider using the
:mod:`argparse` module instead.
This module is considered feature complete. A more declarative and
extensible alternative to this API is provided in the :mod:`optparse`
module. Further functional enhancements for command line parameter
processing are provided either as third party modules on PyPI,
or else as features in the :mod:`argparse` module.

--------------

Expand All @@ -28,6 +23,13 @@ the special meanings of arguments of the form '``-``' and '``--``'). Long
options similar to those supported by GNU software may be used as well via an
optional third argument.

Users who are unfamiliar with the Unix :c:func:`!getopt` function should consider
using the :mod:`argparse` module instead. Users who are familiar with the Unix
:c:func:`!getopt` function, but would like to get equivalent behavior while
writing less code and getting better help and error messages should consider
using the :mod:`optparse` module. See :ref:`choosing-an-argument-parser` for
additional details.

This module provides two functions and an
exception:

Expand Down Expand Up @@ -150,13 +152,27 @@ In a script, typical usage is something like this:
output = a
else:
assert False, "unhandled option"
# ...
process(args, output=output, verbose=verbose)

if __name__ == "__main__":
main()

Note that an equivalent command line interface could be produced with less code
and more informative help and error messages by using the :mod:`argparse` module:
and more informative help and error messages by using the :mod:`optparse` module:

.. testcode::

import optparse

if __name__ == '__main__':
parser = optparse.OptionParser()
parser.add_option('-o', '--output')
parser.add_option('-v', dest='verbose', action='store_true')
opts, args = parser.parse_args()
process(args, output=opts.output, verbose=opts.verbose)

A roughly equivalent command line interface for this case can also be
produced by using the :mod:`argparse` module:

.. testcode::

Expand All @@ -166,12 +182,18 @@ and more informative help and error messages by using the :mod:`argparse` module
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--output')
parser.add_argument('-v', dest='verbose', action='store_true')
parser.add_argument('rest', nargs='*')
args = parser.parse_args()
# ... do something with args.output ...
# ... do something with args.verbose ..
process(args.rest, output=args.output, verbose=args.verbose)

See :ref:`choosing-an-argument-parser` for details on how the ``argparse``
version of this code differs in behaviour from the ``optparse`` (and
``getopt``) version.

.. seealso::

Module :mod:`argparse`
Alternative command line option and argument parsing library.
Module :mod:`optparse`
Declarative command line option parsing.

Module :mod:`argparse`
More opinionated command line option and argument parsing library.
1 change: 1 addition & 0 deletions Doc/library/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ the `Python Package Index <https://pypi.org>`_.
fileformats.rst
crypto.rst
allos.rst
cmdlinelibs.rst
concurrency.rst
ipc.rst
netdata.rst
Expand Down
139 changes: 125 additions & 14 deletions Doc/library/optparse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,135 @@

.. module:: optparse
:synopsis: Command-line option parsing library.
:deprecated:

.. moduleauthor:: Greg Ward <gward@python.net>
.. sectionauthor:: Greg Ward <gward@python.net>

**Source code:** :source:`Lib/optparse.py`

.. deprecated:: 3.2
The :mod:`optparse` module is :term:`soft deprecated` and will not be
developed further; development will continue with the :mod:`argparse`
module.

--------------

.. _choosing-an-argument-parser:

Choosing an argument parsing library
------------------------------------

The standard library includes three argument parsing libraries:

* :mod:`getopt`: a module that closely mirrors the procedural C ``getopt`` API.
Included in the standard library since before the initial Python 1.0 release.
* :mod:`optparse`: a declarative replacement for ``getopt`` that
provides equivalent functionality without requiring each application
to implement its own procedural option parsing logic. Included
in the standard library since the Python 2.3 release.
* :mod:`argparse`: a more opinionated alternative to ``optparse`` that
provides more functionality by default, at the expense of reduced application
flexibility in controlling exactly how arguments are processed. Included in
the standard library since the Python 2.7 and Python 3.2 releases.

In the absence of more specific argument parsing design constraints, :mod:`argparse`
is the recommended choice for implementing command line applications, as it offers
the highest level of baseline functionality with the least application level code.

:mod:`getopt` is retained almost entirely for backwards compatibility reasons.
However, it also serves a niche use case as a tool for prototyping and testing
command line argument handling in ``getopt``-based C applications.

:mod:`optparse` should be considered as an alternative to :mod:`argparse` in the
following cases:

* an application is already using :mod:`optparse` and doesn't want to risk the
subtle behavioural changes that may arise when migrating to :mod:`argparse`
* the application requires additional control over the way options and
positional parameters are interleaved on the command line (including
the ability to disable the interleaving feature completely)
* the application requires additional control over the incremental parsing
of command line elements (while ``argparse`` does support this, the
exact way it works in practice is undesirable for some use cases)
* the application requires additional control over the handling of options
which accept parameter values that may start with ``-`` (such as delegated
options to be passed to invoked subprocesses)
* the application requires some other command line parameter processing
behavior which ``argparse`` does not support, but which can be implemented
in terms of the lower level interface offered by ``optparse``

These considerations also mean that :mod:`optparse` is likely to provide a
better foundation for library authors writing third party command line
argument processing libraries.

As a concrete example, consider the following two command line argument
parsing configurations, the first using ``optparse``, and the second
using ``argparse``:

.. testcode::

import optparse

if __name__ == '__main__':
parser = optparse.OptionParser()
parser.add_option('-o', '--output')
parser.add_option('-v', dest='verbose', action='store_true')
opts, args = parser.parse_args()
process(args, output=opts.output, verbose=opts.verbose)

.. testcode::

import argparse

if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--output')
parser.add_argument('-v', dest='verbose', action='store_true')
parser.add_argument('rest', nargs='*')
args = parser.parse_args()
process(args.rest, output=args.output, verbose=args.verbose)

The most obvious difference is that in the ``optparse`` version, the non-option
arguments are processed separately by the application after the option processing
is complete. In the ``argparse`` version, positional arguments are declared and
processed in the same way as the named options.

However, the ``argparse`` version will also handle some parameter combination
differently from the way the ``optparse`` version would handle them.
For example (amongst other differences):

* supplying ``-o -v`` gives ``output="-v"`` and ``verbose=False``
when using ``optparse``, but a usage error with ``argparse``
(complaining that no value has been supplied for ``-o/--output``,
since ``-v`` is interpreted as meaning the verbosity flag)
* similarly, supplying ``-o --`` gives ``output="--"`` and ``args=()``
when using ``optparse``, but a usage error with ``argparse``
(also complaining that no value has been supplied for ``-o/--output``,
since ``--`` is interpreted as terminating the option processing
and treating all remaining values as positional arguments)
* supplying ``-o=foo`` gives ``output="=foo"`` when using ``optparse``,
but gives ``output="foo"`` with ``argparse`` (since ``=`` is special
cased as an alternative separator for option parameter values)

Whether these differing behaviors in the ``argparse`` version are
considered desirable or a problem will depend on the specific command line
application use case.

.. seealso::

:pypi:`click` is a third party argument processing library (originally
based on ``optparse``), which allows command line applications to be
developed as a set of decorated command implementation functions.

Other third party libraries, such as :pypi:`typer` or :pypi:`msgspec-click`,
allow command line interfaces to be specified in ways that more effectively
integrate with static checking of Python type annotations.


Introduction
------------

:mod:`optparse` is a more convenient, flexible, and powerful library for parsing
command-line options than the old :mod:`getopt` module. :mod:`optparse` uses a
more declarative style of command-line parsing: you create an instance of
:class:`OptionParser`, populate it with options, and parse the command
line. :mod:`optparse` allows users to specify options in the conventional
command-line options than the minimalist :mod:`getopt` module.
:mod:`optparse` uses a more declarative style of command-line parsing:
you create an instance of :class:`OptionParser`,
populate it with options, and parse the command line.
:mod:`optparse` allows users to specify options in the conventional
GNU/POSIX syntax, and additionally generates usage and help messages for you.

Here's an example of using :mod:`optparse` in a simple script::
Expand Down Expand Up @@ -82,10 +192,11 @@ Background
----------

:mod:`optparse` was explicitly designed to encourage the creation of programs
with straightforward, conventional command-line interfaces. To that end, it
supports only the most common command-line syntax and semantics conventionally
used under Unix. If you are unfamiliar with these conventions, read this
section to acquaint yourself with them.
with straightforward command-line interfaces that follow the conventions
established by the :c:func:`!getopt` family of functions available to C developers.
To that end, it supports only the most common command-line syntax and semantics
conventionally used under Unix. If you are unfamiliar with these conventions,
reading this section will allow you to acquaint yourself with them.


.. _optparse-terminology:
Expand Down
Loading

0 comments on commit 6f3c2c8

Please sign in to comment.