Pylance shows non-public exports in completion suggestions

[binste]

While trying to make Altair a typed library, we were trying to hide deprecated functions from the code completions on the top-level package. The approach was discussed here #3709 (comment). However, Pylance still shows all symbols in the code completion even though some are no longer public exports, see vega/altair#2927 for the proposed code changes, expected and actual behaviour as well as screenshots.

In case these two threads do not provide enough context or are too complex to go through let me know and I can try to make a more minimal example.

Environment data

• Language Server version: 2023.3.10
• OS and version: macOS Ventura 13.1
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.11 via the 3.11-bullseye docker image

[debonte]

@binste, I'm unable to repro this issue using Pylance 2023.4.11. I confirmed that without your changes, selection is shown in the completion list. However, after applying your changes to __init__.py and adding a py.typed file, I no longer see it.

Here's what I did:

1. Create new folder
2. python -m venv .venv
3. .venv\Scripts\activate
4. pip install altair which gave me v4.2.2
5. Replaced the 4.2.2 __init__.py with the version in your branch
6. Added empty py.typed file
7. Opened folder in VS Code
8. Selected .venv
9. Created empty .py file
10. Added import altair as alt
11. Typed alt.sel and I see this:
    

Are you still seeing this? If so, can you help us repro it?

[binste]

Thank you @debonte for looking into it. I followed every step of your instructions and I still see selection showing up. I'm now on Pylance 2023.4.30 and VS Code 1.77.3.

Screen.Recording.2023-04-26.at.20.29.28.mov

Here's the location of the py.typed file I created in case that would be wrong:

As you can see in the video above, selection is greyed out and so Pylance seems to be aware that it is not accessed in __all__.

I also restarted VS Code and executed the Python: Restart Language Server command but to no effect. I'm no longer running this in a Docker container but instead directly on a Macbook. I will test this as well on a Windows machine in the coming days.

[debonte]

@binste, I'm still unable to repro this. I tried both Windows and Linux.

Are you sure that you have the right virtual environment selected in VS Code?

If so, can you please set "python.analysis.logLevel": "Trace", then repro the issue, and provide the resulting "Python Language Server" log from the Output pane?

[binste]

Trace_2022_05_18_MacOS.txt

Sorry that it took me a while to get back to this. Some context for the log: I opened the project again, updated all extensions, then reopened the window, verified that the interpreter is set to the correct one in the .venv folder, set the loglevel to trace in the workspace settings file, opened test.py, and imported altair and typed alt.selec and selection showed up.

[debonte]

Can you post the contents of the following file, assuming you haven't changed it since you collected the log above?

(40209) [FG] binding: /Users/stefanbinder/Library/CloudStorage/Dropbox/Programming/altair_playground/vscode_completion/.venv/lib/python3.10/site-packages/altair/__init__.py

[binste]

Sure, the file is still the same as when I collected the log:

__init__.py

# flake8: noqa
__version__ = "4.3.0.dev0"


# The content of __all__ is automatically written by
# tools/update_init_file.py. Do not modify directly.
__all__ = [
    "Aggregate",
    "AggregateOp",
    "AggregateTransform",
    "AggregatedFieldDef",
    "Align",
    "AllSortString",
    "Angle",
    "AngleDatum",
    "AngleValue",
    "AnyMark",
    "AnyMarkConfig",
    "AreaConfig",
    "ArgmaxDef",
    "ArgminDef",
    "AutoSizeParams",
    "AutosizeType",
    "Axis",
    "AxisConfig",
    "AxisOrient",
    "AxisResolveMap",
    "BarConfig",
    "BaseTitleNoValueRefs",
    "Baseline",
    "Bin",
    "BinExtent",
    "BinParams",
    "BinTransform",
    "BindCheckbox",
    "BindDirect",
    "BindInput",
    "BindRadioSelect",
    "BindRange",
    "Binding",
    "Blend",
    "BoxPlot",
    "BoxPlotConfig",
    "BoxPlotDef",
    "BrushConfig",
    "CalculateTransform",
    "Categorical",
    "Chart",
    "Color",
    "ColorDatum",
    "ColorDef",
    "ColorName",
    "ColorScheme",
    "ColorValue",
    "Column",
    "CompositeMark",
    "CompositeMarkDef",
    "CompositionConfig",
    "ConcatChart",
    "ConcatSpecGenericSpec",
    "ConditionalAxisColor",
    "ConditionalAxisLabelAlign",
    "ConditionalAxisLabelBaseline",
    "ConditionalAxisLabelFontStyle",
    "ConditionalAxisLabelFontWeight",
    "ConditionalAxisNumber",
    "ConditionalAxisNumberArray",
    "ConditionalAxisPropertyAlignnull",
    "ConditionalAxisPropertyColornull",
    "ConditionalAxisPropertyFontStylenull",
    "ConditionalAxisPropertyFontWeightnull",
    "ConditionalAxisPropertyTextBaselinenull",
    "ConditionalAxisPropertynumberArraynull",
    "ConditionalAxisPropertynumbernull",
    "ConditionalAxisPropertystringnull",
    "ConditionalAxisString",
    "ConditionalMarkPropFieldOrDatumDef",
    "ConditionalMarkPropFieldOrDatumDefTypeForShape",
    "ConditionalParameterMarkPropFieldOrDatumDef",
    "ConditionalParameterMarkPropFieldOrDatumDefTypeForShape",
    "ConditionalParameterStringFieldDef",
    "ConditionalParameterValueDefGradientstringnullExprRef",
    "ConditionalParameterValueDefTextExprRef",
    "ConditionalParameterValueDefnumber",
    "ConditionalParameterValueDefnumberArrayExprRef",
    "ConditionalParameterValueDefnumberExprRef",
    "ConditionalParameterValueDefstringExprRef",
    "ConditionalParameterValueDefstringnullExprRef",
    "ConditionalPredicateMarkPropFieldOrDatumDef",
    "ConditionalPredicateMarkPropFieldOrDatumDefTypeForShape",
    "ConditionalPredicateStringFieldDef",
    "ConditionalPredicateValueDefAlignnullExprRef",
    "ConditionalPredicateValueDefColornullExprRef",
    "ConditionalPredicateValueDefFontStylenullExprRef",
    "ConditionalPredicateValueDefFontWeightnullExprRef",
    "ConditionalPredicateValueDefGradientstringnullExprRef",
    "ConditionalPredicateValueDefTextBaselinenullExprRef",
    "ConditionalPredicateValueDefTextExprRef",
    "ConditionalPredicateValueDefnumber",
    "ConditionalPredicateValueDefnumberArrayExprRef",
    "ConditionalPredicateValueDefnumberArraynullExprRef",
    "ConditionalPredicateValueDefnumberExprRef",
    "ConditionalPredicateValueDefnumbernullExprRef",
    "ConditionalPredicateValueDefstringExprRef",
    "ConditionalPredicateValueDefstringnullExprRef",
    "ConditionalStringFieldDef",
    "ConditionalValueDefGradientstringnullExprRef",
    "ConditionalValueDefTextExprRef",
    "ConditionalValueDefnumber",
    "ConditionalValueDefnumberArrayExprRef",
    "ConditionalValueDefnumberExprRef",
    "ConditionalValueDefstringExprRef",
    "ConditionalValueDefstringnullExprRef",
    "Config",
    "CsvDataFormat",
    "Cursor",
    "Cyclical",
    "Data",
    "DataFormat",
    "DataSource",
    "Datasets",
    "DateTime",
    "DatumChannelMixin",
    "DatumDef",
    "Day",
    "DensityTransform",
    "DerivedStream",
    "Description",
    "DescriptionValue",
    "Detail",
    "Dict",
    "DictInlineDataset",
    "DictSelectionInit",
    "DictSelectionInitInterval",
    "Diverging",
    "DomainUnionWith",
    "DsvDataFormat",
    "Element",
    "Encoding",
    "EncodingSortField",
    "ErrorBand",
    "ErrorBandConfig",
    "ErrorBandDef",
    "ErrorBar",
    "ErrorBarConfig",
    "ErrorBarDef",
    "ErrorBarExtent",
    "EventStream",
    "EventType",
    "Expr",
    "ExprRef",
    "Facet",
    "FacetChart",
    "FacetEncodingFieldDef",
    "FacetFieldDef",
    "FacetMapping",
    "FacetSpec",
    "FacetedEncoding",
    "FacetedUnitSpec",
    "Field",
    "FieldChannelMixin",
    "FieldDefWithoutScale",
    "FieldEqualPredicate",
    "FieldGTEPredicate",
    "FieldGTPredicate",
    "FieldLTEPredicate",
    "FieldLTPredicate",
    "FieldName",
    "FieldOneOfPredicate",
    "FieldOrDatumDefWithConditionDatumDefGradientstringnull",
    "FieldOrDatumDefWithConditionDatumDefnumber",
    "FieldOrDatumDefWithConditionDatumDefnumberArray",
    "FieldOrDatumDefWithConditionDatumDefstringnull",
    "FieldOrDatumDefWithConditionMarkPropFieldDefGradientstringnull",
    "FieldOrDatumDefWithConditionMarkPropFieldDefTypeForShapestringnull",
    "FieldOrDatumDefWithConditionMarkPropFieldDefnumber",
    "FieldOrDatumDefWithConditionMarkPropFieldDefnumberArray",
    "FieldOrDatumDefWithConditionStringDatumDefText",
    "FieldOrDatumDefWithConditionStringFieldDefText",
    "FieldOrDatumDefWithConditionStringFieldDefstring",
    "FieldRange",
    "FieldRangePredicate",
    "FieldValidPredicate",
    "Fill",
    "FillDatum",
    "FillOpacity",
    "FillOpacityDatum",
    "FillOpacityValue",
    "FillValue",
    "FilterTransform",
    "Fit",
    "FlattenTransform",
    "FoldTransform",
    "FontStyle",
    "FontWeight",
    "Generator",
    "GenericUnitSpecEncodingAnyMark",
    "GeoJsonFeature",
    "GeoJsonFeatureCollection",
    "Gradient",
    "GradientStop",
    "GraticuleGenerator",
    "GraticuleParams",
    "HConcatChart",
    "HConcatSpecGenericSpec",
    "Header",
    "HeaderConfig",
    "HexColor",
    "Href",
    "HrefValue",
    "Impute",
    "ImputeMethod",
    "ImputeParams",
    "ImputeSequence",
    "ImputeTransform",
    "InlineData",
    "InlineDataset",
    "Interpolate",
    "IntervalSelectionConfig",
    "IntervalSelectionConfigWithoutType",
    "JoinAggregateFieldDef",
    "JoinAggregateTransform",
    "JsonDataFormat",
    "Key",
    "LabelOverlap",
    "LatLongDef",
    "LatLongFieldDef",
    "Latitude",
    "Latitude2",
    "Latitude2Datum",
    "Latitude2Value",
    "LatitudeDatum",
    "LayerChart",
    "LayerRepeatMapping",
    "LayerRepeatSpec",
    "LayerSpec",
    "LayoutAlign",
    "Legend",
    "LegendBinding",
    "LegendConfig",
    "LegendOrient",
    "LegendResolveMap",
    "LegendStreamBinding",
    "LineConfig",
    "LinearGradient",
    "LocalMultiTimeUnit",
    "LocalSingleTimeUnit",
    "Locale",
    "LoessTransform",
    "LogicalAndPredicate",
    "LogicalNotPredicate",
    "LogicalOrPredicate",
    "Longitude",
    "Longitude2",
    "Longitude2Datum",
    "Longitude2Value",
    "LongitudeDatum",
    "LookupData",
    "LookupSelection",
    "LookupTransform",
    "Mark",
    "MarkConfig",
    "MarkDef",
    "MarkPropDefGradientstringnull",
    "MarkPropDefnumber",
    "MarkPropDefnumberArray",
    "MarkPropDefstringnullTypeForShape",
    "MarkType",
    "MaxRowsError",
    "MergedStream",
    "Month",
    "MultiTimeUnit",
    "NamedData",
    "NonArgAggregateOp",
    "NonLayerRepeatSpec",
    "NonNormalizedSpec",
    "NumberLocale",
    "NumericArrayMarkPropDef",
    "NumericMarkPropDef",
    "OffsetDef",
    "Opacity",
    "OpacityDatum",
    "OpacityValue",
    "Order",
    "OrderFieldDef",
    "OrderValue",
    "OrderValueDef",
    "Orient",
    "Orientation",
    "OverlayMarkDef",
    "Padding",
    "Parameter",
    "ParameterExpression",
    "ParameterExtent",
    "ParameterName",
    "ParameterPredicate",
    "Parse",
    "ParseValue",
    "PivotTransform",
    "PointSelectionConfig",
    "PointSelectionConfigWithoutType",
    "PolarDef",
    "Position2Def",
    "PositionDatumDef",
    "PositionDatumDefBase",
    "PositionDef",
    "PositionFieldDef",
    "PositionFieldDefBase",
    "PositionValueDef",
    "Predicate",
    "PredicateComposition",
    "PrimitiveValue",
    "Projection",
    "ProjectionConfig",
    "ProjectionType",
    "QuantileTransform",
    "RadialGradient",
    "Radius",
    "Radius2",
    "Radius2Datum",
    "Radius2Value",
    "RadiusDatum",
    "RadiusValue",
    "RangeConfig",
    "RangeEnum",
    "RangeRaw",
    "RangeRawArray",
    "RangeScheme",
    "RectConfig",
    "RegressionTransform",
    "RelativeBandSize",
    "RepeatChart",
    "RepeatMapping",
    "RepeatRef",
    "RepeatSpec",
    "Resolve",
    "ResolveMode",
    "Root",
    "Row",
    "RowColLayoutAlign",
    "RowColboolean",
    "RowColnumber",
    "RowColumnEncodingFieldDef",
    "SCHEMA_URL",
    "SCHEMA_VERSION",
    "SampleTransform",
    "Scale",
    "ScaleBinParams",
    "ScaleBins",
    "ScaleConfig",
    "ScaleDatumDef",
    "ScaleFieldDef",
    "ScaleInterpolateEnum",
    "ScaleInterpolateParams",
    "ScaleResolveMap",
    "ScaleType",
    "SchemaBase",
    "SchemeParams",
    "SecondaryFieldDef",
    "SelectionConfig",
    "SelectionExpression",
    "SelectionInit",
    "SelectionInitInterval",
    "SelectionInitIntervalMapping",
    "SelectionInitMapping",
    "SelectionParameter",
    "SelectionPredicateComposition",
    "SelectionResolution",
    "SelectionType",
    "SequenceGenerator",
    "SequenceParams",
    "SequentialMultiHue",
    "SequentialSingleHue",
    "Shape",
    "ShapeDatum",
    "ShapeDef",
    "ShapeValue",
    "SharedEncoding",
    "SingleDefUnitChannel",
    "SingleTimeUnit",
    "Size",
    "SizeDatum",
    "SizeValue",
    "Sort",
    "SortArray",
    "SortByChannel",
    "SortByChannelDesc",
    "SortByEncoding",
    "SortField",
    "SortOrder",
    "Spec",
    "SphereGenerator",
    "StackOffset",
    "StackTransform",
    "StandardType",
    "Step",
    "StepFor",
    "Stream",
    "StringFieldDef",
    "StringFieldDefWithCondition",
    "StringValueDefWithCondition",
    "Stroke",
    "StrokeCap",
    "StrokeDash",
    "StrokeDashDatum",
    "StrokeDashValue",
    "StrokeDatum",
    "StrokeJoin",
    "StrokeOpacity",
    "StrokeOpacityDatum",
    "StrokeOpacityValue",
    "StrokeValue",
    "StrokeWidth",
    "StrokeWidthDatum",
    "StrokeWidthValue",
    "StyleConfigIndex",
    "SymbolShape",
    "TOPLEVEL_ONLY_KEYS",
    "Text",
    "TextBaseline",
    "TextDatum",
    "TextDef",
    "TextDirection",
    "TextValue",
    "Theta",
    "Theta2",
    "Theta2Datum",
    "Theta2Value",
    "ThetaDatum",
    "ThetaValue",
    "TickConfig",
    "TickCount",
    "TimeInterval",
    "TimeIntervalStep",
    "TimeLocale",
    "TimeUnit",
    "TimeUnitParams",
    "TimeUnitTransform",
    "Title",
    "TitleAnchor",
    "TitleConfig",
    "TitleFrame",
    "TitleOrient",
    "TitleParams",
    "Tooltip",
    "TooltipContent",
    "TooltipValue",
    "TopLevelConcatSpec",
    "TopLevelFacetSpec",
    "TopLevelHConcatSpec",
    "TopLevelLayerSpec",
    "TopLevelMixin",
    "TopLevelParameter",
    "TopLevelRepeatSpec",
    "TopLevelSelectionParameter",
    "TopLevelSpec",
    "TopLevelUnitSpec",
    "TopLevelVConcatSpec",
    "TopoDataFormat",
    "Transform",
    "Type",
    "TypeForShape",
    "TypedFieldDef",
    "URI",
    "Undefined",
    "UnitSpec",
    "UnitSpecWithFrame",
    "Url",
    "UrlData",
    "UrlValue",
    "UtcMultiTimeUnit",
    "UtcSingleTimeUnit",
    "VConcatChart",
    "VConcatSpecGenericSpec",
    "VEGAEMBED_VERSION",
    "VEGALITE_VERSION",
    "VEGA_VERSION",
    "ValueChannelMixin",
    "ValueDefWithConditionMarkPropFieldOrDatumDefGradientstringnull",
    "ValueDefWithConditionMarkPropFieldOrDatumDefTypeForShapestringnull",
    "ValueDefWithConditionMarkPropFieldOrDatumDefnumber",
    "ValueDefWithConditionMarkPropFieldOrDatumDefnumberArray",
    "ValueDefWithConditionMarkPropFieldOrDatumDefstringnull",
    "ValueDefWithConditionStringFieldDefText",
    "ValueDefnumber",
    "ValueDefnumberwidthheightExprRef",
    "VariableParameter",
    "Vector10string",
    "Vector12string",
    "Vector2DateTime",
    "Vector2Vector2number",
    "Vector2boolean",
    "Vector2number",
    "Vector2string",
    "Vector3number",
    "Vector7string",
    "VegaLite",
    "VegaLiteSchema",
    "ViewBackground",
    "ViewConfig",
    "WindowEventType",
    "WindowFieldDef",
    "WindowOnlyOp",
    "WindowTransform",
    "X",
    "X2",
    "X2Datum",
    "X2Value",
    "XDatum",
    "XError",
    "XError2",
    "XError2Value",
    "XErrorValue",
    "XOffset",
    "XOffsetDatum",
    "XOffsetValue",
    "XValue",
    "Y",
    "Y2",
    "Y2Datum",
    "Y2Value",
    "YDatum",
    "YError",
    "YError2",
    "YError2Value",
    "YErrorValue",
    "YOffset",
    "YOffsetDatum",
    "YOffsetValue",
    "YValue",
    "api",
    "binding",
    "binding_checkbox",
    "binding_radio",
    "binding_range",
    "binding_select",
    "channels",
    "check_fields_and_encodings",
    "concat",
    "condition",
    "core",
    "curry",
    "data",
    "data_transformers",
    "datasets",
    "datum",
    "default_data_transformer",
    "display",
    "expr",
    "graticule",
    "hconcat",
    "layer",
    "limit_rows",
    "load_ipython_extension",
    "load_schema",
    "mixins",
    "overload",
    "param",
    "parse_shorthand",
    "pipe",
    "renderers",
    "repeat",
    "sample",
    "schema",
    "selection_interval",
    "selection_point",
    "sequence",
    "sphere",
    "theme",
    "themes",
    "to_csv",
    "to_json",
    "to_values",
    "topo_feature",
    "utils",
    "v5",
    "value",
    "vconcat",
    "vegalite",
    "with_property_setters",
]


def __dir__():
    return __all__


from .vegalite import (
    Aggregate,
    AggregateOp,
    AggregateTransform,
    AggregatedFieldDef,
    Align,
    AllSortString,
    Angle,
    AngleDatum,
    AngleValue,
    AnyMark,
    AnyMarkConfig,
    AreaConfig,
    ArgmaxDef,
    ArgminDef,
    AutoSizeParams,
    AutosizeType,
    Axis,
    AxisConfig,
    AxisOrient,
    AxisResolveMap,
    BarConfig,
    BaseTitleNoValueRefs,
    Baseline,
    Bin,
    BinExtent,
    BinParams,
    BinTransform,
    BindCheckbox,
    BindDirect,
    BindInput,
    BindRadioSelect,
    BindRange,
    Binding,
    Blend,
    BoxPlot,
    BoxPlotConfig,
    BoxPlotDef,
    BrushConfig,
    CalculateTransform,
    Categorical,
    Chart,
    Color,
    ColorDatum,
    ColorDef,
    ColorName,
    ColorScheme,
    ColorValue,
    Column,
    CompositeMark,
    CompositeMarkDef,
    CompositionConfig,
    ConcatChart,
    ConcatSpecGenericSpec,
    ConditionalAxisColor,
    ConditionalAxisLabelAlign,
    ConditionalAxisLabelBaseline,
    ConditionalAxisLabelFontStyle,
    ConditionalAxisLabelFontWeight,
    ConditionalAxisNumber,
    ConditionalAxisNumberArray,
    ConditionalAxisPropertyAlignnull,
    ConditionalAxisPropertyColornull,
    ConditionalAxisPropertyFontStylenull,
    ConditionalAxisPropertyFontWeightnull,
    ConditionalAxisPropertyTextBaselinenull,
    ConditionalAxisPropertynumberArraynull,
    ConditionalAxisPropertynumbernull,
    ConditionalAxisPropertystringnull,
    ConditionalAxisString,
    ConditionalMarkPropFieldOrDatumDef,
    ConditionalMarkPropFieldOrDatumDefTypeForShape,
    ConditionalParameterMarkPropFieldOrDatumDef,
    ConditionalParameterMarkPropFieldOrDatumDefTypeForShape,
    ConditionalParameterStringFieldDef,
    ConditionalParameterValueDefGradientstringnullExprRef,
    ConditionalParameterValueDefTextExprRef,
    ConditionalParameterValueDefnumber,
    ConditionalParameterValueDefnumberArrayExprRef,
    ConditionalParameterValueDefnumberExprRef,
    ConditionalParameterValueDefstringExprRef,
    ConditionalParameterValueDefstringnullExprRef,
    ConditionalPredicateMarkPropFieldOrDatumDef,
    ConditionalPredicateMarkPropFieldOrDatumDefTypeForShape,
    ConditionalPredicateStringFieldDef,
    ConditionalPredicateValueDefAlignnullExprRef,
    ConditionalPredicateValueDefColornullExprRef,
    ConditionalPredicateValueDefFontStylenullExprRef,
    ConditionalPredicateValueDefFontWeightnullExprRef,
    ConditionalPredicateValueDefGradientstringnullExprRef,
    ConditionalPredicateValueDefTextBaselinenullExprRef,
    ConditionalPredicateValueDefTextExprRef,
    ConditionalPredicateValueDefnumber,
    ConditionalPredicateValueDefnumberArrayExprRef,
    ConditionalPredicateValueDefnumberArraynullExprRef,
    ConditionalPredicateValueDefnumberExprRef,
    ConditionalPredicateValueDefnumbernullExprRef,
    ConditionalPredicateValueDefstringExprRef,
    ConditionalPredicateValueDefstringnullExprRef,
    ConditionalStringFieldDef,
    ConditionalValueDefGradientstringnullExprRef,
    ConditionalValueDefTextExprRef,
    ConditionalValueDefnumber,
    ConditionalValueDefnumberArrayExprRef,
    ConditionalValueDefnumberExprRef,
    ConditionalValueDefstringExprRef,
    ConditionalValueDefstringnullExprRef,
    Config,
    CsvDataFormat,
    Cursor,
    Cyclical,
    Data,
    DataFormat,
    DataSource,
    Datasets,
    DateTime,
    DatumChannelMixin,
    DatumDef,
    Day,
    DensityTransform,
    DerivedStream,
    Description,
    DescriptionValue,
    Detail,
    Dict,
    DictInlineDataset,
    DictSelectionInit,
    DictSelectionInitInterval,
    Diverging,
    DomainUnionWith,
    DsvDataFormat,
    Element,
    Encoding,
    EncodingSortField,
    ErrorBand,
    ErrorBandConfig,
    ErrorBandDef,
    ErrorBar,
    ErrorBarConfig,
    ErrorBarDef,
    ErrorBarExtent,
    EventStream,
    EventType,
    Expr,
    ExprRef,
    Facet,
    FacetChart,
    FacetEncodingFieldDef,
    FacetFieldDef,
    FacetMapping,
    FacetSpec,
    FacetedEncoding,
    FacetedUnitSpec,
    Field,
    FieldChannelMixin,
    FieldDefWithoutScale,
    FieldEqualPredicate,
    FieldGTEPredicate,
    FieldGTPredicate,
    FieldLTEPredicate,
    FieldLTPredicate,
    FieldName,
    FieldOneOfPredicate,
    FieldOrDatumDefWithConditionDatumDefGradientstringnull,
    FieldOrDatumDefWithConditionDatumDefnumber,
    FieldOrDatumDefWithConditionDatumDefnumberArray,
    FieldOrDatumDefWithConditionDatumDefstringnull,
    FieldOrDatumDefWithConditionMarkPropFieldDefGradientstringnull,
    FieldOrDatumDefWithConditionMarkPropFieldDefTypeForShapestringnull,
    FieldOrDatumDefWithConditionMarkPropFieldDefnumber,
    FieldOrDatumDefWithConditionMarkPropFieldDefnumberArray,
    FieldOrDatumDefWithConditionStringDatumDefText,
    FieldOrDatumDefWithConditionStringFieldDefText,
    FieldOrDatumDefWithConditionStringFieldDefstring,
    FieldRange,
    FieldRangePredicate,
    FieldValidPredicate,
    Fill,
    FillDatum,
    FillOpacity,
    FillOpacityDatum,
    FillOpacityValue,
    FillValue,
    FilterTransform,
    Fit,
    FlattenTransform,
    FoldTransform,
    FontStyle,
    FontWeight,
    Generator,
    GenericUnitSpecEncodingAnyMark,
    GeoJsonFeature,
    GeoJsonFeatureCollection,
    Gradient,
    GradientStop,
    GraticuleGenerator,
    GraticuleParams,
    HConcatChart,
    HConcatSpecGenericSpec,
    Header,
    HeaderConfig,
    HexColor,
    Href,
    HrefValue,
    Impute,
    ImputeMethod,
    ImputeParams,
    ImputeSequence,
    ImputeTransform,
    InlineData,
    InlineDataset,
    Interpolate,
    IntervalSelectionConfig,
    IntervalSelectionConfigWithoutType,
    JoinAggregateFieldDef,
    JoinAggregateTransform,
    JsonDataFormat,
    Key,
    LabelOverlap,
    LatLongDef,
    LatLongFieldDef,
    Latitude,
    Latitude2,
    Latitude2Datum,
    Latitude2Value,
    LatitudeDatum,
    LayerChart,
    LayerRepeatMapping,
    LayerRepeatSpec,
    LayerSpec,
    LayoutAlign,
    Legend,
    LegendBinding,
    LegendConfig,
    LegendOrient,
    LegendResolveMap,
    LegendStreamBinding,
    LineConfig,
    LinearGradient,
    LocalMultiTimeUnit,
    LocalSingleTimeUnit,
    Locale,
    LoessTransform,
    LogicalAndPredicate,
    LogicalNotPredicate,
    LogicalOrPredicate,
    Longitude,
    Longitude2,
    Longitude2Datum,
    Longitude2Value,
    LongitudeDatum,
    LookupData,
    LookupSelection,
    LookupTransform,
    Mark,
    MarkConfig,
    MarkDef,
    MarkPropDefGradientstringnull,
    MarkPropDefnumber,
    MarkPropDefnumberArray,
    MarkPropDefstringnullTypeForShape,
    MarkType,
    MaxRowsError,
    MergedStream,
    Month,
    MultiTimeUnit,
    NamedData,
    NonArgAggregateOp,
    NonLayerRepeatSpec,
    NonNormalizedSpec,
    NumberLocale,
    NumericArrayMarkPropDef,
    NumericMarkPropDef,
    OffsetDef,
    Opacity,
    OpacityDatum,
    OpacityValue,
    Order,
    OrderFieldDef,
    OrderValue,
    OrderValueDef,
    Orient,
    Orientation,
    OverlayMarkDef,
    Padding,
    Parameter,
    ParameterExpression,
    ParameterExtent,
    ParameterName,
    ParameterPredicate,
    Parse,
    ParseValue,
    PivotTransform,
    PointSelectionConfig,
    PointSelectionConfigWithoutType,
    PolarDef,
    Position2Def,
    PositionDatumDef,
    PositionDatumDefBase,
    PositionDef,
    PositionFieldDef,
    PositionFieldDefBase,
    PositionValueDef,
    Predicate,
    PredicateComposition,
    PrimitiveValue,
    Projection,
    ProjectionConfig,
    ProjectionType,
    QuantileTransform,
    RadialGradient,
    Radius,
    Radius2,
    Radius2Datum,
    Radius2Value,
    RadiusDatum,
    RadiusValue,
    RangeConfig,
    RangeEnum,
    RangeRaw,
    RangeRawArray,
    RangeScheme,
    RectConfig,
    RegressionTransform,
    RelativeBandSize,
    RepeatChart,
    RepeatMapping,
    RepeatRef,
    RepeatSpec,
    Resolve,
    ResolveMode,
    Root,
    Row,
    RowColLayoutAlign,
    RowColboolean,
    RowColnumber,
    RowColumnEncodingFieldDef,
    SCHEMA_URL,
    SCHEMA_VERSION,
    SampleTransform,
    Scale,
    ScaleBinParams,
    ScaleBins,
    ScaleConfig,
    ScaleDatumDef,
    ScaleFieldDef,
    ScaleInterpolateEnum,
    ScaleInterpolateParams,
    ScaleResolveMap,
    ScaleType,
    SchemaBase,
    SchemeParams,
    SecondaryFieldDef,
    SelectionConfig,
    SelectionExpression,
    SelectionInit,
    SelectionInitInterval,
    SelectionInitIntervalMapping,
    SelectionInitMapping,
    SelectionParameter,
    SelectionPredicateComposition,
    SelectionResolution,
    SelectionType,
    SequenceGenerator,
    SequenceParams,
    SequentialMultiHue,
    SequentialSingleHue,
    Shape,
    ShapeDatum,
    ShapeDef,
    ShapeValue,
    SharedEncoding,
    SingleDefUnitChannel,
    SingleTimeUnit,
    Size,
    SizeDatum,
    SizeValue,
    Sort,
    SortArray,
    SortByChannel,
    SortByChannelDesc,
    SortByEncoding,
    SortField,
    SortOrder,
    Spec,
    SphereGenerator,
    StackOffset,
    StackTransform,
    StandardType,
    Step,
    StepFor,
    Stream,
    StringFieldDef,
    StringFieldDefWithCondition,
    StringValueDefWithCondition,
    Stroke,
    StrokeCap,
    StrokeDash,
    StrokeDashDatum,
    StrokeDashValue,
    StrokeDatum,
    StrokeJoin,
    StrokeOpacity,
    StrokeOpacityDatum,
    StrokeOpacityValue,
    StrokeValue,
    StrokeWidth,
    StrokeWidthDatum,
    StrokeWidthValue,
    StyleConfigIndex,
    SymbolShape,
    TOPLEVEL_ONLY_KEYS,
    Text,
    TextBaseline,
    TextDatum,
    TextDef,
    TextDirection,
    TextValue,
    Theta,
    Theta2,
    Theta2Datum,
    Theta2Value,
    ThetaDatum,
    ThetaValue,
    TickConfig,
    TickCount,
    TimeInterval,
    TimeIntervalStep,
    TimeLocale,
    TimeUnit,
    TimeUnitParams,
    TimeUnitTransform,
    Title,
    TitleAnchor,
    TitleConfig,
    TitleFrame,
    TitleOrient,
    TitleParams,
    Tooltip,
    TooltipContent,
    TooltipValue,
    TopLevelConcatSpec,
    TopLevelFacetSpec,
    TopLevelHConcatSpec,
    TopLevelLayerSpec,
    TopLevelMixin,
    TopLevelParameter,
    TopLevelRepeatSpec,
    TopLevelSelectionParameter,
    TopLevelSpec,
    TopLevelUnitSpec,
    TopLevelVConcatSpec,
    TopoDataFormat,
    Transform,
    Type,
    TypeForShape,
    TypeVar,
    TypedFieldDef,
    URI,
    Undefined,
    UnitSpec,
    UnitSpecWithFrame,
    Url,
    UrlData,
    UrlValue,
    UtcMultiTimeUnit,
    UtcSingleTimeUnit,
    VConcatChart,
    VConcatSpecGenericSpec,
    VEGAEMBED_VERSION,
    VEGALITE_VERSION,
    VEGA_VERSION,
    ValueChannelMixin,
    ValueDefWithConditionMarkPropFieldOrDatumDefGradientstringnull,
    ValueDefWithConditionMarkPropFieldOrDatumDefTypeForShapestringnull,
    ValueDefWithConditionMarkPropFieldOrDatumDefnumber,
    ValueDefWithConditionMarkPropFieldOrDatumDefnumberArray,
    ValueDefWithConditionMarkPropFieldOrDatumDefstringnull,
    ValueDefWithConditionStringFieldDefText,
    ValueDefnumber,
    ValueDefnumberwidthheightExprRef,
    VariableParameter,
    Vector10string,
    Vector12string,
    Vector2DateTime,
    Vector2Vector2number,
    Vector2boolean,
    Vector2number,
    Vector2string,
    Vector3number,
    Vector7string,
    VegaLite,
    VegaLiteSchema,
    ViewBackground,
    ViewConfig,
    WindowEventType,
    WindowFieldDef,
    WindowOnlyOp,
    WindowTransform,
    X,
    X2,
    X2Datum,
    X2Value,
    XDatum,
    XError,
    XError2,
    XError2Value,
    XErrorValue,
    XOffset,
    XOffsetDatum,
    XOffsetValue,
    XValue,
    Y,
    Y2,
    Y2Datum,
    Y2Value,
    YDatum,
    YError,
    YError2,
    YError2Value,
    YErrorValue,
    YOffset,
    YOffsetDatum,
    YOffsetValue,
    YValue,
    __builtins__,
    __cached__,
    __doc__,
    __file__,
    __loader__,
    __name__,
    __package__,
    __path__,
    __spec__,
    api,
    binding,
    binding_checkbox,
    binding_radio,
    binding_range,
    binding_select,
    channels,
    check_fields_and_encodings,
    concat,
    condition,
    core,
    curry,
    data,
    data_transformers,
    datum,
    default_data_transformer,
    display,
    expr,
    graticule,
    hashlib,
    hconcat,
    io,
    itertools,
    json,
    jsonschema,
    layer,
    limit_rows,
    list_datasets,
    load_dataset,
    load_schema,
    mixins,
    overload,
    param,
    parse_shorthand,
    pd,
    pipe,
    pkgutil,
    renderers,
    repeat,
    sample,
    schema,
    selection,
    selection_interval,
    selection_multi,
    selection_point,
    selection_single,
    sequence,
    sphere,
    theme,
    themes,
    to_csv,
    to_json,
    to_values,
    topo_feature,
    utils,
    v5,
    value,
    vconcat,
    warnings,
    with_property_setters,
)


def load_ipython_extension(ipython):
    from ._magics import vegalite

    ipython.register_magic_function(vegalite, "cell")

[debonte]

@erictraut, this issue is related to interim files. Can you give me the executive summary of what their purpose is?

Interim files always have _isThirdPartyPyTypedPresent set to false, which causes the binder to not call setPrivatePyTypedImport for symbols that aren't in __all__. When the bug repros we're getting the module symbol table from the interim SourceFile for the altair __init__.py.

[erictraut]

The concept of an "interim file" (initially misspelled as "intrim file") was introduced via a sync with pylance on Jan 4, 2023. I don't know why it was added. There are no comments in the program.ts module that explain the concept. Since the change came in as part of a sync from pylance, I can't tell who initially wrote this code. The intent behind the change is obscured because it's combined with a bunch of other changes, and there are no detailed commit comments. I'm guessing that Heejae made this change based on the coding style and the wording in the comments.

[rchiodo]

I believe interim files were added for the pytest support. They were used to allow declarations for 3rd party libraries to be referenced when finding fixtures.

https://github.com/microsoft/pyrx/blame/d17388c14ab6393a6d79702eab39d0f260978ad2/packages/pylance-internal/src/pytest/fixtureFinder.ts#L428

[debonte]

Looks like it evolved out of the "shadowed file" concept in https://github.com/microsoft/pyrx/pull/3016/files#diff-c31e28784be62cd19c158d342a0ad07bd73858853a14f1f78b2931045cf1e8f5.

Shadowed files also always had _isThirdPartyPyTypedPresent set to false. Not sure if that was safe for some reason or already a source of bugs.

[debonte]

@erictraut, this issue is related to a Teams discussion that you recently had with HeeJae.

As he mentions there, we have "features that open library files through some implicit relation rather than explicit import statements." The "interim file" mechanism is used to add these files to the program. But currently it doesn't know what values the new SourceFile should have for isThirdPartyPyTypedPresent and isThirdPartyImport, so they are always false.

In this particular issue, the feature that causes _createInterimFileInfo to be called is module name completions. The interim SourceFile is created as you type the "altair" portion of import altair as alt. This repros in Pyright also.

HeeJae's proposal in the Teams chat above was to modify the return value of ImportResolver._getModuleNameForImport to include these flag values. Possibly by reusing the pyTypedInfo logic in _resolveAbsoluteImport to calculate isThirdPartyPyTypedPresent.

Does this seem like a reasonable approach?

[erictraut]

I'd need to review a bunch of code and page this all back into my head to answer your question. I probably won't have time to do that in the next week, so don't wait for me. This sounds like a reasonable approach — worth exploring.

[debonte]

Repro steps:

1. Create new folder
2. python -m venv .venv
3. .venv\Scripts\activate (or equivalent)
4. pip install altair==4.2.2
5. Replace .venv\lib\site-packages\altair\__init__.py with the version here
6. Add empty .venv\lib\site-packages\altair\py.typed file
7. Open folder in VS Code
8. Create empty .py file
9. Select .venv
10. Type in import altair as alt -- You must type it rather than pasting! Well, at least the altair module name. I've found that I need to type it at a fairly good clip (normal typing pace I guess). If I type it too slowly the issue doesn't repro. I believe this is because _removeUnneededFiles kicks in and deletes the interim file.
11. Immediately type the following on the next line: alt.selection

Expected: The only suggestions starting with lowercase selection should be selection_interval and selection_point, since they are the only ones in altair's __all__ collection.

Actual:

selection, selection_multi, and selection_single are also included:

[debonte]

Depends on https://github.com/microsoft/pyrx/issues/4270