SNOW-1633757: Error upon deploying Streamlit App ( ProgrammingError: 253006: 253006: File doesn't exist: ['pages/*.py'] )

[ankitsr92]

SnowCLI version

2.7

Python version

Python 3.9.10

Platform

Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.31

What happened

As per the documentation, snow streamlit deploy --replace will deploy the application as per configuration in snowflake.yml
and also upload files from pages/ if exists
https://docs.snowflake.com/en/developer-guide/snowflake-cli-v2/streamlit-apps/manage-apps/deploy-app#how-to-deploy-a-streamlit-app

I am trying to deploy a streamlit app, which isn't a multi page app hence the pages/ directory is empty ( No *.py files present )

As per the documentation, snow streamlit deploy --replace should deploy the app but its failing with below error,

ProgrammingError: 253006: 253006: File doesn't exist: ['pages/*.py']

Console output

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/commands/snow_typer.py:101 in           │
│ command_callable_decorator                                                                       │
│                                                                                                  │
│    98 │   │   │   │   execution = ExecutionMetadata()                                            │
│    99 │   │   │   │   self.pre_execute(execution)                                                │
│   100 │   │   │   │   try:                                                                       │
│ ❱ 101 │   │   │   │   │   result = command_callable(*args, **kw)                                 │
│   102 │   │   │   │   │   self.process_result(result)                                            │
│   103 │   │   │   │   │   execution.complete(ExecutionStatus.SUCCESS)                            │
│   104 │   │   │   │   except Exception as err:                                                   │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/commands/decorators.py:157 in wrapper   │
│                                                                                                  │
│   154 │   def wrapper(**options):                                                                │
│   155 │   │   if execute_before_command_using_new_options:                                       │
│   156 │   │   │   execute_before_command_using_new_options(**options)                            │
│ ❱ 157 │   │   return func(**options)                                                             │
│   158 │                                                                                          │
│   159 │   wrapper.__signature__ = _extend_signature_with_additional_options(func, additional_o   │
│   160 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/commands/decorators.py:157 in wrapper   │
│                                                                                                  │
│   154 │   def wrapper(**options):                                                                │
│   155 │   │   if execute_before_command_using_new_options:                                       │
│   156 │   │   │   execute_before_command_using_new_options(**options)                            │
│ ❱ 157 │   │   return func(**options)                                                             │
│   158 │                                                                                          │
│   159 │   wrapper.__signature__ = _extend_signature_with_additional_options(func, additional_o   │
│   160 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/commands/decorators.py:157 in wrapper   │
│                                                                                                  │
│   154 │   def wrapper(**options):                                                                │
│   155 │   │   if execute_before_command_using_new_options:                                       │
│   156 │   │   │   execute_before_command_using_new_options(**options)                            │
│ ❱ 157 │   │   return func(**options)                                                             │
│   158 │                                                                                          │
│   159 │   wrapper.__signature__ = _extend_signature_with_additional_options(func, additional_o   │
│   160 │   return wrapper                                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/plugins/streamlit/commands.py:157 in        │
│ streamlit_deploy                                                                                 │
│                                                                                                  │
│   154 │                                                                                          │
│   155 │   streamlit_id = FQN.from_identifier_model(streamlit).using_context()                    │
│   156 │                                                                                          │
│ ❱ 157 │   url = StreamlitManager().deploy(                                                       │
│   158 │   │   streamlit_id=streamlit_id,                                                         │
│   159 │   │   environment_file=Path(environment_file),                                           │
│   160 │   │   pages_dir=Path(pages_dir),                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/plugins/streamlit/manager.py:194 in deploy  │
│                                                                                                  │
│   191 │   │   │   │   f"{stage_name}/{streamlit_name_for_root_location}"                         │
│   192 │   │   │   )                                                                              │
│   193 │   │   │                                                                                  │
│ ❱ 194 │   │   │   self._put_streamlit_files(                                                     │
│   195 │   │   │   │   root_location,                                                             │
│   196 │   │   │   │   main_file,                                                                 │
│   197 │   │   │   │   environment_file,                                                          │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/plugins/streamlit/manager.py:62 in          │
│ _put_streamlit_files                                                                             │
│                                                                                                  │
│    59 │   │   │   stage_manager.put(environment_file, root_location, 4, True)                    │
│    60 │   │                                                                                      │
│    61 │   │   if pages_dir and pages_dir.exists():                                               │
│ ❱  62 │   │   │   stage_manager.put(pages_dir / "*.py", f"{root_location}/pages", 4, True)       │
│    63 │   │                                                                                      │
│    64 │   │   if additional_source_files:                                                        │
│    65 │   │   │   for file in additional_source_files:                                           │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/plugins/stage/manager.py:267 in put         │
│                                                                                                  │
│   264 │   │   │   stage_path = self.get_standard_stage_prefix(stage_path)                        │
│   265 │   │   │   local_resolved_path = path_resolver(str(local_path))                           │
│   266 │   │   │   log.info("Uploading %s to %s", local_resolved_path, stage_path)                │
│ ❱ 267 │   │   │   cursor = self._execute_query(                                                  │
│   268 │   │   │   │   f"put {self._to_uri(local_resolved_path)} {self.quote_stage_name(stage_p   │
│   269 │   │   │   │   f"auto_compress={str(auto_compress).lower()} parallel={parallel} overwri   │
│   270 │   │   │   )                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/sql_execution.py:85 in _execute_query   │
│                                                                                                  │
│    82 │   │   return stream_generator if return_cursors else list()                              │
│    83 │                                                                                          │
│    84 │   def _execute_query(self, query: str, **kwargs):                                        │
│ ❱  85 │   │   *_, last_result = self._execute_queries(query, **kwargs)                           │
│    86 │   │   return last_result                                                                 │
│    87 │                                                                                          │
│    88 │   def _execute_queries(self, queries: str, **kwargs):                                    │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/cli/api/sql_execution.py:89 in _execute_queries │
│                                                                                                  │
│    86 │   │   return last_result                                                                 │
│    87 │                                                                                          │
│    88 │   def _execute_queries(self, queries: str, **kwargs):                                    │
│ ❱  89 │   │   return list(self._execute_string(dedent(queries), **kwargs))                       │
│    90 │                                                                                          │
│    91 │   def use(self, object_type: ObjectType, name: str):                                     │
│    92 │   │   try:                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/connection.py:890 in execute_stream   │
│                                                                                                  │
│    887 │   │   non_empty_statements = [e for e in split_statements_list if e[0]]                 │
│    888 │   │   for sql, is_put_or_get in non_empty_statements:                                   │
│    889 │   │   │   cur = self.cursor(cursor_class=cursor_class)                                  │
│ ❱  890 │   │   │   cur.execute(sql, _is_put_get=is_put_or_get, **kwargs)                         │
│    891 │   │   │   yield cur                                                                     │
│    892 │                                                                                         │
│    893 │   def __set_error_attributes(self) -> None:                                             │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/cursor.py:1046 in execute             │
│                                                                                                  │
│   1043 │   │   │   │   │   multipart_threshold=data.get("threshold"),                            │
│   1044 │   │   │   │   │   use_s3_regional_url=self._connection.enable_stage_s3_privatelink_for  │
│   1045 │   │   │   │   )                                                                         │
│ ❱ 1046 │   │   │   │   sf_file_transfer_agent.execute()                                          │
│   1047 │   │   │   │   data = sf_file_transfer_agent.result()                                    │
│   1048 │   │   │   │   self._total_rowcount = len(data["rowset"]) if "rowset" in data else -1    │
│   1049                                                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py:390 in execute │
│                                                                                                  │
│    387 │                                                                                         │
│    388 │   def execute(self) -> None:                                                            │
│    389 │   │   self._parse_command()                                                             │
│ ❱  390 │   │   self._init_file_metadata()                                                        │
│    391 │   │                                                                                     │
│    392 │   │   if self._command_type == CMD_TYPE_UPLOAD:                                         │
│    393 │   │   │   self._process_file_compression_type()                                         │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py:986 in         │
│ _init_file_metadata                                                                              │
│                                                                                                  │
│    983 │   │   │   │   │   if "data" in self._ret and "src_locations" in self._ret["data"]       │
│    984 │   │   │   │   │   else "None"                                                           │
│    985 │   │   │   │   )                                                                         │
│ ❱  986 │   │   │   │   Error.errorhandler_wrapper(                                               │
│    987 │   │   │   │   │   self._cursor.connection,                                              │
│    988 │   │   │   │   │   self._cursor,                                                         │
│    989 │   │   │   │   │   ProgrammingError,                                                     │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/errors.py:290 in errorhandler_wrapper │
│                                                                                                  │
│   287 │   │   │   exception to the first handler in that order.                                  │
│   288 │   │   """                                                                                │
│   289 │   │                                                                                      │
│ ❱ 290 │   │   handed_over = Error.hand_to_other_handler(                                         │
│   291 │   │   │   connection,                                                                    │
│   292 │   │   │   cursor,                                                                        │
│   293 │   │   │   error_class,                                                                   │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/errors.py:345 in                      │
│ hand_to_other_handler                                                                            │
│                                                                                                  │
│   342 │   │   │   connection.messages.append((error_class, error_value))                         │
│   343 │   │   if cursor is not None:                                                             │
│   344 │   │   │   cursor.messages.append((error_class, error_value))                             │
│ ❱ 345 │   │   │   cursor.errorhandler(connection, cursor, error_class, error_value)              │
│   346 │   │   │   return True                                                                    │
│   347 │   │   elif connection is not None:                                                       │
│   348 │   │   │   connection.errorhandler(connection, cursor, error_class, error_value)          │
│                                                                                                  │
│ /usr/local/lib/python3.9/site-packages/snowflake/connector/errors.py:221 in default_errorhandler │
│                                                                                                  │
│   218 │   │   """                                                                                │
│   219 │   │   errno = error_value.get("errno")                                                   │
│   220 │   │   done_format_msg = error_value.get("done_format_msg")                               │
│ ❱ 221 │   │   raise error_class(                                                                 │
│   222 │   │   │   msg=error_value.get("msg"),                                                    │
│   223 │   │   │   errno=None if errno is None else int(errno),                                   │
│   224 │   │   │   sqlstate=error_value.get("sqlstate"),                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ProgrammingError: 253006: 253006: File doesn't exist: ['pages/*.py']

How to reproduce

Create a single page app with no file in pages/ directory.
Try to run snow streamlit deploy --replace to see if it is able to deploy

[sfc-gh-astus]

Fixed in #1499

[ankitsr92]

@sfc-gh-astus Why would we end up failing the deploy if the pages directory is empty? Why not skip the directory if there are no *.py files.
The deploy is only working upon removing pages/ directory and removing entry from snowflake.yml "pages_dir: pages/"

It is failing if I have "pages_dir: pages/" in snowflake.yml and pages/ directory exists with .placeholder ( no .py file )

When we are setting up projects we still want to keep the directories available and config available to be used if pages are added in future.

Ideally the code should be independent of it and should ignore if *.py files are not present.

Using snowflake cli version 2.8