Trino connection fails for upload to csv with superset 4.1.0rc2

[abhs50]

`WARNING:superset.views.error_handling:CommandException
Traceback (most recent call last):
File "/app/superset/commands/database/uploaders/base.py", line 114, in _dataframe_to_database
database.db_engine_spec.df_to_sql(
File "/app/superset/db_engine_specs/trino.py", line 515, in df_to_sql
if database.has_table_by_name(table.table, table.schema):
AttributeError: 'Database' object has no attribute 'has_table_by_name'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 119, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1364, in time_function
response = func(*args, **kwargs)
File "/app/superset/utils/log.py", line 303, in wrapper
value = f(*args, **kwargs)
File "/app/superset/views/base_api.py", line 106, in wraps
return f(self, *args, **kwargs)
File "/app/superset/databases/api.py", line 1727, in csv_upload
).run()
File "/app/superset/utils/decorators.py", line 266, in wrapped
return on_error(ex)
File "/app/superset/utils/decorators.py", line 236, in on_error
raise ex
File "/app/superset/utils/decorators.py", line 259, in wrapped
result = func(*args, **kwargs)
File "/app/superset/commands/database/uploaders/base.py", line 154, in run
self._reader.read(self._file, self._model, self._table_name, self._schema)
File "/app/superset/commands/database/uploaders/base.py", line 86, in read
self._dataframe_to_database(
File "/app/superset/commands/database/uploaders/base.py", line 129, in _dataframe_to_database
raise DatabaseUploadFailed(exception=ex) from ex
superset.commands.database.exceptions.DatabaseUploadFailed: Database upload file failed`

[dosubot[bot]]

Hey there, @abhs50! I'm here to help you with any bugs, questions, or contributions you might have. Let's tackle this together!

The Trino connection failure when uploading a CSV in Superset 4.1.0rc2 is due to an AttributeError caused by the Database object not having the has_table_by_name attribute. This issue arises in the df_to_sql method, where the code attempts to check if a table exists using database.has_table_by_name(table.table, table.schema). Since this method is not implemented in the Database class, it results in an AttributeError ^[1].

A similar issue was discussed in a closed thread, where the suggested solution was to use the trino==0.318.0 PyPI module, which might help resolve the problem ^[2].

To continue talking to Dosu, mention @dosu.

[abhs50]

@dosu : This might be due to Trino not supporting database.has_table_by_name(table.table, table.schema) even in the latest version trino-0.330.0

[dosubot[bot]]

The issue you're encountering is likely because the Database class in Superset does not have a has_table_by_name method implemented for Trino, which is causing the AttributeError. This method is not universally available across all database engine specifications in Superset, and it seems that Trino's implementation does not include it.

To address this, you might need to modify the df_to_sql method in trino.py to use an alternative approach for checking if a table exists. One such alternative is using the validate_table_exists method from the DatasetDAO class, which attempts to retrieve the table using the get_table method and returns True if successful ^[1]. This could be a viable workaround until a more permanent solution is implemented in the codebase.

To continue talking to Dosu, mention @dosu.

[kar0t]

We might use 'replace' instead of 'fail' strategy.
Current implementation looks like

        if to_sql_kwargs["if_exists"] == "fail":
            if database.has_table_by_name(table.table, table.schema):
                raise SupersetException("Table already exists")
        elif to_sql_kwargs["if_exists"] == "replace":
            with cls.get_engine(database) as engine:
                engine.execute(f"DROP TABLE IF EXISTS {str(table)}")

[kar0t]

After then I got this error message

Traceback (most recent call last):
  File "/app/superset/commands/database/uploaders/base.py", line 114, in _dataframe_to_database
    database.db_engine_spec.df_to_sql(
  File "/app/superset/db_engine_specs/trino.py", line 546, in df_to_sql
    location=upload_to_s3(
  File "/app/superset/db_engine_specs/hive.py", line 77, in upload_to_s3
    raise Exception(  # pylint: disable=broad-exception-raised
Exception: No upload bucket specified. You can specify one in the config file.

[kar0t]

Related Config: https://github.com/apache/superset/blob/master/superset/config.py#L1138C1-L1177C1

[devlatte]

I just created a bug issue related to this: #31784. Please take a look.

Even after adding the relevant config to config.py, it doesn't work due to the boto3 version issue. This functionality worked perfectly in version 4.0.2, but I'm in a situation where rolling back is difficult. It's quite frustrating...