1. Overview

DBSim codebase structure

Introduction to the main folders and files in dbsim/

1. adapters/: data storage
2. aggregates/: basic aggregate functions (count, sum, etc.)
3. compilers/: compilers for converting logical plan into executable physical plan
4. examples/: example code snippets as tutorials
5. extensions/: modules and utilities for building and managing extended syntax
6. functions/: some basic SQL built-in functions (cos, sqrt, etc.)
7. gui/: code for the GUI
8. planners/: query optimizer, optimization rules and logical cost
9. tests/: unit tests
10. utils/: global utilities
11. ast.py: defines Expr, the base class for AST and logical plan nodes, and all necessary concrete classes for those nodes. Some related utilities like the method to traverse an AST are also defined in this file.
12. compat.py: utilities for compatibility on different versions of Python
13. config.py: configuration parameters
14. dataset.py: the runtime organizer and manager of data adapters and relations
15. field.py: defines fields in schema
16. operations.py: defines visitors and methods to walk on and manipulate AST in visitor pattern
17. query_parser_toolbox.py: core query parsing functions
18. query_parser.py: interfaces to call the core parsing functions
19. query.py: defines class Query for managing and executing queries
20. schema_interpreter.py: functions for resolving schema information to convert AST into logical plan
21. schema.py: defines class Schema to manage schema information
22. table.py: defines abstract class Table, used by data adapters

Content of examples/

1. run_query_end2end.py: example of running a SQL query end-to-end in DBSim
2. parse_query_to_ast.py: example of manually parsing a query into AST and visualizing the AST
3. manipulate_ast_and_logical_plan.py: example of manipulating AST and logical plan, including transforming AST into logical plan, traversing a plan tree, deep copying a plan, etc.
4. run_query_plan.py: example of manually running a query plan (instead of a query) in DBSim
5. manually_optimize_logical_plan.py: example of manually operating the optimizer to optimize a logical plan
6. compute_logical_plan_cost.py: example of using utilities in LogicalCost to compute the logical cost of a plan
7. load_data_from_file_and_run_query.py: example of loading data from file and running queries on it