Skip to content

Latest commit

 

History

History
318 lines (283 loc) · 9.09 KB

NOTES.md

File metadata and controls

318 lines (283 loc) · 9.09 KB

Targets: Module System Requirements

  • Support simple module organization:
    • Module names from file paths
    • Optional mod.pr for module reorganization (future)
  • Module file organization:
    • Each .pr file is a module based on its path
    • Optional mod.pr for module grouping (future)
    • Re-exports via 'pub use' statements
  • Support file-based modules with path-based naming
  • Import syntax: use module.{symbol} and use module as alias
  • Relative imports: ..module and .module
  • Public/private visibility control with pub keyword
  • Hierarchical module namespacing
  • Prevent circular dependencies
  • Module interface type checking
  • Module-level initialization code
  • Caching of compiled modules
  • Proper error handling for missing/incompatible modules

Symbol Resolution

  • Support both qualified (math.vector.dot) and unqualified (dot) names. Example: src/math/vector.pr exports dot -> math.vector.dot
  • Handle name conflicts across modules
  • Track symbol visibility (public/private)
  • Support re-exports of symbols through 'pub use'
  • Maintain debugging information for symbols
  • Handle cross-module type checking

Build System

  • Construct dependency graph from file paths and use statements
  • Support parallel compilation of independent modules
  • Implement incremental compilation
  • Cache build artifacts
  • Track file modifications for minimal rebuilds
  • Support different optimization levels
  • Generate useful error messages

Compilation Pipeline

  1. Module Resolution:

    • Scan project directory for .pr files
    • Map file paths to module names
    • Build dependency graph from 'use' statements
    • Validate imports and re-exports

  2. Parsing & Analysis:

    • Parse source to AST
    • Perform semantic analysis
    • Cross-module type checking

  3. IR Generation:

    • Convert to 3-address code
    • Build control flow graph
    • Handle module boundaries
    • Preserve debug information

  4. Optimization Passes:

    • Constant folding
    • Dead code elimination
    • Function inlining
    • Cross-module optimization
    • Register allocation

  5. Code Generation:

    • Target-specific instruction selection
    • Handle platform calling conventions
    • Generate machine code or LLVM IR

Technical Considerations

  • Maintain source locations through all passes
  • Support different optimization levels
  • Consider LLVM integration
  • Handle platform-specific details
  • Support debugging information
  • Enable link-time optimization
  • Consider compile-time memory usage
  • Support incremental compilation

Error Handling

  • Clear module-related error messages
  • Cycle detection in imports
  • Type mismatch across modules
  • Missing imported files
  • Version conflicts
  • Invalid visibility usage
  • Platform-specific errors

Testing Requirements

  • Unit tests for each compilation phase
  • Integration tests across modules
  • Performance benchmarks
  • Error case coverage
  • Cross-platform testing
  • Build system verification

Phases for Targets: Phase 1 Plan - Basic Module Discovery and Loading

  1. First: Script Mode Infrastructure

    • Create CompileContext struct to track compilation state
    • Support script-mode import resolution
    • Track dependencies starting from main script
    • Load and parse imported files
    • Handle relative paths (. and ..)

  2. Then: Project Mode Infrastructure

    • Add project.pr file parsing
    • Define project structure conventions (src/ etc)
    • Map project paths to module names
    • Support project-wide dependency tracking

  3. Shared Module System Components:

    • Module dependency graph building
    • Circular dependency detection
    • File path to module name mapping
    • Module symbol visibility tracking
    • Import resolution and validation

  4. Processing Flow:

Script Mode:
script.pr -> find imports -> resolve paths -> load files -> parse -> dependency graph

Project Mode:
project.pr -> find src/ -> scan for .pr files -> build module tree -> parse -> dependency graph

  1. Key Components Needed:

    • CompileContext: Tracks overall compilation state
    • ModuleLoader: Handles file loading and path resolution
    • DependencyGraph: Tracks module relationships
    • ModuleResolver: Maps between file paths and module names
    • ImportResolver: Handles use statements and visibility

  2. Error Handling:

    • Missing files
    • Invalid imports
    • Circular dependencies
    • Visibility violations
    • Path resolution errors

Phase 2: Symbol Management

  1. Enhance symbol table to:
    • Support hierarchical scoping
    • Track module-level symbols
    • Handle qualified names
    • Manage symbol visibility
  2. Modify semantic analyzer to:
    • Resolve symbols across modules
    • Check visibility rules
    • Handle import statements
    • Track cross-module dependencies

Phase 3: Type System Integration

  1. Extend type checking to:
    • Handle types from other modules
    • Check visibility of imported types
    • Support qualified type names
  2. Update error reporting for:
    • Module-related errors
    • Cross-module type mismatches
    • Visibility violations

Phase 4: Build System Enhancement

  1. Implement dependency tracking:
    • Build module dependency graph
    • Detect circular dependencies
    • Track file modifications
  2. Add compilation phases:
    • Parse all dependent modules
    • Resolve cross-module symbols
    • Type check across modules
    • Generate code in correct order

Phase 5: Caching and Performance

  1. Add caching system:
    • Cache parsed modules
    • Cache type information
    • Track file modifications
  2. Implement incremental compilation:
    • Rebuild only changed modules
    • Reuse cached information
    • Track interface changes

Phase 6: Error Handling and Diagnostics

  1. Enhance error reporting:
    • Module-related errors
    • Import resolution errors
    • Cross-module type errors
  2. Add debugging support:
    • Module dependency visualization
    • Symbol resolution tracing
    • Type checking diagnostics

Future Considerations:

  • Symbol table improvements:
    • String interning for identifiers
    • Perfect hashing for common keywords/operators
    • Arena allocation for AST nodes
    • Pre-allocated type structures for common types
  • Type checking optimizations:
    • Cache type equivalence results
    • Structural type comparison optimization (hash-consing)
    • Parallel type checking for independent modules
    • Incremental type checking on file changes
    • Fast path for primitive type operations
  • Memory efficiency:
    • Custom allocators for different compilation phases
    • Reduce cloning with lifetime management
    • Reference counting optimization
    • Pool allocators for similar-sized AST nodes

Compile-Time Execution

  • Safe operations whitelist: 
    - Arithmetic operations
- String manipulation
- Type checks/assertions
- Pattern matching
- Pure function evaluation
- Array/tuple operations
- Conditional branching
  • Restricted operations blacklist: 
    - Memory allocation/deallocation
- File I/O operations
- Network access
- System calls
- Random number generation
- Time-dependent operations
- Pointer arithmetic
  • Compile-time features:
    • Type-level computation
    • Constant expression evaluation
    • Static assertions
    • Generic specialization
    • Template metaprogramming
    • Custom compile-time procedures
    • Code generation helpers

Safety and Constraints

  • Resource limits for compile-time execution:
    • Maximum iteration count
    • Stack depth limits
    • Computation time bounds
    • Memory usage caps
  • Determinism guarantees:
    • Ensure reproducible builds
    • Platform-independent results
    • Version-stable evaluation
  • Static analysis:
    • Effect system for compile-time safety
    • Resource usage tracking
    • Termination checking
    • Purity analysis

Advanced Type System Features

  • Type inference improvements:
    • Bidirectional type checking
    • Local type inference
    • Higher-ranked type support
    • Type class inference
  • Constraint solving:
    • Efficient unification
    • Incremental constraint solving
    • Subtyping relationship caching
  • Specialization:
    • Generic instantiation caching
    • Monomorphization optimization
    • Trait specialization

Error Recovery and Reporting

  • Improved error recovery:
    • Continue type checking after errors
    • Suggest likely fixes
    • Track error dependencies
  • Error message quality:
    • Context-aware suggestions
    • Type difference explanation
    • Visual error formatting
    • Code snippets in errors

Profiling and Diagnostics

  • Compilation profiling:
    • Phase timing
    • Memory usage tracking
    • Cache hit rates
    • Type checking statistics
  • Debug features:
    • AST visualization
    • Type inference traces
    • Constraint solving visualization
    • Compile-time execution traces

Build System Enhancements

  • Dependency tracking:
    • Fine-grained dependency tracking
    • Interface-level dependencies
    • Cross-module optimization opportunities
  • Caching strategies:
    • Incremental type checking cache
    • Compile-time evaluation cache
    • Cross-build caching
    • Distributed build cache

Future mod.pr Support

  • Module documentation and metadata
  • Module-level initialization
  • Conditional exports based on features
  • Module-level attributes
  • Bulk re-exports
  • Module-level constants and configuration