Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Split token support * Defining new structures to store token data * IPR work on porting to a new tokenization method. * Actually compiles lol * Making pretty good progress. Back to trying test cases * Tokens now get split, and we have new kLAngle and kRAngle token kinds (used by the printer). Next up is token alignment. * Working reasonably well with new clang patches. Next up is porting Bounds.cpp * Updated Bounds.cpp code now compiles. * Progress on split tokens and improved bounds. * More in-progress work. Trying to track down the source of some non-converted kLess and kGreater. * Fix segfault. Mitigate issue where control of printing goes from pasta into clang. Eventually solve this properly. * Improvements to bounds checking. Turn on new option in patched clang. * Fixes for token provenance * Bug fix for empty files. * Bounds improvements * Add printing support for nested name specifiers * Fixes a few more bounds issues; disables logging * Changes to hook the new attribute printer stuff. * Retain original whitespace and original comments (not from macros though) in the pre-processed files. * Added some tracking to find directives within the parsed tokens * Add some APIs for getting from macros back to parsed tokens. * Adds APIs to get from macros back to parsed tokens. * Fix split token tracker * Bug fixes and improvements. * Bug fixes for macro test cases. * Bug fixes. * Simplify API. * Update to use RemappedDecl * Add decl contexts to token contexts. * Add DeclContext::From(decl) * Bounds improvements. * Working on bounds, nullables. * Update after bootstrap. Fiddle with bounds. * Fix bounds with template type params. Fix UAF with decl printer. * Disable align and lex debug * Fix for variable form issue with bounds for function protos. * Inverted logic * Adds PrintedTokenRange::AdoptWhitespace. Makes printing of template arguments try to use canonical types. * Pag/track split token: Redefine methods for finding substitutions that align with tokens (#118) * Remove print-tokens target from test suite - Remove the deleted target `print-tokens` from the list of dependencies in the `test` directory's CMakeLists - Remove tool substitutions for `print-tokens` in the LIT Python configuration file. * Redefine methods for finding aligned substitutions - Redefine the methods `AlignedSubstitutions()`, `NextMacroToken()`, and `MacroDerivationChain()` to work with recent changes to `Token` class. * Return a vector from AlignedParameterSubstitutions - Change `AlignedParameterSubstitutions()` to return a vector instead of a map to preserve the order of macro parameters in the return value. * Fixes to token alignment, making merging less aggressive. * Improves pretty printing * Fix bounds/printer/align issues. * Fix eof/bof marker assert when there is a -include. * Pag/track split token: Redefine methods for finding substitutions that align with tokens (#118) * Remove print-tokens target from test suite - Remove the deleted target `print-tokens` from the list of dependencies in the `test` directory's CMakeLists - Remove tool substitutions for `print-tokens` in the LIT Python configuration file. * Redefine methods for finding aligned substitutions - Redefine the methods `AlignedSubstitutions()`, `NextMacroToken()`, and `MacroDerivationChain()` to work with recent changes to `Token` class. * Return a vector from AlignedParameterSubstitutions - Change `AlignedParameterSubstitutions()` to return a vector instead of a map to preserve the order of macro parameters in the return value. * Bringing back function type locs, at least partially. Might affect bounds in the future, but was really affecting the beginning of functions in C code. Added a fix for musl libc in preprocessor tokens. * Added recovery mode to token alignment. Fixes in bounds. * update is interface * Inject extra whitespace. Fixup bounds. * Disable bounds on cxxdeductionguides. * Add null check for BaseType * Weaken conditions on ExpandRange for FunctionDecls. Needed for lambdas. Make preprocessor output thread-id specific. * Improvements to bounds to handle lambdas * Changes some logic in decl/type printing for nested namespace specifiers. Re-bootstraps with by-reference template arguments. * Don't print attributes from pragmas. Add exclusive upper bound on searches for r_parens when the function has a variable form. * Take CookedLiteral nullable bootstrap from kumarak/split_token. * Mark CXXForRangeStmt::Initializer as nullable * Add nullable. * Re-bootstrap pasta for nullable methods. * Change Generator -> Iterator for python schema return type. * Don't trigger assertion with CXXCtorInitializer::IsBaseVirtual * Type printing for template specializations and noexcept clauses * Fix sigsegv * Merge with pag/track_split_token (#120) * Fix ellipsis token asset and assert due to parsed token * Strip BOM from UTF8 and update bootstrap * Update lexer to handler BOM and update RemoveBOM * Move HasBOM to FileManager.h * Bootstrap getVisibility methods * Fix source range for call expr * Update Globals.cpp * Update Stmt.cpp --------- Co-authored-by: Peter Goodman <peter@trailofbits.com> * Strips carriage returns from file data, after stripping BOMs. Fiddle with type printing * Fix pasta wrapper to handle assert in clang * Update pre-processor flag to clear ImplicitPCHInclude * Integrate target and aux target features into running of the compile command * Fix stmt printer to avoid issues due to comments parsing (#121) * Add ARM builtins. * Bump vendored llvm-project. * Integrate patch from Michael Zappa related to nanobind. * Remove hard-coded path --------- Co-authored-by: Brent Pappas <pappasbrent@gmail.com> Co-authored-by: Akshay K <iit.akshay@gmail.com>
- Loading branch information