Modernizing Compiler Design for Carbon Toolchain - Chandler Carruth - CppNow 2023

Is lexing ever a bottleneck though? Even if you push for lexing and parsing 10M lines/second [1], I'd argue that semantic analysis and codegen (for AOT-compiled languages) will dominate the timings. That said, there's no reason not to squeeze every bit of performance out of it! [1]: In this talk about the Carbon language, Chandler Carruth shows and explains some goals/challenges regarding performance: ▶ this video

Chandler discusses it in this video though! ▶ this video You get some traversals for free with this layout (preorder, reverse post order). Can search for subtrees with string searching algorithms or more complex things with regex.

Cache the position of quotes/apostrophes, etc. Optimizing for your edge case would require everyone writing and reading code to conform to this extra thing, which seems completely unnecessary. Machines are pretty fast. ▶ this video Modernizing Compiler Design for Carbon Toolchain - Chandler Carruth - CppNow 2023

No but it is endemic to how we think about compilers up to now, and especially when the rust compiler arch was massively developed, more than 10 years ago. I recommend a watch of this for a recent take on trying to change how we think of these. It is an open research field ▶ this video