Skip to content
#

lexer

A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.

Here are 865 public repositories matching this topic...

AccessViolator
AccessViolator commented Feb 24, 2020

Everything in diagrams.css should be scoped to some wrapping css class, because as is it cannot be bundled with the rest of an app's css because of styles like this:

div {
    -webkit-touch-callout: none;
    -webkit-user-select: none;
    -khtml-user-select: none;
    -moz-user-select: none;
    -ms-user-select: none;
    user-select: none;
}

svg {
    width: 100%;
}

Curre

logos
ExoticMatter
ExoticMatter commented Jun 13, 2020

It would be nice if subpatterns could be used within subpatterns.

#[derive(Logos, Copy, Clone, Debug)]
#[logos(subpattern a = r"[a-zA-Z!$%&*/:<=>?^_~]")]
#[logos(subpattern b = r"(?&a)|[0-9+\-.@]")]
pub enum Token {
    #[error]
    Error,
    // ...
}

Currently, this is not supported and causes a regex parse error.

suzizecat
suzizecat commented Apr 2, 2021

Input test case

Any sv file.

Describe what is wrong or missing

As far as I can tell, the verible-verilog-kythe-extractor tool cannot directly write its output into a file.
The issue is that it force having a potentially huge buffer in memory.

It would be nice to have the capability to output the result of the kythe extraction directly to an output file without having to rely on

SteelPhase
SteelPhase commented Nov 26, 2019

Would it be possible to have the regex parser support character classes like \w within other character classes? I had a regex pattern earlier that used the character class [0-9a-zA-Z_\.-], and I attempted to simplify it with [\w\.\-]. I didn't notice this library doesn't support doing that, and was wondering just how difficult that would be to implement. For the time being i'm just expanding

The regex-centric, fast lexical analyzer generator for C++ with full Unicode support. Faster than Flex. Accepts Flex specifications. Generates reusable source code that is easy to understand. Introduces indent/dedent anchors, lazy quantifiers, functions for lex/syntax error reporting, and more. Seamlessly integrates with Bison and other parsers.

  • Updated Mar 16, 2021
  • C++