Enum som_lexer::Token

pub enum Token {
    Not,
    And,
    Or,
    Star,
    Div,
    Mod,
    Plus,
    Minus,
    Equal,
    More,
    Less,
    Comma,
    At,
    Per,
    NewBlock,
    EndBlock,
    Colon,
    Period,
    Exit,
    Assign,
    NewTerm,
    EndTerm,
    NewArray,
    Pound,
    Primitive,
    Separator,
    LitInteger(i64),
    LitBigInteger(String),
    LitDouble(f64),
    LitString(String),
    LitSymbol(String),
    Identifier(String),
    Keyword(String),
    OperatorSequence(String),
    Comment(String),
    Whitespace,
}

Represents a token from the lexer.

Variants

Not

A tilde, the bitwise ‘not’ operator (~).

And

A ampersand, the binary ‘and’ operator (&).

Or

A vertical bar, the binary ‘or’ operator (|).

Star

A star, the multiplication operator (*).

Div

A forward slash, the division operator (/).

Mod

A backward slash (\\).

Plus

A plus, the addition operator (‘+’).

Minus

A minus sign, the substraction operator (-)

Equal

An equal sign, the equality operator (=).

More

A greater-than sign, the greater-than operator (>).

Less

A lesser-than sign, the lesser-than operator (>).

Comma

A comma (,).

At

An at-sign (@).

Per

A percentage sign, the modulo operator (%).

NewBlock

An opening square-bracket ([).

EndBlock

A closing square-bracket (]).

Colon

A colon (:).

Period

A period, the statement terminator (.).

Exit

A caret, the return operator (^).

Assign

The assignment operator (:=).

NewTerm

An opening parenthesis (().

EndTerm

A closing parenthesis ()).

NewArray

A pound sign immediately followed by opening parenthesis (#().

Pound

A pound sign (#).

Primitive

The primitive keyword (primitive).

Separator

The separator sequence (-------).

LitInteger(i64)

An integer literal (10).

LitBigInteger(String)

A big integer literal (1542252643255252434).

LitDouble(f64)

A floating-point literal (10.6).

LitString(String)

A string literal ('hello, world').

LitSymbol(String)

A symbol literal (#foo).

Identifier(String)

An identifier (foo).

Keyword(String)

A keyword (fromString:).

OperatorSequence(String)

A sequence of operators (eg: >>>).

Comment(String)

A comment ("what a beautiful and majestic piece of code").

Whitespace

Some whitespace ( ).

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token