Utility functions for the parser module. More...

#include <stdio.h>
#include <string.h>
#include "parser.h"

Include dependency graph for parser_utils.c:

Functions
bool	p_has_tokens (Parser *psr)
	Checks if there are more tokens available for parsing.

Token	p_peek (Parser *psr, size_t offset)
	Peeks at a token at the specified offset from current position.

Token	p_current (Parser *psr)
	Gets the current token without advancing the parser position.

Token	p_advance (Parser *psr)
	Advances to the next token and returns the current token.

Token	p_consume (Parser psr, LumaTokenType type, const char error_msg)
	Consumes a token of the expected type or reports an error.

char *	get_name (Parser *psr)
	Extracts and duplicates the current token's string value.

bool	init_parser_arrays (Parser parser, GrowableArray stmts, GrowableArray *modules)

const char *	parse_module_declaration (Parser parser, char *out_module_doc)

char *	collect_doc_comments (Parser *parser)
	Collects consecutive documentation comments before a declaration.

void	consume_doc_comments (Parser *parser)

Detailed Description

Utility functions for the parser module.

This file contains essential utility functions that provide the core infrastructure for token navigation and manipulation during parsing. These functions abstract the low-level details of token stream management and provide a clean interface for parser operations.

The utilities include:

Token stream navigation (current, peek, advance)
Token consumption with error reporting
Token validation and bounds checking
String extraction and memory management for identifiers

All functions are designed to be safe and handle edge cases like end-of-stream conditions gracefully by returning EOF tokens.

Author: [Your Name]

Date: [Current Date]

Version: 1.0

Function Documentation

◆ collect_doc_comments()

char * collect_doc_comments ( Parser * parser )

Collects consecutive documentation comments before a declaration.

Accumulates all doc comments (/// or //!) that appear immediately before the current token. Returns them as a single string with newlines preserved.

Parameters

parser Pointer to the parser instance

Returns: Concatenated doc comment string, or NULL if no doc comments found

◆ consume_doc_comments()

void consume_doc_comments ( Parser * parser )

◆ get_name()

char * get_name ( Parser * psr )

Extracts and duplicates the current token's string value.

This function creates a null-terminated string copy of the current token's value using the arena allocator. It's primarily used for extracting identifier names, string literals, and other textual token content that needs to be preserved in the AST.

Parameters

psr	Pointer to the parser instance

Returns: Pointer to a null-terminated string containing the current token's value, allocated using the parser's arena allocator

Note: The returned string is allocated using arena_alloc() for automatic memory management; The string is properly null-terminated for safe use with standard string functions; The original token value is copied, so the returned string is independent; Memory is automatically managed by the arena - no manual deallocation needed

Warning: This function assumes the current token has a valid value field; Should only be called when you know the current token contains string data

See also: CURRENT_TOKEN_VALUE(), CURRENT_TOKEN_LENGTH(), arena_alloc()

◆ init_parser_arrays()

bool init_parser_arrays	(	Parser *	parser,
		GrowableArray *	stmts,
		GrowableArray *	modules
	)

◆ p_advance()

Token p_advance ( Parser * psr )

Advances to the next token and returns the current token.

This function moves the parser position forward by one token and returns the token that was current before advancing. This is the primary mechanism for consuming tokens during parsing.

Parameters

psr	Pointer to the parser instance

Returns: The token that was current before advancing, or EOF token if already at the end of the token stream

Note: The parser position is incremented only if there are tokens available; Safe to call at end of stream - returns EOF token and doesn't advance; This is a mutating operation that changes parser state

Warning: After calling this function, subsequent calls to p_current() will return the next token in the stream

See also: p_current(), p_has_tokens(), p_consume()

◆ p_consume()

Token p_consume	(	Parser *	psr,
		LumaTokenType	type,
		const char *	error_msg
	)

Consumes a token of the expected type or reports an error.

This function is used when the parser expects a specific token type at the current position. If the current token matches the expected type, it advances and returns the token. If not, it reports a syntax error with the provided error message.

Parameters

psr	Pointer to the parser instance
type	The expected token type that should be at the current position
error_msg	Error message to display if the token doesn't match

Returns: The consumed token if it matches the expected type, or an EOF token if there's a mismatch (indicating an error)

Note: This function combines token validation and consumption in one operation; Error reporting includes current line, column, and token length information; The parser position advances only on successful token match; On error, the parser position remains unchanged

Warning: Always check the return value's type if you need to handle parsing errors

See also: p_current(), p_advance(), parser_error(), LumaTokenType

◆ p_current()

Token p_current ( Parser * psr )

Gets the current token without advancing the parser position.

This function returns the token at the current parser position without modifying the parser state. It's the most frequently used function for examining the current token during parsing.

Parameters

psr	Pointer to the parser instance

Returns: Current token, or an EOF token if at the end of the token stream

Note: This function is safe to call even when at the end of the token stream; The returned token contains all token information: type, value, position, etc.; Does not modify parser state - can be called multiple times safely

See also: p_advance(), p_peek(), Token

◆ p_has_tokens()

bool p_has_tokens ( Parser * psr )

Checks if there are more tokens available for parsing.

This function determines whether the parser has reached the end of the token stream. It's used throughout the parser to control parsing loops and prevent buffer overruns when accessing tokens.

Parameters

psr	Pointer to the parser instance

Returns: true if there are more tokens to process (and current token is not EOF), false if at end of stream or current token is EOF

Note

This function performs two checks:

Position is within the token array bounds
Current token is not TOK_EOF (which marks end of input)

See also: p_current(), p_advance()

◆ p_peek()

Token p_peek	(	Parser *	psr,
		size_t	offset
	)

Peeks at a token at the specified offset from current position.

This function allows looking ahead in the token stream without advancing the current position. It's useful for making parsing decisions based on upcoming tokens (lookahead parsing).

Parameters

psr	Pointer to the parser instance
offset	Number of positions ahead to look (0 = current, 1 = next, etc.)

Returns: Token at the specified offset, or an EOF token if the offset goes beyond the end of the token stream

Note

Common usage patterns:

p_peek(parser, 0) is equivalent to p_current(parser)
p_peek(parser, 1) looks at the next token
Safe to use with any offset; returns EOF token for out-of-bounds access

See also: p_current(), Token, TOK_EOF

◆ parse_module_declaration()

const char * parse_module_declaration	(	Parser *	parser,
		char **	out_module_doc
	)

Functions

Detailed Description

Function Documentation

◆ collect_doc_comments()

◆ consume_doc_comments()

◆ get_name()

◆ init_parser_arrays()

◆ p_advance()

◆ p_consume()

◆ p_current()

◆ p_has_tokens()

◆ p_peek()

◆ parse_module_declaration()