Struct TokenStream

pub struct TokenStream(/* private fields */);

The main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and, conversely, collecting a number of token trees into one stream.

This is both the input and output of #[proc_macro], #[proc_macro_attribute] and #[proc_macro_derive] definitions.

Implementations

impl TokenStream

pub fn new() -> TokenStream

Returns an empty TokenStream containing no token trees.

pub fn is_empty(&self) -> bool

Checks if this TokenStream is empty.

pub fn expand_expr(&self) -> Result<TokenStream, ExpandError>

🔬This is a nightly-only experimental API. (proc_macro_expand #90765)

Parses this TokenStream as an expression and attempts to expand any macros within it. Returns the expanded TokenStream.

Currently only expressions expanding to literals will succeed, although this may be relaxed in the future.

NOTE: In error conditions, expand_expr may leave macros unexpanded, report an error, failing compilation, and/or return an Err(..). The specific behavior for any error condition, and what conditions are considered errors, is unspecified and may change in the future.

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

Prints token in a form convenient for debugging.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenStream

fn default() -> Self

Returns the “default value” for a type.

impl Display for TokenStream

Prints the token stream as a string that is supposed to be losslessly convertible back into the same token stream (modulo spans), except for possibly TokenTree::Groups with Delimiter::None delimiters and negative numeric literals.

Note: the exact form of the output is subject to change, e.g. there might be changes in the whitespace used between tokens. Therefore, you should not do any kind of simple substring matching on the output string (as produced by to_string) to implement a proc macro, because that matching might stop working if such changes happen. Instead, you should work at the TokenTree level, e.g. matching against TokenTree::Ident, TokenTree::Punct, or TokenTree::Literal.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Extend<TokenStream> for TokenStream

fn extend<I: IntoIterator<Item = TokenStream>>(&mut self, streams: I)

Extends a collection with the contents of an iterator.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one #72631)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one #72631)

Reserves capacity in a collection for the given number of additional elements.

impl Extend<TokenTree> for TokenStream

fn extend<I: IntoIterator<Item = TokenTree>>(&mut self, trees: I)

Extends a collection with the contents of an iterator.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one #72631)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one #72631)

Reserves capacity in a collection for the given number of additional elements.

impl From<TokenTree> for TokenStream

Creates a token stream containing a single token tree.

fn from(tree: TokenTree) -> TokenStream

Converts to this type from the input type.

impl FromIterator<TokenStream> for TokenStream

A “flattening” operation on token streams, collects token trees from multiple token streams into a single stream.

fn from_iter<I: IntoIterator<Item = TokenStream>>(streams: I) -> Self

Creates a value from an iterator.

impl FromIterator<TokenTree> for TokenStream

Collects a number of token trees into a single stream.

fn from_iter<I: IntoIterator<Item = TokenTree>>(trees: I) -> Self

Creates a value from an iterator.

impl FromStr for TokenStream

Attempts to break the string into tokens and parse those tokens into a token stream. May fail for a number of reasons, for example, if the string contains unbalanced delimiters or characters not existing in the language. All tokens in the parsed stream get Span::call_site() spans.

NOTE: some errors may cause panics instead of returning LexError. We reserve the right to change these errors into LexErrors later.

type Err = LexError

The associated error which can be returned from parsing.

fn from_str(src: &str) -> Result<TokenStream, LexError>

Parses a string s to return a value of this type.

impl IntoIterator for TokenStream

type Item = TokenTree

The type of the elements being iterated over.

type IntoIter = IntoIter

Which kind of iterator are we turning this into?

fn into_iter(self) -> IntoIter

Creates an iterator from a value.

impl ToTokens for TokenStream

fn to_tokens(&self, tokens: &mut TokenStream)

🔬This is a nightly-only experimental API. (proc_macro_totokens #130977)

Write self to the given TokenStream.

fn into_token_stream(self) -> TokenStream

🔬This is a nightly-only experimental API. (proc_macro_totokens #130977)

Convert self directly into a TokenStream object.

fn to_token_stream(&self) -> TokenStream

🔬This is a nightly-only experimental API. (proc_macro_totokens #130977)

Convert self directly into a TokenStream object.

impl !Send for TokenStream

impl !Sync for TokenStream