Marked 中文网

扩展 Marked

¥Extending Marked

为了倡导单一责任和开放/封闭原则，我们试图让扩展 Marked 变得相对轻松。如果你希望添加自定义功能，那么这就是开始的地方。

¥To champion the single-responsibility and open/closed principles, we have tried to make it relatively painless to extend Marked. If you are looking to add custom functionality, this is the place to start.

marked.use()

marked.use(extension) 是扩展 Marked 的推荐方法。extension 对象可以包含 Marked 中可用的任何 option：

¥marked.use(extension) is the recommended way to extend Marked. The extension object can contain any option available in Marked:

import { marked } from 'marked';

marked.use({
  pedantic: false,
  gfm: true,
  breaks: false
});

你还可以一次提供多个 extension 对象。

¥You can also supply multiple extension objects at once.

marked.use(myExtension, extension2, extension3);

\\ EQUIVALENT TO:

marked.use(myExtension);
marked.use(extension2);
marked.use(extension3);

所有选项都将覆盖先前设置的选项，但以下选项除外，这些选项将与现有框架合并，可用于更改或扩展 Marked 的功能：renderer、tokenizer、hooks、walkTokens 和 extensions。

¥All options will overwrite those previously set, except for the following options which will be merged with the existing framework and can be used to change or extend the functionality of Marked: renderer, tokenizer, hooks, walkTokens, and extensions.

• renderer、tokenizer 和 hooks 选项是具有函数的对象，它们将分别合并到内置的 renderer 和 tokenizer 中。

  ¥The renderer, tokenizer, and hooks options are objects with functions that will be merged into the built-in renderer and tokenizer respectively.

• walkTokens 选项是一个函数，将在渲染之前调用该函数对每个标记进行后处理。

  ¥The walkTokens option is a function that will be called to post-process every token before rendering.

• extensions 选项是一个对象数组，可以包含在任何默认解析逻辑发生之前执行的其他自定义 renderer 和 tokenizer 步骤。

  ¥The extensions option is an array of objects that can contain additional custom renderer and tokenizer steps that will execute before any of the default parsing logic occurs.

---

标记管道

在构建自定义扩展之前，了解 Marked 用于将 Markdown 转换为 HTML 的组件非常重要：

¥Before building your custom extensions, it is important to understand the components that Marked uses to translate from Markdown to HTML:

1. 用户提供带有要翻译的输入字符串的标记。

   ¥The user supplies Marked with an input string to be translated.

2. lexer 将输入文本字符串的片段输入到每个 tokenizer 中，并从它们的输出中生成一系列嵌套树结构中的标记。

   ¥The lexer feeds segments of the input text string into each tokenizer, and from their output, generates a series of tokens in a nested tree structure.

3. 每个 tokenizer 都会接收一段 Markdown 文本，如果匹配特定模式，则会生成包含任何相关信息的标记对象。

   ¥Each tokenizer receives a segment of Markdown text and, if it matches a particular pattern, generates a token object containing any relevant information.

4. walkTokens 函数将遍历树中的每个标记并对标记内容执行任何最终调整。

   ¥The walkTokens function will traverse every token in the tree and perform any final adjustments to the token contents.

5. parser 遍历标记树并将每个标记输入到适当的 renderer 中，并将它们的输出连接到最终的 HTML 结果中。

   ¥The parser traverses the token tree and feeds each token into the appropriate renderer, and concatenates their outputs into the final HTML result.

6. 每个 renderer 都会接收一个标记并操纵其内容以生成一段 HTML。

   ¥Each renderer receives a token and manipulates its contents to generate a segment of HTML.

Marked 提供直接覆盖任何现有令牌类型的 renderer 和 tokenizer 的方法，以及插入额外的自定义 renderer 和 tokenizer 函数来处理完全自定义的语法。例如，使用 marked.use({renderer}) 将修改渲染器，而 marked.use({extensions: [{renderer}]}) 将添加新的渲染器。有关如何执行此操作的见解，请参阅 自定义扩展示例。

¥Marked provides methods for directly overriding the renderer and tokenizer for any existing token type, as well as inserting additional custom renderer and tokenizer functions to handle entirely custom syntax. For example, using marked.use({renderer}) would modify a renderer, whereas marked.use({extensions: [{renderer}]}) would add a new renderer. See the custom extensions example for insight on how to execute this.

---

The Renderer : renderer

渲染器定义给定标记的 HTML 输出。如果你在传递给 marked.use() 的选项对象中提供 renderer，则对象中的任何函数都将覆盖该标记类型的默认处理。

¥The renderer defines the HTML output of a given token. If you supply a renderer in the options object passed to marked.use(), any functions in the object will override the default handling of that token type.

多次调用 marked.use() 覆盖同一函数将优先考虑最后分配的版本。覆盖函数可以返回 false 以返回到序列中的上一个覆盖，或者如果所有覆盖都返回 false，则恢复默认行为。返回任何其他值（包括无）将阻止回退行为。

¥Calling marked.use() to override the same function multiple times will give priority to the version that was assigned last. Overriding functions can return false to fall back to the previous override in the sequence, or resume default behavior if all overrides return false. Returning any other value (including nothing) will prevent fallback behavior.

示例：通过添加嵌入式锚标记（如在 GitHub 上）覆盖默认 heading 标记的输出。

¥Example: Overriding output of the default heading token by adding an embedded anchor tag like on GitHub.

// Create reference instance
import { marked } from 'marked';

// Override function
const renderer = {
  heading({ tokens, depth }) {
    const text = this.parser.parseInline(tokens);
    const escapedText = text.toLowerCase().replace(/[^\w]+/g, '-');

    return `
            <h${depth}>
              <a name="${escapedText}" class="anchor" href="#${escapedText}">
                <span class="header-link"></span>
              </a>
              ${text}
            </h${depth}>`;
  }
};

marked.use({ renderer });

// Run marked
console.log(marked.parse('# heading+'));

输出：

¥Output:

<h1>
  <a name="heading-" class="anchor" href="#heading-">
    <span class="header-link"></span>
  </a>
  heading+
</h1>

注意：以以下方式调用 marked.use() 将避免覆盖 heading 令牌输出，但会在此过程中创建一个新的 heading 渲染器。

¥Note: Calling marked.use() in the following way will avoid overriding the heading token output but create a new heading renderer in the process.

marked.use({
 extensions: [{
    name: 'heading',
    renderer(token) {
      return /* ... */
    }
  }]
})

块级渲染器方法

¥Block-level renderer methods

• space(token: Tokens.Space): string

• code(token: Tokens.Code): string

• blockquote(token: Tokens.Blockquote): string

• html(token: Tokens.HTML | Tokens.Tag): string

• heading(token: Tokens.Heading): string

• hr(token: Tokens.Hr): string

• list(token: Tokens.List): string

• listitem(token: Tokens.ListItem): string

• checkbox(token: Tokens.Checkbox): string

• paragraph(token: Tokens.Paragraph): string

• table(token: Tokens.Table): string

• tablerow(token: Tokens.TableRow): string

• tablecell(token: Tokens.TableCell): string

内联级别渲染器方法

¥Inline-level renderer methods

• strong(token: Tokens.Strong): string

• em(token: Tokens.Em): string

• codespan(token: Tokens.Codespan): string

• br(token: Tokens.Br): string

• del(token: Tokens.Del): string

• link(token: Tokens.Link): string

• image(token: Tokens.Image): string

• text(token: Tokens.Text | Tokens.Escape | Tokens.Tag): string

可以在 此处 中找到 Tokens.* 属性。

¥The Tokens.* properties can be found here.

---

The Tokenizer : tokenizer

标记器定义如何将 markdown 文本转换为标记。如果你向 Marked 选项提供 tokenizer 对象，它将与内置标记器合并，并且其中的任何函数都将覆盖该标记类型的默认处理。

¥The tokenizer defines how to turn markdown text into tokens. If you supply a tokenizer object to the Marked options, it will be merged with the built-in tokenizer and any functions inside will override the default handling of that token type.

多次调用 marked.use() 覆盖同一函数将优先考虑最后分配的版本。覆盖函数可以返回 false 以返回到序列中的上一个覆盖，或者如果所有覆盖都返回 false，则恢复默认行为。返回任何其他值（包括无）将阻止回退行为。

¥Calling marked.use() to override the same function multiple times will give priority to the version that was assigned last. Overriding functions can return false to fall back to the previous override in the sequence, or resume default behavior if all overrides return false. Returning any other value (including nothing) will prevent fallback behavior.

示例：覆盖默认 codespan 标记器以包含 LaTeX。

¥Example: Overriding default codespan tokenizer to include LaTeX.

// Create reference instance
import { marked } from 'marked';

// Override function
const tokenizer = {
  codespan(src) {
    const match = src.match(/^\$+([^\$\n]+?)\$+/);
    if (match) {
      return {
        type: 'codespan',
        raw: match[0],
        text: match[1].trim()
      };
    }

    // return false to use original codespan tokenizer
    return false;
  }
};

marked.use({ tokenizer });

// Run marked
console.log(marked.parse('$ latex code $\n\n` other code `'));

输出：

¥Output:

<p><code>latex code</code></p>
<p><code>other code</code></p>

注意：这不完全支持乳胶，请参阅问题 #1948。

¥NOTE: This does not fully support latex, see issue #1948.

块级标记器方法

¥Block level tokenizer methods

• space(src: string): Tokens.Space

• code(src: string): Tokens.Code

• fences(src: string): Tokens.Code

• heading(src: string): Tokens.Heading

• hr(src: string): Tokens.Hr

• blockquote(src: string): Tokens.Blockquote

• list(src: string): Tokens.List

• html(src: string): Tokens.HTML

• def(src: string): Tokens.Def

• table(src: string): Tokens.Table

• lheading(src: string): Tokens.Heading

• paragraph(src: string): Tokens.Paragraph

• text(src: string): Tokens.Text

内联级别标记器方法

¥Inline level tokenizer methods

• escape(src: string): Tokens.Escape

• tag(src: string): Tokens.Tag

• link(src: string): Tokens.Link | Tokens.Image

• reflink(src: string, links: object): Tokens.Link | Tokens.Image | Tokens.Text

• emStrong(src: string, maskedSrc: string, prevChar: string): Tokens.Em | Tokens.Strong

• codespan(src: string): Tokens.Codespan

• br(src: string): Tokens.Br

• del(src: string): Tokens.Del

• autolink(src: string): Tokens.Link

• url(src: string): Tokens.Link

• inlineText(src: string): Tokens.Text

可以在 此处 中找到 Tokens.* 属性。

¥The Tokens.* properties can be found here.

---

Walk Tokens : walkTokens

walkTokens 函数会随每个标记一起调用。在转到兄弟令牌之前，会调用子令牌。每个标记都通过引用传递，因此在传递给解析器时更新会保留下来。启用 async 模式后，将等待返回值。否则返回值将被忽略。

¥The walkTokens function gets called with every token. Child tokens are called before moving on to sibling tokens. Each token is passed by reference so updates are persisted when passed to the parser. When async mode is enabled, the return value is awaited. Otherwise the return value is ignored.

可以使用不同的 walkTokens 函数多次调用 marked.use()。每个函数将按顺序调用，从最后分配的函数开始。

¥marked.use() can be called multiple times with different walkTokens functions. Each function will be called in order, starting with the function that was assigned last.

示例：覆盖标题标记以从 h2 开始。

¥Example: Overriding heading tokens to start at h2.

import { marked } from 'marked';

// Override function
const walkTokens = (token) => {
  if (token.type === 'heading') {
    token.depth += 1;
  }
};

marked.use({ walkTokens });

// Run marked
console.log(marked.parse('# heading 2\n\n## heading 3'));

输出：

¥Output:

<h2 id="heading-2">heading 2</h2>
<h3 id="heading-3">heading 3</h3>

---

Hooks : hooks

钩子是钩入 marked 某些部分的方法。以下钩子可用：

¥Hooks are methods that hook into some part of marked. The following hooks are available:

signature	description
preprocess(markdown: string): string	在将 markdown 发送给 marked 之前处理 markdown。
postprocess(html: string): string	在 marked 解析完成后处理 html。
processAllTokens(tokens: Token[]): Token[]	在 walk token 之前处理所有 token。
provideLexer(): (src: string, options?: MarkedOptions) => Token[]	提供标记 markdown 的函数。
provideParser(): (tokens: Token[], options?: MarkedOptions) => string	提供解析 token 的函数。

可以使用不同的 hooks 函数多次调用 marked.use()。每个函数将按顺序调用，从最后分配的函数开始。

¥marked.use() can be called multiple times with different hooks functions. Each function will be called in order, starting with the function that was assigned last.

示例：根据 front-matter 设置选项

¥Example: Set options based on front-matter

import { marked } from 'marked';
import fm from 'front-matter';

// Override function
function preprocess(markdown) {
  const { attributes, body } = fm(markdown);
  for (const prop in attributes) {
    if (prop in this.options) {
    this.options[prop] = attributes[prop];
    }
  }
  return body;
}

marked.use({ hooks: { preprocess } });

// Run marked
console.log(marked.parse(`
---
breaks: true
---

line1
line2
`.trim()));

输出：

¥Output:

<p>line1<br>line2</p>

示例：使用 isomorphic-dompurify 清理 HTML

¥Example: Sanitize HTML with isomorphic-dompurify

import { marked } from 'marked';
import DOMPurify from 'isomorphic-dompurify';

// Override function
function postprocess(html) {
  return DOMPurify.sanitize(html);
}

marked.use({ hooks: { postprocess } });

// Run marked
console.log(marked.parse(`
<img src=x onerror=alert(1)//>
`));

输出：

¥Output:

<img src="x">

示例：保存 reflinks 以进行分块渲染

¥Example: Save reflinks for chunked rendering

import { marked, Lexer } from 'marked';

let refLinks = {};

// Override function
function processAllTokens(tokens) {
  refLinks = tokens.links;
  return tokens;
}

function provideLexer(src, options) {
  return (src, options) => {
    const lexer = new Lexer(options);
    lexer.tokens.links = refLinks;
    return this.block ? lexer.lex(src) : lexer.inlineTokens(src);
  };
}

marked.use({ hooks: { processAllTokens, provideLexer } });

// Parse reflinks separately from markdown that uses them
marked.parse(`
[test]: http://example.com
`);

console.log(marked.parse(`
[test link][test]
`));

输出：

¥Output:

<p><a href="http://example.com">test link</a></p>

---

Custom Extensions : extensions

你可以向 options 对象提供 extensions 数组。此数组可以包含任意数量的 extension 对象，使用以下属性：

¥You may supply an extensions array to the options object. This array can contain any number of extension objects, using the following properties:

name

A string used to identify the token that will be handled by this extension.

如果名称与现有扩展名或上面列出的 tokenizer/renderer 方法中的现有方法匹配，它们将覆盖先前分配的行为，优先考虑最后分配的扩展。扩展可以返回 false 以恢复到以前的行为。

¥If the name matches an existing extension name, or an existing method in the tokenizer/renderer methods listed above, they will override the previously assigned behavior, with priority on the extension that was assigned last. An extension can return false to fall back to the previous behavior.

level

A string to determine when to run the extension tokenizer. Must be equal to 'block' or 'inline'.

块级扩展将在上述任何块级标记器方法之前处理，通常由 'container-type' 文本（段落、表格、块引用等）组成。

¥A block-level extension will be handled before any of the block-level tokenizer methods listed above, and generally consists of 'container-type' text (paragraphs, tables, blockquotes, etc.).

内联级扩展将在每个块级标记内处理，在上面列出的任何内联级标记器方法之前。这些通常由 'style-type' 文本（斜体、粗体等）组成。

¥An inline-level extension will be handled inside each block-level token, before any of the inline-level tokenizer methods listed above. These generally consist of 'style-type' text (italics, bold, etc.).

start(string src)

A function that returns the index of the next potential start of the custom token.

索引可以是 src.match().index 的结果，甚至是简单的 src.index()。Marked 将使用此功能确保它不会跳过任何应属于自定义标记一部分的文本。

¥The index can be the result of a src.match().index, or even a simple src.index(). Marked will use this function to ensure that it does not skip over any text that should be part of the custom token.

tokenizer(string src, array tokens)

A function that reads string of Markdown text and returns a generated token. The token pattern should be found at the beginning of the src string. Accordingly, if using a Regular Expression to detect a token, it should be anchored to the string start (`^`). The tokens parameter contains the array of tokens that have been generated by the lexer up to that point, and can be used to access the previous token, for instance.

返回值应为具有以下参数的对象：

¥The return value should be an object with the following parameters:

type

A string that matches the name parameter of the extension.

raw

A string containing all of the text that this token consumes from the source.

tokens [optional]

An array of child tokens that will be traversed by the walkTokens function by default.

返回的标记还可以包含你选择的任何其他自定义参数，你的自定义 renderer 可能需要访问这些参数。

¥The returned token can also contain any other custom parameters of your choice that your custom renderer might need to access.

标记器函数可以访问 this 对象中的词法分析器，如果需要进一步解析字符串的任何内部部分（例如处理块标记内文本上的任何内联语法），则可以使用该词法分析器。可能有用的关键功能包括：

¥The tokenizer function has access to the lexer in the this object, which can be used if any internal section of the string needs to be parsed further, such as in handling any inline syntax on the text within a block token. The key functions that may be useful include:

this.lexer.blockTokens(string text, array tokens)

This runs the block tokenizer functions (including any block-level extensions) on the provided text, and appends any resulting tokens onto the tokens array. The tokens array is also returned by the function. You might use this, for example, if your extension creates a "container"-type token (such as a blockquote) that can potentially include other block-level tokens inside.

this.lexer.inline(string text, array tokens)

Parsing of inline-level tokens only occurs after all block-level tokens have been generated. This function adds text and tokens to a queue to be processed using inline-level tokenizers (including any inline-level extensions) at that later step. Tokens will be generated using the provided text, and any resulting tokens will be appended to the tokens array. Note that this function does **NOT** return anything since the inline processing cannot happen until the block-level processing is complete.

this.lexer.inlineTokens(string text, array tokens)

Sometimes an inline-level token contains further nested inline tokens (such as a

**strong**

token inside of a

### Heading

). This runs the inline tokenizer functions (including any inline-level extensions) on the provided text, and appends any resulting tokens onto the tokens array. The tokens array is also returned by the function.

renderer(object token)

A function that reads a token and returns the generated HTML output string.

渲染器函数可以访问 this 对象中的解析器，如果需要进一步解析标记的任何部分（例如任何子标记），则可以使用该解析器。可能有用的关键功能包括：

¥The renderer function has access to the parser in the this object, which can be used if any part of the token needs needs to be parsed further, such as any child tokens. The key functions that may be useful include:

this.parser.parse(array tokens)

Runs the block renderer functions (including any extensions) on the provided array of tokens, and returns the resulting HTML string output. This is used to generate the HTML from any child block-level tokens, for example if your extension is a "container"-type token (such as a blockquote) that can potentially include other block-level tokens inside.

this.parser.parseInline(array tokens)

Runs the inline renderer functions (including any extensions) on the provided array of tokens, and returns the resulting HTML string output. This is used to generate the HTML from any child inline-level tokens.

childTokens [optional]

An array of strings that match the names of any token parameters that should be traversed by the walkTokens functions. For instance, if you want to use a second custom parameter to contain child tokens in addition to tokens, it could be listed here. If childTokens is provided, the tokens array will not be walked by default unless it is also included in the childTokens array.

示例： 添加自定义语法以生成 <dl> 描述列表。

¥Example: Add a custom syntax to generate <dl> description lists.

const descriptionList = {
  name: 'descriptionList',
  level: 'block',                                     // Is this a block-level or inline-level tokenizer?
  start(src) { return src.match(/:[^:\n]/)?.index; }, // Hint to Marked.js to stop and check for a match
  tokenizer(src, tokens) {
    const rule = /^(?::[^:\n]+:[^:\n]*(?:\n|$))+/;    // Regex for the complete token, anchor to string start
    const match = rule.exec(src);
    if (match) {
      const token = {                                 // Token to generate
        type: 'descriptionList',                      // Should match "name" above
        raw: match[0],                                // Text to consume from the source
        text: match[0].trim(),                        // Additional custom properties
        tokens: []                                    // Array where child inline tokens will be generated
      };
      this.lexer.inline(token.text, token.tokens);    // Queue this data to be processed for inline tokens
      return token;
    }
  },
  renderer(token) {
    return `<dl>${this.parser.parseInline(token.tokens)}\n</dl>`; // parseInline to turn child tokens into HTML
  }
};

const description = {
  name: 'description',
  level: 'inline',                                 // Is this a block-level or inline-level tokenizer?
  start(src) { return src.match(/:/)?.index; },    // Hint to Marked.js to stop and check for a match
  tokenizer(src, tokens) {
    const rule = /^:([^:\n]+):([^:\n]*)(?:\n|$)/;  // Regex for the complete token, anchor to string start
    const match = rule.exec(src);
    if (match) {
      return {                                         // Token to generate
        type: 'description',                           // Should match "name" above
        raw: match[0],                                 // Text to consume from the source
        dt: this.lexer.inlineTokens(match[1].trim()),  // Additional custom properties, including
        dd: this.lexer.inlineTokens(match[2].trim())   //   any further-nested inline tokens
      };
    }
  },
  renderer(token) {
    return `\n<dt>${this.parser.parseInline(token.dt)}</dt><dd>${this.parser.parseInline(token.dd)}</dd>`;
  },
  childTokens: ['dt', 'dd'],                 // Any child tokens to be visited by walkTokens
};

function walkTokens(token) {                        // Post-processing on the completed token tree
  if (token.type === 'strong') {
    token.text += ' walked';
    token.tokens = this.Lexer.lexInline(token.text)
  }
}
marked.use({ extensions: [descriptionList, description], walkTokens });

// EQUIVALENT TO:

marked.use({ extensions: [descriptionList] });
marked.use({ extensions: [description]     });
marked.use({ walkTokens })

console.log(marked.parse('A Description List:\n'
                 + ':   Topic 1   :  Description 1\n'
                 + ': **Topic 2** : *Description 2*'));

输出

¥Output

<p>A Description List:</p>
<dl>
<dt>Topic 1</dt><dd>Description 1</dd>
<dt><strong>Topic 2 walked</strong></dt><dd><em>Description 2</em></dd>
</dl>

---

Async Marked : async

如果 async 选项为真，Marked 将返回一个 promise。async 选项将告诉 marked 在解析标记并返回 HTML 字符串之前等待任何 walkTokens 函数。

¥Marked will return a promise if the async option is true. The async option will tell marked to await any walkTokens functions before parsing the tokens and returning an HTML string.

简单示例：

¥Simple Example:

const walkTokens = async (token) => {
  if (token.type === 'link') {
    try {
      await fetch(token.href);
    } catch (ex) {
      token.title = 'invalid';
    }
  }
};

marked.use({ walkTokens, async: true });

const markdown = `
[valid link](https://example.com)

[invalid link](https://invalidurl.com)
`;

const html = await marked.parse(markdown);

自定义扩展示例：

¥Custom Extension Example:

const importUrl = {
  extensions: [{
    name: 'importUrl',
    level: 'block',
    start(src) { return src.indexOf('\n:'); },
    tokenizer(src) {
      const rule = /^:(https?:\/\/.+?):/;
      const match = rule.exec(src);
      if (match) {
        return {
          type: 'importUrl',
          raw: match[0],
          url: match[1],
          html: '' // will be replaced in walkTokens
        };
      }
    },
    renderer(token) {
      return token.html;
    }
  }],
  async: true, // needed to tell marked to return a promise
  async walkTokens(token) {
    if (token.type === 'importUrl') {
      const res = await fetch(token.url);
      token.html = await res.text();
    }
  }
};

marked.use(importUrl);

const markdown = `
# example.com

:https://example.com:
`;

const html = await marked.parse(markdown);

词法分析器

词法分析器获取 markdown 字符串并调用 tokenizer 函数。

¥The lexer takes a markdown string and calls the tokenizer functions.

解析器

解析器将标记作为输入并调用渲染器函数。

¥The parser takes tokens as input and calls the renderer functions.

访问词法分析器和解析器

如果你愿意，你还可以直接访问词法分析器和解析器。词法分析器和解析器选项与传递给 marked.setOptions() 的选项相同，只是它们必须是完整的选项对象，它们不会与当前或默认选项合并。

¥You also have direct access to the lexer and parser if you so desire. The lexer and parser options are the same as passed to marked.setOptions() except they have to be full options objects, they don't get merged with the current or default options.

const tokens = marked.lexer(markdown, options);
console.log(marked.parser(tokens, options));

const lexer = new marked.Lexer(options);
const tokens = lexer.lex(markdown);
console.log(tokens);
console.log(lexer.tokenizer.rules.block); // block level rules used
console.log(lexer.tokenizer.rules.inline); // inline level rules used
console.log(marked.Lexer.rules.block); // all block level rules
console.log(marked.Lexer.rules.inline); // all inline level rules

请注意，词法分析器可以以两种不同的方式使用：

¥Note that the lexer can be used in two different ways:

• marked.lexer()：此方法标记字符串并返回其标记。对 lexer() 的后续调用会忽略任何先前的调用。

  ¥marked.lexer(): this method tokenizes a string and returns its tokens. Subsequent calls to lexer() ignore any previous calls.

• new marked.Lexer().lex()：此实例标记字符串并返回其标记以及任何先前的标记。对 lex() 的后续调用会累积令牌。

  ¥new marked.Lexer().lex(): this instance tokenizes a string and returns its tokens along with any previous tokens. Subsequent calls to lex() accumulate tokens.

$ node
> require('marked').lexer('> I am using marked.')
[
  {
    type: "blockquote",
    raw: "> I am using marked.",
    tokens: [
      {
        type: "paragraph",
        raw: "I am using marked.",
        text: "I am using marked.",
        tokens: [
          {
            type: "text",
            raw: "I am using marked.",
            text: "I am using marked."
          }
        ]
      }
    ]
  },
  links: {}
]

Lexer 构建一个标记数组，这些标记将传递给解析器。解析器处理标记数组中的每个标记：

¥The Lexer builds an array of tokens, which will be passed to the Parser. The Parser processes each token in the token array:

import { marked } from 'marked';

const md = `
  # heading

  [link][1]

  [1]: #heading "heading"
`;

const tokens = marked.lexer(md);
console.log(tokens);

const html = marked.parser(tokens);
console.log(html);

[
  {
    type: "heading",
    raw: "  # heading\n\n",
    depth: 1,
    text: "heading",
    tokens: [
      {
        type: "text",
        raw: "heading",
        text: "heading"
      }
    ]
  },
  {
    type: "paragraph",
    raw: "  [link][1]",
    text: "  [link][1]",
    tokens: [
      {
        type: "text",
        raw: "  ",
        text: "  "
      },
      {
        type: "link",
        raw: "[link][1]",
        text: "link",
        href: "#heading",
        title: "heading",
        tokens: [
          {
            type: "text",
            raw: "link",
            text: "link"
          }
        ]
      }
    ]
  },
  {
    type: "space",
    raw: "\n\n"
  },
  links: {
    "1": {
      href: "#heading",
      title: "heading"
    }
  }
]
<h1 id="heading">heading</h1>
<p>  <a href="#heading" title="heading">link</a></p>