Wikipedia Easy Search

Redirect to:

Lexical analysis#Tokenization

This page is a redirect. The following categories are used to track and monitor this redirect:

From a merge: This is a redirect from a page that was merged into another page. This redirect was kept in order to preserve the edit history of this page after its content was merged into the content of the target page. Please do not remove the tag that generates this text (unless the need to recreate content on this page has been demonstrated) or delete this page.
- For redirects with substantive page histories that did not result from page merges use {{R with history}} instead.

To a section: This is a redirect from a topic that does not have its own page to a section of a page on the subject. For redirects to embedded anchors on a page, use {{R to anchor}} instead.

From a cross-project redirect: This is a redirect from a title linked to an item on Wikidata. The Wikidata item linked to this page is tokenization (Q2438971).
- Use this template only on hard redirects – for soft redirects use {{Soft redirect with Wikidata item}}.

When appropriate, protection levels are automatically sensed, described and categorized.