The individual forms (words) of a corpus. The sum of the tokens is the size of the corpus. The term contrasts with type in order to distinguish how we are observing the form, whether as one instance in the corpus (token), or as combined instances relating to its frequency within a corpus (type).
Published by
ghoti 
Leave a Reply