BasicClasses.Token

Module BasicClasses :: Class Token

[show private | hide private]

Class Token

    object --+    
             |    
      BIObject --+
                 |
    object --+   |
             |   |
BIXMLWriteable --+
                 |
                Token

The representation of a single token as collection of subtokens.

Method Summary
	`__init__(self, oStack, attrs, **args)`
	`addSubToken(self, subToken)` Add a new `subToken` to the list of subtokens `self.subTokens`.
	`getText(self, sep)` The text of the token, assembled from the subtoken texts.
	`regenId(self)` Regenerate the `id` instance variables.
	`writeXMLNestedItems(self, out, indent)`
Inherited from BIXMLWriteable
	`writeXML(self, out, indent, **kwArgs)` Outputs the complete XML representation of the given object.
	`writeXMLClose(self, out, indent)` Prints the closing tag of the object: </tag>.
	`writeXMLOpen(self, out, indent, closing)` Writes opening tag into the XML output.
	`XMLAttrs(self)` Computes the string representation of the object's attributes, which are included into the XML tag.
Inherited from object
	`__delattr__(...)` x.__delattr__('name') <==> del x.name
	`__getattribute__(...)` x.__getattribute__('name') <==> x.name
	`__hash__(x)` x.__hash__() <==> hash(x)
	`__reduce__(...)` helper for pickle
	`__reduce_ex__(...)` helper for pickle
	`__repr__(x)` x.__repr__() <==> repr(x)
	`__setattr__(...)` x.__setattr__('name', value) <==> x.name = value
	`__str__(x)` x.__str__() <==> str(x)
Inherited from type
	`__new__(T, S, ...)` T.__new__(S, ...) -> a new object with type S, a subtype of T

Instance Variable Summary
	`charOffset`: An offset (zero-based) to the string `self.sentence.origText`, that is, the original untokenized text of the sentence.
string	`id`: A unique id assigned to the token in the corpus.
`sentence`	`sentence`: The sentence to which this token belongs.
list	`subTokens`: A list of `subtokens`.

Class Variable Summary
`list`	`persistentAttrs`: The list of attributes to be included in the XML output.
`str`	`XMLTag` = `'token'`

Method Details

addSubToken(self, subToken)

Add a new subToken to the list of subtokens self.subTokens. The subtokens must be added in the correct order.

Parameters:: subToken - The subtoken to be added.
(type=SubToken)

getText(self, sep='')

The text of the token, assembled from the subtoken texts.

Parameters:: sep - The separator to be used when joining the subtoken texts.

regenId(self)

Regenerate the id instance variables. Only to be used when the corpus data was changed (re-tokenized, etc...). This is not to be used unless you know exactly what you are doing.

Instance Variable Details

charOffset

An offset (zero-based) to the string self.sentence.origText, that is, the original untokenized text of the sentence. To be used when translating the corpus to some of the character-offset based formats.

id

A unique id assigned to the token in the corpus.

Type:: string

sentence

The sentence to which this token belongs.

Type:: sentence

subTokens

A list of subtokens. The subtokens are ordered in the same order as they appear in the XML file, which is the same order as they appear in the sentence. This order must be preserved.

Type:: list

Class Variable Details

persistentAttrs

The list of attributes to be included in the XML output. By default an empty list, that is, no attributes are included by default.

Type:: list
Value:: ['id', 'charOffset']

XMLTag

Type:: str
Value:: 'token'

Generated by Epydoc 2.0 on Tue Nov 21 15:32:22 2006

http://epydoc.sf.net