article package

Submodules

Article implementation.

class core_utils.ctlr.article.article.Article(url, article_id)

Bases: object

Article class implementation.

Parameters:

• url (str | None) –

• article_id (int) –

__init__(url, article_id)

Initialize an instance of Article.

Parameters:

• url (str | None) – Site url

• article_id (int) – Article id

Return type:

None

_conllu_info: str

ConLLU information

_date_to_text()

Convert datetime object to text.

Returns:

Datetime object

Return type:

str

date: datetime | None

A date

get_cleaned_text()

Get cleaned text.

Returns:

Cleaned text.

Return type:

str

get_conllu_info()

Get the sentences from ConlluArticle.

Returns:

Sentences from ConlluArticle

Return type:

str

get_conllu_text(include_morphological_tags)

Get the text in the CONLL-U format.

Parameters:

include_morphological_tags (bool) – Flag to include morphological information

Returns:

A text in the CONLL-U format

Return type:

str

get_file_path(kind)

Get a proper filepath for an Article instance.

Parameters:

kind (ArtifactType) – A variant of a file

Returns:

Path to Article instance

Return type:

pathlib.Path

get_meta()

Get all meta params.

Returns:

Meta params

Return type:

dict

get_meta_file_path()

Get path for requested article’s meta info.

Returns:

Path to requested article’s meta info

Return type:

pathlib.Path

get_pos_freq()

Get a pos_frequency parameter.

Returns:

POS frequency

Return type:

dict

get_raw_text()

Get raw text from the article.

Returns:

Raw text from the article

Return type:

str

get_raw_text_path()

Get path for requested raw article.

Returns:

Path to requested raw article

Return type:

pathlib.Path

set_conllu_info(info)

Set the conllu_sentences_attribute.

Parameters:

info (str) – CONLL-U sentences

Return type:

None

set_patterns_info(pattern_matches)

Set patterns frequencies attribute.

Parameters:

pattern_matches (dict) – Syntactic patterns

Return type:

None

set_pos_info(pos_freq)

Set POS frequencies attribute.

Parameters:

pos_freq (dict) – POS frequencies

Return type:

None

class core_utils.ctlr.article.article.ArtifactType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Types of artifacts that can be created by text processing pipelines.

CLEANED = 'cleaned'

UDPIPE_CONLLU = 'udpipe'

core_utils.ctlr.article.article.date_from_meta(date_txt)

Convert text date to datetime object.

Parameters:

date_txt (str) – Date in text format

Returns:

Datetime object

Return type:

datetime.datetime

core_utils.ctlr.article.article.get_article_id_from_filepath(path)

Extract the article id from its path.

Parameters:

path (pathlib.Path) – Path to article

Returns:

Article id

Return type:

int

core_utils.ctlr.article.article.split_by_sentence(text)

Splits the given text by sentence separators.

Parameters:

text (str) – raw text to split

Returns:

List of sentences

Return type:

list[str]

I/O operations for Article.

core_utils.ctlr.article.io.from_meta(path, article=None)

Load meta.json file into the Article abstraction.

Parameters:

• path (pathlib.Path | str) – Path to meta info

• article (Article | None) – Article instance. Defaults to None.

Returns:

Article instance

Return type:

Article

core_utils.ctlr.article.io.from_raw(path, article=None)

Load raw text and create an Article with it.

Parameters:

• path (pathlib.Path | str) – Path to article raw text

• article (Article | None) – Article instance. Defaults to None.

Returns:

Article instance

Return type:

Article

core_utils.ctlr.article.io.to_cleaned(article)

Save cleaned text.

Parameters:

article (Article) – Article instance

Return type:

None

core_utils.ctlr.article.io.to_meta(article)

Save metafile.

Parameters:

article (Article) – Article instance

Return type:

None

core_utils.ctlr.article.io.to_raw(article)

Save raw text.

Parameters:

article (Article) – Article instance

Return type:

None