Labeling Models

Bidirectional LSTM Model

class kashgari.tasks.labeling.BiLSTM_Model(embedding=None, sequence_length=None, hyper_parameters=None)[source]

Bases: kashgari.tasks.labeling.abc_model.ABCLabelingModel

__init__(embedding=None, sequence_length=None, hyper_parameters=None)
Parameters
  • embedding (Optional[kashgari.embeddings.abc_embedding.ABCEmbedding]) – embedding object

  • sequence_length (Optional[int]) – target sequence length

  • hyper_parameters (Optional[Dict[str, Dict[str, Any]]]) – hyper_parameters to overwrite

build_model(x_data, y_data)

Build Model with x_data and y_data

This function will setup a CorpusGenerator,

then call ABCClassificationModel.build_model_gen() for preparing processor and model

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

Return type

None

Returns:

build_model_arc()[source]
Return type

None

build_model_generator(generators)
Parameters

generators (List[kashgari.generators.CorpusGenerator]) –

Return type

None

compile_model(loss=None, optimizer=None, metrics=None, **kwargs)

Configures the model for training. call tf.keras.Model.predict() to compile model with custom loss, optimizer and metrics

Examples

>>> model = BiLSTM_Model()
# Build model with corpus
>>> model.build_model(train_x, train_y)
# Compile model with custom loss, optimizer and metrics
>>> model.compile(loss='categorical_crossentropy', optimizer='rsm', metrics = ['accuracy'])
Parameters
  • loss (Optional[Any]) – name of objective function, objective function or tf.keras.losses.Loss instance.

  • optimizer (Optional[Any]) – name of optimizer or optimizer instance.

  • metrics (object) – List of metrics to be evaluated by the model during training and testing.

  • kwargs (Any) – additional params passed to tf.keras.Model.predict`().

Return type

None

classmethod default_hyper_parameters()[source]

The default hyper parameters of the model dict, all models must implement this function.

You could easily change model’s hyper-parameters.

For example, change the LSTM unit in BiLSTM_Model from 128 to 32.

>>> from kashgari.tasks.classification import BiLSTM_Model
>>> hyper = BiLSTM_Model.default_hyper_parameters()
>>> print(hyper)
{'layer_bi_lstm': {'units': 128, 'return_sequences': False}, 'layer_output': {}}
>>> hyper['layer_bi_lstm']['units'] = 32
>>> model = BiLSTM_Model(hyper_parameters=hyper)
Returns

hyper params dict

Return type

Dict[str, Dict[str, Any]]

evaluate(x_data, y_data, batch_size=32, digits=4, truncating=False)

Build a text report showing the main labeling metrics.

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

  • batch_size (int) –

  • digits (int) –

  • truncating (bool) –

Returns

A report dict

Return type

Dict

Example

>>> from kashgari.tasks.labeling import BiGRU_Model
>>> model = BiGRU_Model()
>>> model.fit(train_x, train_y, valid_x, valid_y)
>>> report = model.evaluate(test_x, test_y)
           precision    recall  f1-score   support

          ORG     0.0665    0.1108    0.0831       984
          LOC     0.1870    0.2086    0.1972      1951
          PER     0.1685    0.0882    0.1158       884

    micro avg     0.1384    0.1555    0.1465      3819
    macro avg     0.1516    0.1555    0.1490      3819

>>> print(report)
    {
     'f1-score': 0.14895159934887792,
     'precision': 0.1516294012813676,
     'recall': 0.15553809897879026,
     'support': 3819,
     'detail': {'LOC': {'f1-score': 0.19718992248062014,
                        'precision': 0.18695452457510336,
                        'recall': 0.20861096873398258,
                        'support': 1951},
                'ORG': {'f1-score': 0.08307926829268293,
                        'precision': 0.06646341463414634,
                        'recall': 0.11077235772357724,
                        'support': 984},
                'PER': {'f1-score': 0.11581291759465479,
                        'precision': 0.16846652267818574,
                        'recall': 0.08823529411764706,
                        'support': 884}},
    }
fit(x_train, y_train, x_validate=None, y_validate=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data set list.

Parameters
  • x_train (List[List[str]]) – Array of train feature data (if the model has a single input), or tuple of train feature data array (if the model has multiple inputs)

  • y_train (List[List[str]]) – Array of train label data

  • x_validate (Optional[List[List[str]]]) – Array of validation feature data (if the model has a single input), or tuple of validation feature data array (if the model has multiple inputs)

  • y_validate (Optional[List[List[str]]]) – Array of validation label data

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

fit_generator(train_sample_gen, valid_sample_gen=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data generator.

Data generator must be the subclass of CorpusGenerator

Parameters
  • train_sample_gen (kashgari.generators.CorpusGenerator) – train data generator.

  • valid_sample_gen (Optional[kashgari.generators.CorpusGenerator]) – valid data generator.

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

classmethod load_model(model_path)
Parameters

model_path (str) –

Return type

Union[ABCLabelingModel, ABCClassificationModel]

predict(x_data, *, batch_size=32, truncating=False, predict_kwargs=None)

Generates output predictions for the input samples.

Computation is done in batches.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

array(s) of predictions.

Return type

List[List[str]]

predict_entities(x_data, batch_size=32, join_chunk=' ', truncating=False, predict_kwargs=None)

Gets entities from sequence.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • join_chunk (str) – str or False,

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

list of entity.

Return type

list

save(model_path)
Parameters

model_path (str) –

Return type

str

to_dict()
Return type

Dict[str, Any]

Bidirectional GRU Model

class kashgari.tasks.labeling.BiGRU_Model(embedding=None, sequence_length=None, hyper_parameters=None)[source]

Bases: kashgari.tasks.labeling.abc_model.ABCLabelingModel

__init__(embedding=None, sequence_length=None, hyper_parameters=None)
Parameters
  • embedding (Optional[kashgari.embeddings.abc_embedding.ABCEmbedding]) – embedding object

  • sequence_length (Optional[int]) – target sequence length

  • hyper_parameters (Optional[Dict[str, Dict[str, Any]]]) – hyper_parameters to overwrite

build_model(x_data, y_data)

Build Model with x_data and y_data

This function will setup a CorpusGenerator,

then call ABCClassificationModel.build_model_gen() for preparing processor and model

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

Return type

None

Returns:

build_model_arc()[source]
Return type

None

build_model_generator(generators)
Parameters

generators (List[kashgari.generators.CorpusGenerator]) –

Return type

None

compile_model(loss=None, optimizer=None, metrics=None, **kwargs)

Configures the model for training. call tf.keras.Model.predict() to compile model with custom loss, optimizer and metrics

Examples

>>> model = BiLSTM_Model()
# Build model with corpus
>>> model.build_model(train_x, train_y)
# Compile model with custom loss, optimizer and metrics
>>> model.compile(loss='categorical_crossentropy', optimizer='rsm', metrics = ['accuracy'])
Parameters
  • loss (Optional[Any]) – name of objective function, objective function or tf.keras.losses.Loss instance.

  • optimizer (Optional[Any]) – name of optimizer or optimizer instance.

  • metrics (object) – List of metrics to be evaluated by the model during training and testing.

  • kwargs (Any) – additional params passed to tf.keras.Model.predict`().

Return type

None

classmethod default_hyper_parameters()[source]

The default hyper parameters of the model dict, all models must implement this function.

You could easily change model’s hyper-parameters.

For example, change the LSTM unit in BiLSTM_Model from 128 to 32.

>>> from kashgari.tasks.classification import BiLSTM_Model
>>> hyper = BiLSTM_Model.default_hyper_parameters()
>>> print(hyper)
{'layer_bi_lstm': {'units': 128, 'return_sequences': False}, 'layer_output': {}}
>>> hyper['layer_bi_lstm']['units'] = 32
>>> model = BiLSTM_Model(hyper_parameters=hyper)
Returns

hyper params dict

Return type

Dict[str, Dict[str, Any]]

evaluate(x_data, y_data, batch_size=32, digits=4, truncating=False)

Build a text report showing the main labeling metrics.

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

  • batch_size (int) –

  • digits (int) –

  • truncating (bool) –

Returns

A report dict

Return type

Dict

Example

>>> from kashgari.tasks.labeling import BiGRU_Model
>>> model = BiGRU_Model()
>>> model.fit(train_x, train_y, valid_x, valid_y)
>>> report = model.evaluate(test_x, test_y)
           precision    recall  f1-score   support

          ORG     0.0665    0.1108    0.0831       984
          LOC     0.1870    0.2086    0.1972      1951
          PER     0.1685    0.0882    0.1158       884

    micro avg     0.1384    0.1555    0.1465      3819
    macro avg     0.1516    0.1555    0.1490      3819

>>> print(report)
    {
     'f1-score': 0.14895159934887792,
     'precision': 0.1516294012813676,
     'recall': 0.15553809897879026,
     'support': 3819,
     'detail': {'LOC': {'f1-score': 0.19718992248062014,
                        'precision': 0.18695452457510336,
                        'recall': 0.20861096873398258,
                        'support': 1951},
                'ORG': {'f1-score': 0.08307926829268293,
                        'precision': 0.06646341463414634,
                        'recall': 0.11077235772357724,
                        'support': 984},
                'PER': {'f1-score': 0.11581291759465479,
                        'precision': 0.16846652267818574,
                        'recall': 0.08823529411764706,
                        'support': 884}},
    }
fit(x_train, y_train, x_validate=None, y_validate=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data set list.

Parameters
  • x_train (List[List[str]]) – Array of train feature data (if the model has a single input), or tuple of train feature data array (if the model has multiple inputs)

  • y_train (List[List[str]]) – Array of train label data

  • x_validate (Optional[List[List[str]]]) – Array of validation feature data (if the model has a single input), or tuple of validation feature data array (if the model has multiple inputs)

  • y_validate (Optional[List[List[str]]]) – Array of validation label data

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

fit_generator(train_sample_gen, valid_sample_gen=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data generator.

Data generator must be the subclass of CorpusGenerator

Parameters
  • train_sample_gen (kashgari.generators.CorpusGenerator) – train data generator.

  • valid_sample_gen (Optional[kashgari.generators.CorpusGenerator]) – valid data generator.

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

classmethod load_model(model_path)
Parameters

model_path (str) –

Return type

Union[ABCLabelingModel, ABCClassificationModel]

predict(x_data, *, batch_size=32, truncating=False, predict_kwargs=None)

Generates output predictions for the input samples.

Computation is done in batches.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

array(s) of predictions.

Return type

List[List[str]]

predict_entities(x_data, batch_size=32, join_chunk=' ', truncating=False, predict_kwargs=None)

Gets entities from sequence.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • join_chunk (str) – str or False,

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

list of entity.

Return type

list

save(model_path)
Parameters

model_path (str) –

Return type

str

to_dict()
Return type

Dict[str, Any]

Bidirectional LSTM CRF Model

class kashgari.tasks.labeling.BiLSTM_CRF_Model(embedding=None, sequence_length=None, hyper_parameters=None)[source]

Bases: kashgari.tasks.labeling.abc_model.ABCLabelingModel

__init__(embedding=None, sequence_length=None, hyper_parameters=None)
Parameters
  • embedding (Optional[kashgari.embeddings.abc_embedding.ABCEmbedding]) – embedding object

  • sequence_length (Optional[int]) – target sequence length

  • hyper_parameters (Optional[Dict[str, Dict[str, Any]]]) – hyper_parameters to overwrite

build_model(x_data, y_data)

Build Model with x_data and y_data

This function will setup a CorpusGenerator,

then call ABCClassificationModel.build_model_gen() for preparing processor and model

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

Return type

None

Returns:

build_model_arc()[source]
Return type

None

build_model_generator(generators)
Parameters

generators (List[kashgari.generators.CorpusGenerator]) –

Return type

None

compile_model(loss=None, optimizer=None, metrics=None, **kwargs)[source]

Configures the model for training. call tf.keras.Model.predict() to compile model with custom loss, optimizer and metrics

Examples

>>> model = BiLSTM_Model()
# Build model with corpus
>>> model.build_model(train_x, train_y)
# Compile model with custom loss, optimizer and metrics
>>> model.compile(loss='categorical_crossentropy', optimizer='rsm', metrics = ['accuracy'])
Parameters
  • loss (Optional[Any]) – name of objective function, objective function or tf.keras.losses.Loss instance.

  • optimizer (Optional[Any]) – name of optimizer or optimizer instance.

  • metrics (object) – List of metrics to be evaluated by the model during training and testing.

  • kwargs (Any) – additional params passed to tf.keras.Model.predict`().

Return type

None

classmethod default_hyper_parameters()[source]

The default hyper parameters of the model dict, all models must implement this function.

You could easily change model’s hyper-parameters.

For example, change the LSTM unit in BiLSTM_Model from 128 to 32.

>>> from kashgari.tasks.classification import BiLSTM_Model
>>> hyper = BiLSTM_Model.default_hyper_parameters()
>>> print(hyper)
{'layer_bi_lstm': {'units': 128, 'return_sequences': False}, 'layer_output': {}}
>>> hyper['layer_bi_lstm']['units'] = 32
>>> model = BiLSTM_Model(hyper_parameters=hyper)
Returns

hyper params dict

Return type

Dict[str, Dict[str, Any]]

evaluate(x_data, y_data, batch_size=32, digits=4, truncating=False)

Build a text report showing the main labeling metrics.

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

  • batch_size (int) –

  • digits (int) –

  • truncating (bool) –

Returns

A report dict

Return type

Dict

Example

>>> from kashgari.tasks.labeling import BiGRU_Model
>>> model = BiGRU_Model()
>>> model.fit(train_x, train_y, valid_x, valid_y)
>>> report = model.evaluate(test_x, test_y)
           precision    recall  f1-score   support

          ORG     0.0665    0.1108    0.0831       984
          LOC     0.1870    0.2086    0.1972      1951
          PER     0.1685    0.0882    0.1158       884

    micro avg     0.1384    0.1555    0.1465      3819
    macro avg     0.1516    0.1555    0.1490      3819

>>> print(report)
    {
     'f1-score': 0.14895159934887792,
     'precision': 0.1516294012813676,
     'recall': 0.15553809897879026,
     'support': 3819,
     'detail': {'LOC': {'f1-score': 0.19718992248062014,
                        'precision': 0.18695452457510336,
                        'recall': 0.20861096873398258,
                        'support': 1951},
                'ORG': {'f1-score': 0.08307926829268293,
                        'precision': 0.06646341463414634,
                        'recall': 0.11077235772357724,
                        'support': 984},
                'PER': {'f1-score': 0.11581291759465479,
                        'precision': 0.16846652267818574,
                        'recall': 0.08823529411764706,
                        'support': 884}},
    }
fit(x_train, y_train, x_validate=None, y_validate=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data set list.

Parameters
  • x_train (List[List[str]]) – Array of train feature data (if the model has a single input), or tuple of train feature data array (if the model has multiple inputs)

  • y_train (List[List[str]]) – Array of train label data

  • x_validate (Optional[List[List[str]]]) – Array of validation feature data (if the model has a single input), or tuple of validation feature data array (if the model has multiple inputs)

  • y_validate (Optional[List[List[str]]]) – Array of validation label data

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

fit_generator(train_sample_gen, valid_sample_gen=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data generator.

Data generator must be the subclass of CorpusGenerator

Parameters
  • train_sample_gen (kashgari.generators.CorpusGenerator) – train data generator.

  • valid_sample_gen (Optional[kashgari.generators.CorpusGenerator]) – valid data generator.

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

classmethod load_model(model_path)
Parameters

model_path (str) –

Return type

Union[ABCLabelingModel, ABCClassificationModel]

predict(x_data, *, batch_size=32, truncating=False, predict_kwargs=None)

Generates output predictions for the input samples.

Computation is done in batches.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

array(s) of predictions.

Return type

List[List[str]]

predict_entities(x_data, batch_size=32, join_chunk=' ', truncating=False, predict_kwargs=None)

Gets entities from sequence.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • join_chunk (str) – str or False,

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

list of entity.

Return type

list

save(model_path)
Parameters

model_path (str) –

Return type

str

to_dict()
Return type

Dict[str, Any]

Bidirectional GRU CRF Model

class kashgari.tasks.labeling.BiGRU_CRF_Model(embedding=None, sequence_length=None, hyper_parameters=None)[source]

Bases: kashgari.tasks.labeling.abc_model.ABCLabelingModel

__init__(embedding=None, sequence_length=None, hyper_parameters=None)
Parameters
  • embedding (Optional[kashgari.embeddings.abc_embedding.ABCEmbedding]) – embedding object

  • sequence_length (Optional[int]) – target sequence length

  • hyper_parameters (Optional[Dict[str, Dict[str, Any]]]) – hyper_parameters to overwrite

build_model(x_data, y_data)

Build Model with x_data and y_data

This function will setup a CorpusGenerator,

then call ABCClassificationModel.build_model_gen() for preparing processor and model

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

Return type

None

Returns:

build_model_arc()[source]
Return type

None

build_model_generator(generators)
Parameters

generators (List[kashgari.generators.CorpusGenerator]) –

Return type

None

compile_model(loss=None, optimizer=None, metrics=None, **kwargs)[source]

Configures the model for training. call tf.keras.Model.predict() to compile model with custom loss, optimizer and metrics

Examples

>>> model = BiLSTM_Model()
# Build model with corpus
>>> model.build_model(train_x, train_y)
# Compile model with custom loss, optimizer and metrics
>>> model.compile(loss='categorical_crossentropy', optimizer='rsm', metrics = ['accuracy'])
Parameters
  • loss (Optional[Any]) – name of objective function, objective function or tf.keras.losses.Loss instance.

  • optimizer (Optional[Any]) – name of optimizer or optimizer instance.

  • metrics (object) – List of metrics to be evaluated by the model during training and testing.

  • kwargs (Any) – additional params passed to tf.keras.Model.predict`().

Return type

None

classmethod default_hyper_parameters()[source]

The default hyper parameters of the model dict, all models must implement this function.

You could easily change model’s hyper-parameters.

For example, change the LSTM unit in BiLSTM_Model from 128 to 32.

>>> from kashgari.tasks.classification import BiLSTM_Model
>>> hyper = BiLSTM_Model.default_hyper_parameters()
>>> print(hyper)
{'layer_bi_lstm': {'units': 128, 'return_sequences': False}, 'layer_output': {}}
>>> hyper['layer_bi_lstm']['units'] = 32
>>> model = BiLSTM_Model(hyper_parameters=hyper)
Returns

hyper params dict

Return type

Dict[str, Dict[str, Any]]

evaluate(x_data, y_data, batch_size=32, digits=4, truncating=False)

Build a text report showing the main labeling metrics.

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

  • batch_size (int) –

  • digits (int) –

  • truncating (bool) –

Returns

A report dict

Return type

Dict

Example

>>> from kashgari.tasks.labeling import BiGRU_Model
>>> model = BiGRU_Model()
>>> model.fit(train_x, train_y, valid_x, valid_y)
>>> report = model.evaluate(test_x, test_y)
           precision    recall  f1-score   support

          ORG     0.0665    0.1108    0.0831       984
          LOC     0.1870    0.2086    0.1972      1951
          PER     0.1685    0.0882    0.1158       884

    micro avg     0.1384    0.1555    0.1465      3819
    macro avg     0.1516    0.1555    0.1490      3819

>>> print(report)
    {
     'f1-score': 0.14895159934887792,
     'precision': 0.1516294012813676,
     'recall': 0.15553809897879026,
     'support': 3819,
     'detail': {'LOC': {'f1-score': 0.19718992248062014,
                        'precision': 0.18695452457510336,
                        'recall': 0.20861096873398258,
                        'support': 1951},
                'ORG': {'f1-score': 0.08307926829268293,
                        'precision': 0.06646341463414634,
                        'recall': 0.11077235772357724,
                        'support': 984},
                'PER': {'f1-score': 0.11581291759465479,
                        'precision': 0.16846652267818574,
                        'recall': 0.08823529411764706,
                        'support': 884}},
    }
fit(x_train, y_train, x_validate=None, y_validate=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data set list.

Parameters
  • x_train (List[List[str]]) – Array of train feature data (if the model has a single input), or tuple of train feature data array (if the model has multiple inputs)

  • y_train (List[List[str]]) – Array of train label data

  • x_validate (Optional[List[List[str]]]) – Array of validation feature data (if the model has a single input), or tuple of validation feature data array (if the model has multiple inputs)

  • y_validate (Optional[List[List[str]]]) – Array of validation label data

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

fit_generator(train_sample_gen, valid_sample_gen=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data generator.

Data generator must be the subclass of CorpusGenerator

Parameters
  • train_sample_gen (kashgari.generators.CorpusGenerator) – train data generator.

  • valid_sample_gen (Optional[kashgari.generators.CorpusGenerator]) – valid data generator.

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

classmethod load_model(model_path)
Parameters

model_path (str) –

Return type

Union[ABCLabelingModel, ABCClassificationModel]

predict(x_data, *, batch_size=32, truncating=False, predict_kwargs=None)

Generates output predictions for the input samples.

Computation is done in batches.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

array(s) of predictions.

Return type

List[List[str]]

predict_entities(x_data, batch_size=32, join_chunk=' ', truncating=False, predict_kwargs=None)

Gets entities from sequence.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • join_chunk (str) – str or False,

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

list of entity.

Return type

list

save(model_path)
Parameters

model_path (str) –

Return type

str

to_dict()
Return type

Dict[str, Any]

Bidirectional CNN LSTM Model

class kashgari.tasks.labeling.CNN_LSTM_Model(embedding=None, sequence_length=None, hyper_parameters=None)[source]

Bases: kashgari.tasks.labeling.abc_model.ABCLabelingModel

__init__(embedding=None, sequence_length=None, hyper_parameters=None)
Parameters
  • embedding (Optional[kashgari.embeddings.abc_embedding.ABCEmbedding]) – embedding object

  • sequence_length (Optional[int]) – target sequence length

  • hyper_parameters (Optional[Dict[str, Dict[str, Any]]]) – hyper_parameters to overwrite

build_model(x_data, y_data)

Build Model with x_data and y_data

This function will setup a CorpusGenerator,

then call ABCClassificationModel.build_model_gen() for preparing processor and model

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

Return type

None

Returns:

build_model_arc()[source]
Return type

None

build_model_generator(generators)
Parameters

generators (List[kashgari.generators.CorpusGenerator]) –

Return type

None

compile_model(loss=None, optimizer=None, metrics=None, **kwargs)

Configures the model for training. call tf.keras.Model.predict() to compile model with custom loss, optimizer and metrics

Examples

>>> model = BiLSTM_Model()
# Build model with corpus
>>> model.build_model(train_x, train_y)
# Compile model with custom loss, optimizer and metrics
>>> model.compile(loss='categorical_crossentropy', optimizer='rsm', metrics = ['accuracy'])
Parameters
  • loss (Optional[Any]) – name of objective function, objective function or tf.keras.losses.Loss instance.

  • optimizer (Optional[Any]) – name of optimizer or optimizer instance.

  • metrics (object) – List of metrics to be evaluated by the model during training and testing.

  • kwargs (Any) – additional params passed to tf.keras.Model.predict`().

Return type

None

classmethod default_hyper_parameters()[source]

The default hyper parameters of the model dict, all models must implement this function.

You could easily change model’s hyper-parameters.

For example, change the LSTM unit in BiLSTM_Model from 128 to 32.

>>> from kashgari.tasks.classification import BiLSTM_Model
>>> hyper = BiLSTM_Model.default_hyper_parameters()
>>> print(hyper)
{'layer_bi_lstm': {'units': 128, 'return_sequences': False}, 'layer_output': {}}
>>> hyper['layer_bi_lstm']['units'] = 32
>>> model = BiLSTM_Model(hyper_parameters=hyper)
Returns

hyper params dict

Return type

Dict[str, Dict[str, Any]]

evaluate(x_data, y_data, batch_size=32, digits=4, truncating=False)

Build a text report showing the main labeling metrics.

Parameters
  • x_data (List[List[str]]) –

  • y_data (List[List[str]]) –

  • batch_size (int) –

  • digits (int) –

  • truncating (bool) –

Returns

A report dict

Return type

Dict

Example

>>> from kashgari.tasks.labeling import BiGRU_Model
>>> model = BiGRU_Model()
>>> model.fit(train_x, train_y, valid_x, valid_y)
>>> report = model.evaluate(test_x, test_y)
           precision    recall  f1-score   support

          ORG     0.0665    0.1108    0.0831       984
          LOC     0.1870    0.2086    0.1972      1951
          PER     0.1685    0.0882    0.1158       884

    micro avg     0.1384    0.1555    0.1465      3819
    macro avg     0.1516    0.1555    0.1490      3819

>>> print(report)
    {
     'f1-score': 0.14895159934887792,
     'precision': 0.1516294012813676,
     'recall': 0.15553809897879026,
     'support': 3819,
     'detail': {'LOC': {'f1-score': 0.19718992248062014,
                        'precision': 0.18695452457510336,
                        'recall': 0.20861096873398258,
                        'support': 1951},
                'ORG': {'f1-score': 0.08307926829268293,
                        'precision': 0.06646341463414634,
                        'recall': 0.11077235772357724,
                        'support': 984},
                'PER': {'f1-score': 0.11581291759465479,
                        'precision': 0.16846652267818574,
                        'recall': 0.08823529411764706,
                        'support': 884}},
    }
fit(x_train, y_train, x_validate=None, y_validate=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data set list.

Parameters
  • x_train (List[List[str]]) – Array of train feature data (if the model has a single input), or tuple of train feature data array (if the model has multiple inputs)

  • y_train (List[List[str]]) – Array of train label data

  • x_validate (Optional[List[List[str]]]) – Array of validation feature data (if the model has a single input), or tuple of validation feature data array (if the model has multiple inputs)

  • y_validate (Optional[List[List[str]]]) – Array of validation label data

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

fit_generator(train_sample_gen, valid_sample_gen=None, batch_size=64, epochs=5, callbacks=None, fit_kwargs=None)

Trains the model for a given number of epochs with given data generator.

Data generator must be the subclass of CorpusGenerator

Parameters
  • train_sample_gen (kashgari.generators.CorpusGenerator) – train data generator.

  • valid_sample_gen (Optional[kashgari.generators.CorpusGenerator]) – valid data generator.

  • batch_size (int) – Number of samples per gradient update, default to 64.

  • epochs (int) – Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.

  • callbacks (Optional[List[tensorflow.python.keras.callbacks.Callback]]) – List of tf.keras.callbacks.Callback instances. List of callbacks to apply during training. See tf.keras.callbacks.

  • fit_kwargs (Optional[Dict]) – fit_kwargs: additional arguments passed to tf.keras.Model.fit()

Returns

A tf.keras.callback.History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Return type

tensorflow.python.keras.callbacks.History

classmethod load_model(model_path)
Parameters

model_path (str) –

Return type

Union[ABCLabelingModel, ABCClassificationModel]

predict(x_data, *, batch_size=32, truncating=False, predict_kwargs=None)

Generates output predictions for the input samples.

Computation is done in batches.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

array(s) of predictions.

Return type

List[List[str]]

predict_entities(x_data, batch_size=32, join_chunk=' ', truncating=False, predict_kwargs=None)

Gets entities from sequence.

Parameters
  • x_data (List[List[str]]) – The input data, as a Numpy array (or list of Numpy arrays if the model has multiple inputs).

  • batch_size (int) – Integer. If unspecified, it will default to 32.

  • truncating (bool) – remove values from sequences larger than model.embedding.sequence_length

  • join_chunk (str) – str or False,

  • predict_kwargs (Optional[Dict]) – arguments passed to tf.keras.Model.predict()

Returns

list of entity.

Return type

list

save(model_path)
Parameters

model_path (str) –

Return type

str

to_dict()
Return type

Dict[str, Any]