latentmi.lmi

Classes

EarlyStopper

Early stopping that returns best weights

Functions

train(model, X_train, Y_train, X_test, Y_test[, ...])

training loop for LMI models

learn_representation(Xs, Ys, train_indices, test_indices)

train paired AE model and embed data

estimate(Xs, Ys[, regularizer, alpha, lam, N_dims, ...])

return pMIs (with NaNs for points not included in KSG estimate), embeddings, trained model

Module Contents

class latentmi.lmi.EarlyStopper(patience=1)[source]

Early stopping that returns best weights trying to replicate the Keras callback

early_stop(validation_loss, model)[source]
latentmi.lmi.train(model, X_train, Y_train, X_test, Y_test, batch_size=512, lr=0.0001, epochs=300, patience=30, quiet=True)[source]

training loop for LMI models

Parameters:

  • model – LMI model

  • X_train – train samples, shape (N_samples, N_features)

  • Y_train – train samples, shape (N_samples, N_features)

  • X_test – test samples, shape (N_samples, N_features)

  • Y_test – test samples, shape (N_samples, N_features)

  • batch_size – samples per batch, defaults to 512

  • lr – learning rate for Adam optimizer, defaults to 1e-4

  • epochs – max number of epochs, defaults to 300

  • patience – epochs without val. loss decline before early stopping, defaults to 300

  • quiet – suppress training progress display, defaults to True

latentmi.lmi.learn_representation(Xs, Ys, train_indices, test_indices, regularizer='models.AECross', alpha=1, lam=1, N_dims=8, batch_size=512, lr=0.0001, epochs=300, validation_split=0.3, patience=30, quiet=True, device='cpu')[source]

train paired AE model and embed data

Parameters:

  • Xs – input data, array with shape (N_samples, N_dims). ordering must align with Y.

  • Ys – input data, array with shape(N_samples, N_dims). ordering must align with X.

  • train_indices – list, indices of train samples

:param test_indices:list, indices of test samples

Parameters:

  • regularizer – type of regularization, defaults to AECross. can be changed to ‘models.AEMINE’ or ‘models.AEInfoNCE’ but not recommended.

  • alpha – self-reconstruction loss weight, defaults to 1

  • lam – cross-reconstruction regularization weight, defaults to 1

  • N_dims – dimensions in each latent representation, defaults to 8

  • batch_size – samples per batch, defaults to 512

  • lr – learning rate for Adam optimizer, defaults to 1e-4

  • epochs – max number of epochs, defaults to 300

  • validation_split – fraction train/test split, defaults to 0.5

  • patience – epochs without val. loss decline before early stopping, defaults to 300

  • quiet – suppress training progress display, defaults to True

latentmi.lmi.estimate(Xs, Ys, regularizer='models.AECross', alpha=1, lam=1, N_dims=8, validation_split=0.5, estimate_on_val=True, batch_size=512, lr=0.0001, epochs=300, patience=30, quiet=True, device=None)[source]

return pMIs (with NaNs for points not included in KSG estimate), embeddings, trained model

Parameters:

  • Xs – input data, array with shape (N_samples, N_dims). ordering must align with Y.

  • Ys – input data, array with shape(N_samples, N_dims). ordering must align with X.

  • regularizer – type of regularization, defaults to AECross. can be changed to ‘models.AEMINE’ or ‘models.AEInfoNCE’ but not recommended.

  • alpha – self-reconstruction loss weight, defaults to 1

  • lam – cross-reconstruction regularization weight, defaults to 1

  • N_dims – dimensions in each latent representation, defaults to 8

  • batch_size – samples per batch, defaults to 512

  • lr – learning rate for Adam optimizer, defaults to 1e-4

  • epochs – max number of epochs, defaults to 300

  • validation_split – fraction train/test split, defaults to 0.5

  • patience – epochs without val. loss decline before early stopping, defaults to 300

  • quiet – suppress training progress display, defaults to True

  • device – device for torch to train model, defaults to cuda if available, else cpu.

Returns:

array of pointwise mutual information estimates, order aligned with input. NaNs values not included in KSG estimate. mean of this array is MI estimate.

Returns:

tuple of arrays of coordinates of latent embeddings. first index is X embeddings, second index is Y.

Returns:

Pytorch object for trained representation learning model