util package

Subpackages

• util.data package
  • Submodules
  • util.data.dataset_analytics module
    • Split folders
    • Classes folders
  • util.data.dataset_bidimensional module
  • util.data.dataset_integrity module
  • util.data.dataset_splitter module
  • util.data.get_a_dataset module
  • util.data.remove_whitespace module
  • util.data.shuffle_labels module
  • Module contents
• util.evaluation package
  • Subpackages
    • util.evaluation.metrics package
      • Submodules
      • util.evaluation.metrics.accuracy module
      • util.evaluation.metrics.apk module
      • util.evaluation.metrics.test_accuracy module
      • util.evaluation.metrics.test_apk module
      • Module contents
  • Submodules
  • util.evaluation.similarity_sort_embedding module
  • Module contents
• util.visualization package
  • Submodules
  • util.visualization.confusion_matrix_heatmap module
  • util.visualization.decision_boundaries module
  • util.visualization.embedding module
  • util.visualization.mean_std_plot module
  • util.visualization.visualize_activations module
  • Module contents

Submodules

util.misc module

General purpose utility functions.

class util.misc.AverageMeter

Bases: object

Computes and stores the average and current value

From https://github.com/pytorch/examples/blob/master/imagenet/main.py

reset()

update(val, n=1)

util.misc.HSVToRGB(h, s, v)

util.misc.adjust_learning_rate(lr, optimizer, epoch, decay_lr_epochs, **kwargs)

Sets the learning rate to the initial LR decayed by 10 every N epochs.

Adapted from https://github.com/pytorch/examples/blob/master/imagenet/main.py

Parameters

• lr (float) – Learning rate.

• optimizer (torch.optim object) – The optimizer used for training the network.

• epoch (int) – Current training epoch.

• decay_lr_epochs (int) – Change the learning rate every N epochs.

Returns

Return type

None

util.misc.checkpoint(epoch, new_value, best_value, model, optimizer, log_dir, invert_best=False, checkpoint_all_epochs=False, **kwargs)

Saves the current training checkpoint and the best valued checkpoint to file.

Parameters

• epoch (int) – Current epoch, for logging purpose only.

• new_value (float) – Current value achieved by the model at this epoch. To be compared with ‘best_value’.

• best_value (float) – Best value ever obtained (so the last checkpointed model). To be compared with ‘new_value’.

• model (torch.nn.module object) – The model we are checkpointing, this can be saved on file if necessary.

• optimizer – The optimizer that is being used to train this model. It is necessary if we were to resume the training from a checkpoint.

• log_dir (str) – Output folder where to put the model.

• invert_best (bool) – Changes the scale such that smaller values are better than bigger values (useful when metric evaluted is error rate)

• checkpoint_all_epochs (bool) – If enabled, save checkpoint after every epoch.

• kwargs (dict) – Any additional arguments.

Returns

best_value – Best value ever obtained.

Return type

float

util.misc.get_all_files_in_folders_and_subfolders(root_dir=None)

Get all the files in a folder and sub-folders.

Parameters

root_dir (str) – All files in this directory and it’s sub-folders will be returned by this method.

Returns

paths – List of paths to all files in this folder and it’s subfolders.

Return type

list of str

util.misc.get_distinct_colors(n)

util.misc.has_extension(filename, extensions)

Checks if a file is an allowed extension.

Adapted from https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py.

Parameters

• filename (string) – path to a file

• extensions (list) – extensions to match against

Returns

True if the filename ends with one of given extensions, false otherwise.

Return type

bool

util.misc.int_to_one_hot(x, n_classes)

Read out class encoding from blue channel bit-encoding (1 to [0,0,0,1] -> length determined by the number of classes)

Parameters

• x (int) – (pixel value of Blue channel from RGB image)

• n_classes (int) – number of class labels

Returns

(multi) one-hot encoded list for integer

Return type

list

util.misc.load_numpy_image(dest_filename)

util.misc.make_colour_legend_image(img_name, colour_encoding)

util.misc.make_folder_if_not_exists(path)

util.misc.multi_label_img_to_multi_hot(np_array)

TODO: There must be a faster way of doing this + ajust to correct input format (see gt_tensor_to_one_hot) Convert ground truth label image to multi-one-hot encoded matrix of size image height x image width x #classes

Parameters

np_array (numpy array) – RGB image [W x H x C]

Returns

sparse one-hot encoded multi-class matrix, where #C is the number of classes

Return type

numpy array of size [#C x W x H]

util.misc.multi_one_hot_to_output(matrix)

This function converts the multi-one-hot encoded matrix to an image like it was provided in the ground truth

Parameters

tensor of size [#C x W x H] – sparse one-hot encoded multi-class matrix, where #C is the number of classes

Returns

np_array – RGB image [C x W x H]

Return type

numpy array

util.misc.pil_loader(path)

util.misc.save_image_and_log_to_tensorboard(writer=None, tag=None, image=None, global_step=None)

Utility function to save image in the output folder and also log it to Tensorboard.

Parameters

• writer (tensorboardX.writer.SummaryWriter object) – The writer object for Tensorboard

• tag (str) – Name of the image.

• image (ndarray [W x H x C]) – Image to be saved and logged to Tensorboard.

• global_step (int) – Epoch/Mini-batch counter.

Returns

Return type

None

util.misc.save_numpy_image(dest_filename, image)

util.misc.tensor_to_image(image)

Tries to reshape, convert and do operations necessary to bring the image in a format friendly to be saved and logged to Tensorboard by save_image_and_log_to_tensorboard()

Parameters

image (?) – Image to be converted

Returns

image – Image, as format friendly to be saved and logged to Tensorboard.

Return type

ndarray [W x H x C]

util.misc.to_capital_camel_case(s)

Converts a string to camel case.

Parameters

s (str) – Input string.

Returns

Input string s converted to camel case.

Return type

str

Module contents