trw.train.sequence_sub_batch¶
Module Contents¶
Classes¶
This sequence will split batches in smaller batches if the underlying sequence batch is too large. |
- class trw.train.sequence_sub_batch.SequenceSubBatch(source_split, batch_size, discard_batch_not_full=False)¶
Bases:
trw.train.sequence.Sequence,trw.train.sequence.SequenceIteratorThis sequence will split batches in smaller batches if the underlying sequence batch is too large.
This sequence can be useful to manage very large tensors. Indeed, this class avoids concatenating tensors (as opposed to in
trw.train.SequenceReBatch). Since this operation can be costly as the tensors must be reallocated. In this case, it may be faster to work on a smaller batch by avoiding the concatenation cost.- subsample(self, nb_samples)¶
Sub-sample a sequence to a fixed number of samples.
The purpose is to obtain a smaller sequence, this is particularly useful for the export of augmentations, samples.
- Parameters
nb_samples – the number of samples desired in the original sequence
- Returns
a subsampled Sequence
- subsample_uids(self, uids, uids_name, new_sampler=None)¶
Sub-sample a sequence to samples with specified UIDs.
- Parameters
uids (list) – the uids. If new_sampler keeps the ordering, then the samples of the resampled sequence should follow uids ordering
uids_name (str) – the name of the UIDs
new_sampler (Sampler) – the sampler to be used for the subsampler sequence. If None, re-use the existing
- Returns
a subsampled Sequence
- __next__(self)¶
- Returns
The next batch of data
- __iter__(self)¶
- Returns
An iterator of batches
- close(self)¶
Special method to close and clean the resources of the sequence