grain.experimental.ThreadPrefetchIterDataset#
- class grain.experimental.ThreadPrefetchIterDataset(parent, *, prefetch_buffer_size)#
Iterable dataset that uses a synchronized queue for prefetching.
This is a thread-based alternative to MultiprocessPrefetchIterDataset.
- Parameters:
parent (dataset.IterDataset[T])
prefetch_buffer_size (int)
- parent#
The parent dataset to prefetch from.
- prefetch_buffer_size#
The size of the prefetch buffer. Must be greater than or equal to 0. If 0, prefetching is disabled and this is a noop.
- __init__(parent, *, prefetch_buffer_size)#
- Parameters:
parent (IterDataset[T])
prefetch_buffer_size (int)
Methods
__init__(parent, *, prefetch_buffer_size)batch(batch_size, *[, drop_remainder, batch_fn])Returns a dataset of elements batched along a new first dimension.
filter(transform)Returns a dataset containing only the elements that match the filter.
map(transform)Returns a dataset containing the elements transformed by
transform.map_with_index(transform)Returns a dataset of the elements transformed by the
transform.mp_prefetch([options, worker_init_fn])Returns a dataset prefetching elements in multiple processes.
pipe(func, /, *args, **kwargs)Syntactic sugar for applying a callable to this dataset.
prefetch(multiprocessing_options)Deprecated, use
mp_prefetchinstead.random_map(transform, *[, seed])Returns a dataset containing the elements transformed by
transform.seed(seed)Returns a dataset that uses the seed for default seed generation.
Attributes
parents