Public API: grain package

Subpackages

• grain.checkpoint module
• grain.constants module
• grain.experimental module
• grain.multiprocessing module
• grain.samplers module
• grain.sharding module
• grain.sources module
• grain.transforms module

Flexible low-level pipelines

MapDataset([parents])	Represents a dataset with transformations that support random access.
IterDataset([parents])	Represents a dataset with transformations that support Iterable interface.
DatasetIterator([parents])	IterDataset iterator.
ReadOptions([num_threads, prefetch_buffer_size])	Options for reading data from the DataSource.

Simple high-level pipelines

load(source, *[, num_epochs, shuffle, seed, ...])	Convenient method for simple pipelines on top of a data source.
DataLoader(*, data_source, sampler[, ...])	DataLoader loads and transforms input data.
DataLoaderIterator(data_loader, state)	DataLoader iterator providing get/set state functionality.
Record(metadata, data)
RecordMetadata(index[, record_key, rng])	RecordMetadata contains metadata about indidivual records.